This book follows the life of Demis Hassabis, the founder of DeepMind, from his upbringing in London and studies at Cambridge to the founding of DeepMind, his move into Silicon Valley, meetings with prominent figures such as Elon Musk and Mark Zuckerberg, and the company’s eventual acquisition by Google.

It also traces the evolution of artificial intelligence over the years, from early symbolic and classical machine-learning approaches to deep learning, reinforcement learning, and modern neural networks.

Concepts

Emergence

How a complex entity has property and behaviors that their components don't on their own, and emerge only when they interact together

“It’s not statistics!” Sutskever objected. Then he backed up and tried a better argument. Given the long road that scientists had traveled, featuring word-embeddings, tokenization, attention, and self-supervised learning, the parrot insult got to him. “It is statistics! But what are statistics? “In order to understand those statistics, to compress them, you need to understand what it is about the world that creates those statistics. What is it about people that creates their behaviors? “Well, they have thoughts, they have feelings and ideas, and they behave in certain ways. All of those things can be deduced through next-token prediction.” If computers could understand how humans choose the next word in a sentence, they would have grasped the inner workings of the mind, Sutskever was saying.

“The key in all of this is, where do emergent properties come from? This is the big question in life, in physics, in everything. “You have components, and they don’t have a certain property. And then somehow when you put these components together, the property emerges. “For example, you put some chemicals together and you get life. You put some neurons together and you get intelligence. You put some humans together and you get an economy." - Hassabis

“Let’s say you have a family of amino acid sequences,” Jumper explained. “You mask certain amino acids in the sequences and you ask the network to guess what’s been hidden. If the network learns to do that well, it will also learn a lot of other things along the way—evolution, physics, geometry. It will understand deep truths about proteins. It might even predict protein structure.” Jumper sounded like Ilya Sutskever. A neural network might complete a narrow task, like guessing a concealed token. But something broader would emerge: intelligence."

Symbolic Logic

In the 1950s, scientists thought machines could be intelligent by following symbolic logic. This led to the development of symbolic artificial intelligence.

Programmers chose symbols to represent concepts from the real world: digits, words, physical objects. They supplied the computer with information about these symbols, then added instructions on logical rules—the definitions of “and,” “or,” “not and,” and so on. If the computer was told that the first symbol was green and the second one was blue, it could accurately deduce that they were not both the same color. If it was told that humans are mortal, and that Socrates was human, then it could deduce that Socrates was mortal. The idea was to transform all real-world phenomena into quasi-mathematical syllogisms. If a is b, and c is a, then c is b, also.

The essence of intelligence is the ability to respond flexibly to complex situations. But symbolic programming involves feeding inflexible rules into inflexible machines; inflexibility piled on inflexibility would never conjure flexible intelligence. To rise above this contradiction, future scientists would have to invent a new kind of machine: a machine that discovered the patterns in a near infinity of data.

Deep Learning and Reenforcement Learning

Enterpeneurship

Induction vs Deduction