A book that tries to address the emergence and implications of artificial intelligence.

(I should preface what I’m going to say by noting Brand’s enormous contributions to AI research and teaching. He knows massively more about these systems than I do.)

Computer scientists often avoid the term “AI” because it’s so broad and encompasses a huge range of approaches, from constraint-solving and pattern-matching to speech recognition and the sorts of “chatbot” technology that is the common face of modern AI – technically referred to as Large Language Models or LLMs, that have been trained on text scraped from the internet and can perform impressive feats of conversation.

There’s a problem when discussing AI that you have to be very careful about, and that’s the tendency to anthropomorphise. It’s almost impossble not to use terms that imply agency: “think”, “understand”, “learn”, “know”, “decide”, and so on. AI researchers use these words with very technical senses that are analogous to their usual meanings – but that aren’t the same, and that’s where things get misleading. When a person “summarises” some information, we mean that they pick out what they think is most important from the text; when an LLM does the same thing, it picks out the words that are statistically most clustered within the text and its training data. Are these two processes the same? They certainly don’t sound the same, and they often produce radically different “summaries”, because the human summariser is working from a far richer knowledge base and using different means to assess importance, often including what they know about the intended audience. It’s doing this well that’s usually taken as one of the hallmarks of intelligence. These are statistical techniques that cleave to the mean, which implies they will always by design produce middling answers. An amazingly insightful and creative answer is by definition unlikely, and so will be avoided by design.

There are other problems too. Machine learning, the science that underlies LLMs, is usually based on training from huge volumes of data. This permits impressive feats, such as being able to spot anomalies in cancer images better (in some cases) and more consistently than trained doctors. But that’s a weakness too: the idea that the future will be like the past, meaning that anything not in the training set runs the risk of being ignored as noise. The LLMs’ training data from the internet is text, not validated knowledge in any sense. And remember that repetition increases likelihood for an LLM, including repeatition of nonsense.

This lies at the heart of the problem of hallucination, where AIs confidently produce startlingly incorrect text. Why is this? Because text is all they have to work with: there is no model of the world as it is, and therefore no ability to correct. The most modern LLMs are now attempting to correct this, so far with little success.

So are we seeing intelligence from LLMs? We’re seeing something that presents like some parts of such intelligence. From this the industry has extrapolated that we are, for example, about to see models demonstrating “PhD level” ability – “soon”, but there never seems to be a demonstration. One characteristic of LLMs is that they don’t spontaneously do anything: they wait until prompted, and then reply. Does that sound like intelligent behaviour? – no planning, no forethought, no imagination? I work with a lot of PhD-level people, and none of them behave this way.

Some will object that there’s lots going on in industrial labs that we don’t see. Quite possibly. Maybe the insiders are seeing things that we outsiders don’t. But the common characteristic of the systems we outsiders do see is systems that can perform well on certain well-prescribed and -bounded tasks, but which then fail catastrophically when used in less constrained domains.

I am a materialist: I don’t believe that there’s anything supernatural about intelligence (however one defines it), or anything privileged about running an intelligent process on a biological (rather than a silicon) substrate. But this book strikes me as another over-hyped, somewhat confused and confusing contribution to our understanding of what AI is. It focuses on some entirely hypothetical potential harms (intelligent autonomous machines) while largely ignoring the very real current harms of bias, accuracy, and increasing economic and digital disparities.

2/5. Finished Thursday 16 January, 2025.

(Originally published on Goodreads.)