A Chatbot Is Everything We Wrote Down, Lies Included
A language model is a tool that extends reasoning, built from a record that never marked its lies. Using it takes the oldest skills of a reader: reading, thinking, and weighing a story.
By Geordie Everitt
In 1820, about one adult in eight in the world could read. Schooling belonged to the families who could pay for it, and compulsory attendance is recent enough to have a first state and a last: Massachusetts passed its law in 1852, Mississippi in 1918, sixty-six years later. Trace your family back a few generations and most of the lines probably end in someone who signed with an X.
Their descendants now carry a compressed copy of the written record in a phone and can ask it questions in plain sentences. That is a language model, and it belongs on the same list as the car, the steam engine and the airplane.
A Tool Like the Others
Human language evolved as humans did. The capacity for speech is in the genome, shaped over a span scientists cannot date closely, and writing is a few thousand years old. Computers did not evolve. People designed them, and they count as part of what Richard Dawkins calls the extended phenotype: the effects of an organism's genes that show up outside its body. The beaver's dam is the standard case, a structure that exists because of beaver genes and is made of trees.
I used the termite mound for the same idea in Richard Dawkins Knows AI Isn't Conscious. He Feels It Is Anyway., to explain why a model sounds like us. It also explains what a model is for.
The car extended the leg. The steam engine extended the arm. The airplane gave us a wing no genome ever grew. A language model extends reasoning: it holds more text than a person could read in many lifetimes and can answer from any part of it in seconds.
The Library, Compressed
Until recently a collection like that belonged to people with three things: the ability to read, the leisure to do it, and a university or a rich family to supply the shelves. A model hands the whole collection to anyone who can type a question. The copy is compressed, which means lossy, and in exchange the library answers back.
The interface is language, so the first requirement is reading. The second is thinking: working out what follows from what you were told, and what you would expect to see if it were true. Both are old skills, and both were rare for most of the species' history. In 1820 about one adult in eight could read. Today about one in eight cannot.
The Record Has No Column for True
A language model is trained to predict what a writer would put next. It learns from reports, sermons, advertisements, novels, forged letters, jokes, good-faith errors and lies, all in the same format, none stamped with a verdict. A false sentence somebody wrote is as good a next word as a true sentence somebody wrote. The goal it starts with, guessing the next word, never asks which was which, so it treats the lie and the truth with the same weight. How much sorting the tool does for you afterward depends on the tool and the year.
None of this is new. A written account has never carried its own verdict. Wikipedia has editors, a citation rule and a badge for its best articles, and for about five years starting in 2007 it carried a 4,500-word article on the Bicholim Conflict, a war between the Portuguese rulers of Goa and the Maratha Empire in 1640 and 1641. The war never happened. The article was footnoted, calm and rated Good. It read exactly like the articles about wars that did.
The Bible has the same property, and it contains lies: stories told by characters who stood to gain, and stories about the world that are false. It gives all of them the same standing as the rest, in one voice under one cover, and readers have been arguing about which is which for some two thousand years. A model answers in one voice as well, usually with excellent grammar.
Being able to read does not guarantee being able to weigh. In 1920 Arthur Conan Doyle, who had written the most famous reasoner in English fiction, published two photographs of fairies in The Strand and vouched for them. In 1983 the girls who made them admitted the fairies were paper cutouts propped up with hatpins. The creator of Sherlock Holmes had looked at paper cutouts and concluded fairies.
Some Will Be Better at It
Using a model well takes three things, in order: reading, thinking, and weighing a story for how likely it is to be true. Schools teach the first to nearly everyone. The second and third go to whoever happens to meet a teacher, an editor, a mentor or a bad experience that teaches them. I made the case for teaching the third on purpose in The Dangerous AI Moral Panic: A Crisis of Epistemological Education.
The model widens who can reach the library. How well each person reads what comes back will be spread the way reading skill always has been, unevenly, and some people will be better at it than others.
The car gave ordinary people a faster leg, and the countries that adopted it got good drivers and bad ones. The language model extends reasoning on the same terms.