The part that knows

Ask someone to recall a list of words they heard a few minutes ago and they will often hand back a word that was never on it. Ask a language model about a book it was never shown and it will often describe the book anyway. In both cases the answer is plausible, fluent and wrong, and in both cases the system seems not to notice. It is tempting to conclude that human memory is a kind of language model: a predictor that generates the likely rather than retrieving the true.

Half of that is right, and the half that is right is less damning than it sounds. The other half fails in a specific place. A language model resembles the part of memory that knows things, not the part that remembers them, and the difference is a piece of anatomy that the model does not have.

Reconstruction is the design

The idea that remembering is generation rather than playback is old. In 1932 Frederic Bartlett argued, against the notion of memory as a literal reproduction of the past, that remembering "is an imaginative reconstruction or construction".¹ The best-known modern demonstration is the DRM illusion. People hear a list of words such as candy, sour, sugar, bitter, honey, chocolate, cake and pie, all associated with "sweet", which is never presented; on a later test many of them recall or recognize "sweet" anyway.¹

That is exactly the error a system that stores associations rather than records ought to make. It keeps the gist of the list and regenerates the items, and the most associated word comes out whether or not it was there.

Daniel Schacter, a psychologist at Harvard, argues that such errors are not a malfunction. The processes that produce them are the ones that let memory generalize, and the same machinery that recombines past experience is what we use to imagine the future.¹ A memory that kept exact records would be worse at the thing memory is mostly for, which is preparing for situations that have not happened yet. On this point the comparison with a language model holds. Both produce the typical because both are built from regularities.

Where the model is the literal one

The usual contrast says the machine stores and the person reconstructs. On one axis it runs the other way. Language models emit passages of their training data word for word when prompted the right way, and the amount they can reproduce grows with the size of the model and with the number of times a passage appeared in the training data.² Bartlett built his case on the opposite finding: people's memories distort what they took in, which is why he rejected the idea of remembering as literal reproduction.¹

What people do memorize verbatim, prayers, lyrics, the times table, they memorize the way the model does: by repetition. So the difference here is not prediction against storage. Both systems become literal where the material was repeated and reconstructive where it was not; the model has simply read far more repeated text than anyone could.

The missing store

The real difference is structural. An influential account published in 1995 divides memory in two. The neocortex learns slowly, adjusting a little with each experience and interleaving many of them, which is how it extracts structure. The hippocampus learns fast. It holds new episodes without disturbing that structure and reinstates them so that the slow system can absorb them over time.³ The evidence came from patients with damage to the hippocampus, who keep their older memories but cannot lay down new ones.³

A deployed language model has the slow half. Its training is the gradual, interleaved learning of structure from an enormous number of examples. It also has something like a fast store: the conversation in front of it, which it can use precisely while the conversation lasts. What it lacks is the path between them. Nothing said in the conversation is written back into what the model knows; when the conversation ends, it is gone. Chat products that "remember" earlier conversations do it by saving notes and placing them in front of the model next time. That is closer to reading one's own diary than to consolidation, because what the model knows has not changed. In this respect the model is closer to the hippocampal patient than to a whole memory: rich in knowledge acquired before, unable to keep anything that happens now.

A person usually has some sense of when they learned something and whether it happened or was only imagined, though that sense fails often enough to have its own literature. Imagining an event, for instance, makes people more likely to later remember it as real.¹ A model has much less. Even its makers cannot reliably say where its knowledge of a given subject stops: the effective cutoff of what a model knows often differs from the date its developers report, partly because new collections of web text contain a good deal of old text.⁴ Knowing when and where something happened is an episodic property, and a system that never forms episodes has nothing to attach it to.

So the right comparison is narrower than "memory is a language model" and more useful. A language model is semantic memory at an enormous scale, sometimes literal, always reconstructive, with no episodes. When it fills a gap with something plausible it is doing what a person does. When it cannot tell you when it learned something, or remember yesterday's conversation, it is showing the absence of the part of memory that, in people, turns experience into a past.