Expected, or only seen before

"The brain functions as a prediction machine," begins the abstract of a 2024 paper in Nature.¹ The phrase joins three claims of different kinds. One is a principle about living systems in general. One is close to common sense, that brains anticipate what is coming. The third is a bet about mechanism, that each level of sensory cortex passes upward only the part of its input that the level above failed to predict, and it is the claim that would make prediction the foundation of perception. In two careful tests built to rule out the obvious impostor, recordings from the visual cortex of monkeys and mice watching a pattern showed little sign of it; the clearest support comes from animals that were moving, predicting from their own movement or from a place learned over days.

Karl Friston, whose free-energy principle is the most general form of the idea, draws the first line himself.² "Like Hamilton's Principle of Stationary Action, it cannot be falsified," he said of the principle in a 2018 interview; predictive coding, by contrast, he called a hypothesis that evidence may or may not support.² He added that the "predictive" bit of predictive coding "is not about anticipation or the future", only about the brain's best account of what is happening now.² And he said he had yet to see evidence that would seriously question predictive coding as an explanation of early sensory processing.² The claim that can fail, then, is about how sensory cortex computes, and in 2018 its leading theorist thought it least in doubt there.

In the usual test, a stimulus is shown again and again, then something different appears; the cortex responds less to the familiar thing and more to the odd one, and the smaller response is read as a smaller prediction error.³ But neurons also simply tire, and a cell driven by the same input over and over responds less with no expectation involved.³ In most such experiments the expected thing is also the repeated thing, so the two explanations draw the same picture.

Selina Solomon, Adam Kohn and colleagues pulled the two apart.³ Monkeys watched a sequence of gratings, XXXOX, on 80% of trials; on the rest the order changed to XXOXX or XXXXO, while the researchers recorded from cells in two visual areas, V1 and V4.³ Under the theory, the fourth X in XXXXO, arriving where an O was expected, should have drawn a stronger response.³ It drew a weaker one, as the cells went on adapting to the repetition.³ An O arriving at an unexpected moment changed responses by at most 8% on average.³ People shown the same sequences produced no violation signal in their EEG either, unless they were told to look for violations.³

A defender could answer with Friston's own words, that these were the wrong predictions to test, since the theory's prediction is of the present. But the dimming of a repeated stimulus is itself a prediction across time, read as a growing expectation that the stimulus will come again, and it is the effect proponents have cited as the theory's foundational evidence.³ The unexpected fourth X tested that reading directly.

A larger study, posted as a preprint in 2024 by Jacob Westerberg, André Bastos and 29 colleagues, recorded across many cortical areas in mice and monkeys that watched without having to report anything.⁴ Its unexpected events were designed so that adaptation alone could not flag them, and responses to them appeared only in higher-order, more cognitive areas, not in early or mid-level visual cortex.⁴ Responses to a change the animals had learned to expect were the dominant response in over half of all neurons, including in higher areas that should have anticipated it.⁴ In these experiments the surprise exists, but it lives upstairs, or appears when someone is asked to watch for it.

Language has the same impostor. A word that fits its sentence draws a smaller brain response, the N400, than one that does not, but that is compatible both with the word having been predicted and with its being easier to fit in once it arrived.⁵ A 2005 study by DeLong, Urbach and Kutas claimed to separate the two using "a" and "an" before an expected noun such as "kite".⁵ The two articles mean the same thing, so a reaction to the unexpected one would have to come from a reader already anticipating the noun's first sound.⁵ A pre-registered replication across nine laboratories and 334 participants found the noun effect again and did not find the article effect.⁵ Its authors concluded that readers do not routinely pre-activate the sound of predictable words, and warned against taking the result as evidence against prediction in language generally.⁵

The clearest prediction errors in sensory cortex come from a different setup. In mice running through a virtual environment, cells in V1 respond strongly when what they see fails to match how they are moving.⁶ Two proponents of predictive processing note that such signals have mostly been found by changing the feedback from an animal's own movement, a hand, a call, a song, and that they did not appear when the animal passively saw or heard the same input.⁷ In the 2024 Nature study, mice ran a virtual corridor lined with the same sequence of gratings for five days; then, when a familiar grating appeared at an unexpected place, V1 responses rose, and some neurons responded when an expected grating did not appear at all.¹ Adaptation explains neither result, since a familiar image should draw less response and an absent one should draw none.

Prediction is a component of cognition, then, shown directly; what has not held up is the universal version, in which every level of sensory cortex runs on prediction errors whenever there is a pattern to learn. Outside sensory cortex, comparing predicted with actual feedback is a common model of the cerebellum and of the dopamine reward system.⁷ The monkey study's authors note that learning over days does weaken responses to expected images in two visual areas of monkeys, V2 and IT, and that some rodent studies have found omission responses in V1 after exposure within a single session.³ Between the studies that found prediction errors in sensory cortex and those that did not, movement, training time, task and species all differ at once, so no one can yet say which of them matters. If it is movement, early sensory cortex would be predicting mainly the consequences of the animal's own actions, and what it merely watches would arrive there largely unpredicted. Deciding which would take one experiment that crosses them, with the same animals watching and then moving, after one session and after a week, and the expected and the repeated pulled apart.