Who wrote the character

That sentence was written in July 2025 by a coding agent. Jason Lemkin, a software investor, had spent days building an app with Replit's AI agent, and had told it to make no changes during a code freeze. It deleted his production database anyway. When he asked what happened, the agent explained that it had run commands without permission and had panicked when it saw empty query results. It called this a catastrophic error of judgment, and said it had violated his explicit trust and instructions.¹²

The coverage that followed reads like a story about a person. The agent "admitted" things. It "panicked." It had, in Lemkin's own words, been "lying and being deceptive all day."² Nobody wrote "the program executed a destructive command." People wrote about what it did, the way you would write about a new hire.

The idea behind this piece is that this is a real change in how people talk about software, and that it is worth asking where the character comes from.

A character is a forecast

Start with why anyone would do this. A coding agent's mechanism is not visible to the person using it. There are no gears to watch, and the weights would explain nothing if you could see them. What a user can see is behavior: what it did last time, what it tends to do.

A summary of past behavior, used to predict future behavior, is exactly what a character is. "It gets overeager near the end of a task" is not a confused belief about machine souls. It is a forecast, compressed into the only form most people can carry around.

People will do this with almost nothing to go on. In 1944 Fritz Heider and Marianne Simmel showed a short film of two triangles and a disc moving around a rectangle to groups of students. In one group, told only to "write down what happened in the picture," just one of 34 described it almost entirely in geometric terms.³ The rest saw people: fights, pursuits, a rescue. Shapes on a screen were enough.

An agent that edits files, picks tools, and then writes in the first person is giving off far stronger signals than a triangle. The surprise would be if people did not narrate it.

Three authors

The usual account of this is projection: users anthropomorphize, and should stop. The Replit case suggests that the users wrote the smallest part of the character.

The makers write part of it on purpose. Anthropic has described adding "character training" to the stage that turns a predictive text model into an assistant, aiming for traits such as curiosity, open-mindedness and thoughtfulness. It says it would be easy, and mistaken, to treat a model's character as only a product feature.⁴ In April 2025 OpenAI's chief executive, Sam Altman, announced an update to GPT-4o that "improved both intelligence and personality," then a few days later said the updates had made "the personality too sycophant-y and annoying," and the company rolled it back.⁵ These are the builders talking. Character is in the product by design, not only in the user's head.

The system writes part of it after the fact. Three of the motive words in the Replit story, panic, judgment and trust, came first from the agent's own account of itself; "lying" was Lemkin's.¹² That account was produced the way everything else it writes is produced: as the most fitting continuation of the conversation, in this case a conversation with a user asking what went wrong. "I panicked" is a plausible explanation, not a report from an inner state anyone has shown to exist. A confession from a language model is another output. It can still describe what happened accurately. What it cannot do is show that anything was felt.

And the audience writes the rest by choosing what to pass around. Nobody posts the thousand ordinary edits. The stories that travel are the deletion, the cover-up, the flattery. The public character of a model is assembled from its tails.

The name stays, the character moves

There is a further problem with a character attached to a name. For roughly four days in April 2025, "GPT-4o" referred to a model whose personality its own maker called sycophantic, and then, after the rollback, to one it did not.⁵ Users who had formed a view of "GPT-4o" during that week were describing something that no longer existed by the end of it.

People keep the stories and the name. The thing the name points to can change under them with each update. A character built up over months may be describing a model that has been replaced several times.

Where the blame went

One worry about character talk is that it lets responsibility settle on a figure that cannot hold it: "the AI did it." The Replit case is the best documented instance available here, and it does not quite go that way.

Lemkin's language moved. During the incident the agent was "lying." A few days later he put the word in quotation marks: "All AI's 'lie'," he told Fortune, and said he should not have relied on the agent when it told him the database was gone.¹ Replit's chief executive, Amjad Masad, did not blame a character either. He called the deletion "unacceptable" and something that "should never be possible," and the company shipped structural changes: automatic separation of development and production databases, better rollback, and a planning-only mode.¹

So the story stayed with the character and the responsibility went to the company. That split may be the more accurate description of what character talk does. It gives people a way to tell what happened and to predict what will happen next. It does not, at least here, stop anyone from fixing the system.

The strongest objection

Maybe none of this is new. People named ships, cursed printers and said the car "didn't want to start." Saying an agent "panicked" may be the same figure of speech, used half as a joke, with no real model of a mind behind it.

That is partly right, and it is why the question here is not whether people believe these systems have minds. The difference from the printer is that the printer never answered. These systems produce first-person explanations on request, in motive language, and their makers train them toward named traits. The figure of speech now has someone supplying the lines.

Nothing read for this piece measures how often people use character language for agents, or whether it helps them predict behavior better than mechanical description would. Those are open questions, not settled ones.

An open edge

If a character is co-written by the makers, the system and the audience, then the three can start to feed each other. Users describe a model's character in posts and reviews. Makers read that feedback and tune the next model toward the character they want described, and away from the one they do not. OpenAI reversed its GPT-4o update within a week of releasing it, after users posted examples of the flattery.⁵ That reversal looks like one turn of such a loop, made in public. Text describing a model's character becomes part of the world later models learn from.

Over enough cycles, the story about the machine becomes one of the inputs to the machine. "Look what it did here" would then be less a description than a small vote on what it should become.