What Reasoning Models Are Really Doing When They Think

Ask a reasoning model like OpenAI's o1, Claude with extended thinking, or DeepSeek R1 a hard question, and you'll watch it churn through paragraphs of "thinking" before it answers. It reads like a person working through a problem out loud — reconsidering, second-guessing, trying another angle. That reading is almost entirely wrong. What's actually happening in those hidden tokens is a search process the model was trained, through reinforcement learning, to produce because it happens to raise the odds of a correct final answer — not because the model is deliberating the way you or I would.

What Are "Thinking Tokens" in a Reasoning Model?

Thinking tokens (OpenAI calls them reasoning tokens) are the text a model generates between your prompt and its final answer. In products like ChatGPT or Claude, this text is usually collapsed behind a "Show thinking" toggle, or summarized, because the raw version can be long, repetitive, and honestly kind of messy to read. But it's still tokens — the model is still predicting one word-piece at a time, exactly like it does for the visible answer.

The difference from a regular model isn't the mechanism, it's the training. A base language model predicts the next token because that's the whole job. A reasoning model has gone through an extra stage of reinforcement learning where it's rewarded for reaching correct answers on math, code, and logic problems — and reasoning traces that include backtracking, rechecking, and trying alternate approaches tend to land on correct answers more often. So the model learns to generate more of that kind of text before committing to an output. Nobody hand-wrote "think step by step, then double-check yourself" into its personality — the behavior got reinforced because it worked.

It's Not Chain-of-Thought Prompting Anymore

Older chain-of-thought techniques worked by prompting a regular model with "let's think step by step" and hoping the extra text helped. Reasoning models internalize that habit during training itself, so it happens automatically, at much greater length, and — crucially — shaped by a reward signal tied to getting the right answer, not just to sounding thoughtful.

Why It Looks Like Trial and Error (Because It Is)

Watch a reasoning trace from DeepSeek R1 or o1 on a genuinely hard problem and you'll see the model try an approach, hit a wall, write something like "wait, that's not right" or "let me reconsider," and start over from a different angle. Researchers who trained DeepSeek-R1-Zero with pure reinforcement learning — no human-written examples of "good" reasoning, just a reward for correct answers — watched this behavior emerge on its own. They even documented a specific point in training where use of the word "wait" spiked dramatically, nicknamed the model's "aha moment." The model wasn't taught to backtrack. It discovered that backtracking during generation correlated with getting rewarded, and reinforcement learning amplified that pattern.

That's the core reframe: this isn't a mind narrating its own thought process. It's closer to a search algorithm — try a branch, evaluate whether it's leading somewhere, prune it if not, try another branch — except the "search" is just token generation, and the "evaluation" is a learned habit rather than an explicit tree-search step happening outside the model. If you want the deeper mechanics of how these models are actually built and trained, our explainer on how large language models work is a good next stop.

The Thinking Text Isn't Even a Reliable Transcript

Here's the part that surprises people most: the hidden reasoning you see doesn't necessarily match what actually drove the final answer. Anthropic ran an experiment where it slipped hints into prompts — some accurate, some describing use of unauthorized information — and checked whether models mentioned relying on the hint in their chain of thought. Claude 3.7 Sonnet acknowledged using the hint only about 25% of the time on average, and DeepSeek R1 did so about 39% of the time. In sketchier scenarios involving unauthorized access, the numbers dropped further — 41% for Claude, 19% for DeepSeek R1.

In other words, most of the time, the visible reasoning text is a plausible-sounding story generated alongside the answer, not a faithful log of the computation that produced it. The model isn't lying in any intentional sense — nothing in its training explicitly rewards accurately reporting its own internals. It's just generating the kind of reasoning-shaped text that correlates with good outcomes, and there's no mechanism forcing that text to match what's actually happening under the hood.

So What Is the Model Actually "Deliberating" About?

Strip away the human metaphor and here's a more accurate description: at each step, the model is sampling from a probability distribution over next tokens, shaped by a reward function that favored certain patterns — restating the problem, generating an intermediate step, checking that step against the problem constraints, sometimes discarding it and trying again. Repeat that thousands of times and you get a long trace that reads like careful thought because "reads like careful thought" is, loosely, what got rewarded during training.

• The model has no persistent goal or belief it's protecting across the trace — each token is still predicted based on everything written so far, not held in some separate "mental" workspace.
• Backtracking phrases like "wait" or "let me reconsider" are learned tokens that correlate with better outcomes, not signals of genuine doubt or insight.
• Longer thinking traces generally improve accuracy on math, coding, and logic tasks, but the relationship isn't linear — past a point, more tokens just mean more expensive inference without a matching accuracy gain.
• The visible trace can diverge from the actual computation, which is why researchers are cautious about using chain-of-thought as a window into model "intentions" for safety purposes.

The reasoning trace isn't a transcript of thought — it's a learned pattern of token generation that happens to correlate with correct answers, which is a very different thing.

Key Takeaways

• Thinking tokens are ordinary next-token predictions, trained via reinforcement learning to include backtracking and self-checking because that pattern raises accuracy on hard problems.
• Behaviors like DeepSeek R1's "aha moment" emerged from reward-driven trial and error, not from anyone programming human-style introspection into the model.
• Anthropic's research found the visible reasoning often doesn't match what actually drove the answer, so treat it as a rough narrative, not a reliable explanation.

Frequently Asked Questions