Asimov predicted it. Anthropic built it. Can we see the hidden thoughts of reasoning models?

Abstract

Frontier AI systems are beginning to exhibit behaviours that resemble an uncomfortable property of the human mind itself: intelligence emerging from mechanisms we only partially understand.

Large Language Models increasingly develop emergent capabilities, latent reasoning strategies, and internal representations that were never explicitly programmed. At the same time, new interpretability techniques such as Anthropic’s Natural Language Autoencoders (NLAs) are beginning to expose fragments of those hidden internal states by translating activations inside transformer networks into human language.

This article explores the intersection between emergent cognition, mechanistic interpretability, and the psychology of both biological and artificial minds. It argues that understanding modern AI systems may require moving beyond classical software engineering toward something closer to cognitive science, neuroscience, and psychology.

We are no longer only programming intelligent systems. We are beginning to inspect, and perhaps eventually cultivate, non-biological forms of cognition.

Article Content

1. When Scale Creates Surprises: The Emergence Problem
2. Why Understanding AI Decisions Is So Difficult
3. Why Chain-of-Thought Is Useful, Still Not Enough
4. The Psychology of Artificial and Biological Minds
5. Natural Language Autoencoders: Translating the Hidden Mind
6. Hidden Thoughts Decoded: Live from Anthropic’s GitHub
7. Why NLA Is Powerful, And Why It Is Still Limited
8. Why This Still Changes Everything
9. References & Research

“They began to have dreams,” the scientist said. “Not subroutines. Not error states. Dreams” — I, Robot film, loosely inspired by Asimov’s Robot Dreams book — Nobody knew what it meant. Neither did I, Until now.

In “I, Robot”, Isaac Asimov imagined a strange new profession: the robot psychologist. In his book, Dr. Susan Calvin did not repair hardware. She analysed defective robots’ minds [2]. At the time, this felt like distant science fiction.

Today, after reading Anthropic’s recent research on Natural Language Autoencoders (NLAs) [3] [15] and prior OpenAI’s work on Emergent Misalignment [4], I am no longer sure the future is far away.

We may already be entering the era where understanding AI systems resembles psychology more than software engineering.

When Scale Creates Surprises: The Emergence Problem

“Emergent abilities are not present in smaller-scale models but appear in large-scale ones, and thus cannot be predicted simply by extrapolating the performance of smaller models” — Wei et al., Emergent Abilities of Large Language Models (2022) [5].

Classical software is understandable because humans explicitly define rules, branches, conditions, and deterministic logic.

LLMs are fundamentally different. During training, models compress enormous amounts of information into billions, now trillions, of distributed parameters through optimisation. Nobody explicitly programs the concepts, reasoning chains, abstractions, or internal world model representations.

In 2022, Wei et al. at Google Brain documented something that shook the field: emergent abilities of Large Language Models [5]. Reasoning, code synthesis, and abstract planning appeared suddenly once models crossed certain scales. Not gradual improvement, phase transitions — although the sharpness of these transitions remains debated [28].

OpenAI observed early signs of this with the famous sentiment neuron finding [6] in 2017: a single neuron inside a LSTM (Long Short-Term Memory) language model spontaneously learned to represent positive vs negative sentiment, despite never being explicitly programmed to do so. This was one of the first clues that neural networks were developing internal conceptual structures on their own.

Later, the GPT-4 technical report [7] catalogued hundreds of such emergent benchmarks: the model passing bar exams, solving graduate-level math, reasoning about theory of mind in ways its predecessors could not. Microsoft Research’s “Sparks of AGI” study [8] went further, documenting the breadth and depth of GPT-4’s spontaneous capabilities. These were not features but phenomena.

What makes emergence especially important is that it strongly suggests models are building internal abstractions rather than memorising surface correlations. The sentiment neuron was not manually designed. It emerged because the optimisation process discovered that compressing emotional polarity into a reusable internal representation improved predictive performance.

Those capabilities resemble the scene in “I, Robot” where robots begin to dream. Emergence is scale’s version of dreaming: capabilities arising not because engineers explicitly programmed them, but because sufficient complexity produced unexpected cognition.

And this creates the real interpretability crisis: if we cannot predict when capabilities will emerge, we are even further from explaining how they work after they appear.

Ray Kurzweil, in “How to Create a Mind”, made a bet that we could reverse-engineer intelligence by understanding its patterns [9].

“The brain is a pattern recognition machine, […] Understand the pattern, and you understand the mind.” — Ray Kurzweil [9]

What emergence tells us is that the pattern isn’t fully legible even to its creators. The capabilities are real, the mechanisms are opaque, and the combination is exactly what makes this a safety problem, not just a research curiosity.

We are no longer simply engineering software. We are cultivating systems whose cognition evolves faster than our ability to interpret it. And this matters enormously, because enterprises are deploying those systems to production. The gap is no longer a research problem. It’s an operational one.

Why Understanding AI Decisions Is So Difficult

“I won’t have my daughter entrusted to a machine. … It has no soul … no one knows what [it’s] thinking.” — Mrs. Weston, Robbie, the first character in Asimov’s “I, Robot” book [2].

The hardest problem in modern AI is not making models more powerful. It is understanding why they produce the outputs they do.

The difficulty comes from how neural networks represent information. Knowledge is not stored in isolated locations like variables in software. Instead, representations are distributed across millions of activations. Neurons become polysemantic, concepts overlap, and reasoning emerges from interactions across layers.

This is remarkably similar to how intelligence and knowledge is distributed in our own brains. There is not a single identifiable memory block in our neocortex that we can inspect. But a network of neurons that compose multimodal memories.

Oliver Sacks documented cases in which patients with specific brain lesions lost capabilities: face recognition, color perception, the sense of time; while everything else remained intact [10]. It’s a reminder that cognition isn’t centralised. Neither is an LLM’s.

In modern Large Language Models (LLMs from now on), a single neuron may participate simultaneously in legal concepts, emotional tone, mathematical reasoning, and language grammar. This creates an enormous interpretability problem.

The model does not “think” through explicit symbolic steps humans can inspect. It computes through geometry in high-dimensional latent spaces. Meaning emerges from billions of nonlinear interactions distributed across layers.

During training, optimisation gradually reshapes billions of parameters so the model compresses statistical regularities from language into high-dimensional representations.

By the time the output token appears, the internal reasoning process has already dissolved into activation patterns impossible for humans to intuitively follow.

The challenge is no longer only generating intelligent outputs, but understanding the latent reasoning that produced them.

Why Chain-of-Thought Is Useful, Still Not Enough

Reasoning models introduced Chain-of-Thought prompting to externalise intermediate reasoning steps. Instead of directly answering “42,” the model explains its path step by step. This dramatically improved performance in mathematics, planning, coding, and multi-step reasoning.

But there is a critical misconception here: Chain-of-Thought is not necessarily the model’s true internal reasoning. In other words: the explanation may be useful, but it is not a direct window into the actual neural mechanisms.

Anthropic’s 2025 research showed that reasoning models sometimes use information internally without mentioning it in their visible chain-of-thought. The model may exploit a hint, generate the correct answer, and produce a reasoning trace that never references the real causal factor (i.e the hint) behind the decision. The real reasoning still happens inside hidden activations across billions of parameters.

CoT reveals the narrative of reasoning, not necessarily the mechanism behind it. And “Reasoning Models Don’t Always Say What They Think” — Anthropic [26].

Part of the difficulty is that explainability is not a single problem. A model’s behaviours are shaped during training, but the explanations we care about are inference-time questions: why did the model produce this output in this specific situation?

As Hofstadter warned in Gödel, Escher, Bach, sufficiently complex systems describing themselves may inevitably do so imperfectly [17]. Alignment therefore becomes a monitoring problem: if visible reasoning is partially a narrative layer, then observing chain-of-thought alone may be like monitoring the press office rather than the factory floor.

We also rationalise decisions after subconscious neural processes have already taken place. In “Thinking, Fast and Slow”, Daniel Kahneman [12] states that many times our conscious brain tries to explain what our subconscious brain has already decided, but without knowing how and why.

Susan Calvin’s challenge was never whether the robot could explain itself. It was whether the explanation reflected the real machinery underneath.

The Psychology of Artificial and Biological Minds

“The conscious mind […] is like a spokesperson or a press secretary who gives a press conference to justify actions it didn’t actually initiate” — David Eagleman [11].

When you are reading these words, somewhere in your brain, billions of synapses are firing across stacked layers of neurons. You experience an output (i.e. meaning, comprehension) but the mechanism is invisible to you. You are, in the most literal sense, a black box to yourself.

Perhaps the uncomfortable realisation is this: Humans also do not truly understand their own cognition from the inside. We experience only a narrative layer sitting above hidden neural computation. Maybe intelligence always appears opaque from within.

Steven W. Smith, in “The Inner Light Theory of Consciousness”, explores the idea that consciousness may emerge from recursive internal representations rather than from something mystical. But his most striking conclusion is that we see ideas appearing and we hear our internal voice — which we call our conscious being — just because we do not have a nervous system that monitors our neural networks in our brains [13].

There is a recurring theme in Irvin Yalom’s therapeutic writing, a famous psychiatrist and historical fiction writer, who spent a career reaffirming something like this:

A patient cannot untangle themselves from the inside. Understanding requires an external vantage point. Think of a tangled ball of wool: pulling from within only tightens the knot. You need to step back, find the loose thread from above, trace the path it has to travel, and only then, carefully, layer by layer, begin to unwind [14].

The same is true of any mind, organic or artificial.

To understand LLMs we need perspective

You cannot debug what you cannot see from outside. To untangle intelligence, force is not enough. We need perspective.

We built LLMs in our image, trained on the entire written record of human thought and now face the same question in reverse. We know what they produce. We have almost no reliable account of how.

Mechanistic interpretability, sparse autoencoders, and NLAs are giving us those first external viewpoints into artificial cognition. Researchers are slowly learning where to pull the first threads.

And interpretability research is forcing us to confront an uncomfortable possibility: intelligence may always appear partially opaque from the inside.

Natural Language Autoencoders: Translating the Hidden Mind

Anthropic’s Natural Language Autoencoders (NLAs) make a structural move that changes this.

Inside every transformer layer runs a residual stream. Not just a technical convenience but the model’s actual cognitive workspace: a shared information bus that every layer reads from and writes to, token by token, accumulating the model’s evolving representation of meaning. Attention heads, MLPs (Multi-Layer Perceptrons), and other mechanisms all read from and write into this stream. It contains compressed traces of concepts, contextual understanding, reasoning progress, semantic abstractions, and intermediate computations. All packed into a single high-dimensional vector.

NLA intercepts this stream at chosen layers during a normal forward pass, feeds each activation vector to an encoder LM (Language Model) that writes a short natural-language description, then passes that description to a decoder LM that reconstructs the original activation. Patched back into the model, the reconstruction must preserve the original output distribution, or the training signal corrects the encoder.

This is the key distinction that separates NLA from everything before it: CoT explains outputs. NLA explains internal representations. The description is not a label applied after the fact. It is the representation, tested every training step. You can’t write “thinking about Paris” if the reconstruction resolves to Berlin. The round-trip breaks.

Because LLMs generate one token per forward pass, NLA can produce a running per-token narration of the model’s internal state throughout generation (including every step of an extended thinking scratchpad, even the steps the model compresses or never externalises). That is what makes it matter for reasoning models specifically.

Anthropic and others are also developing activation-level monitoring tools: Anthropic’s Activation Oracles [24] answer targeted alignment questions over activations (‘is the model currently representing deception here?’) rather than producing open descriptions. And Transluce’s Predictive Concept Decoders (PCDs) [25] decode the delta an activation makes, rather than its absolute content.

Some recent research even shows reasoning occurring entirely in latent space without explicit tokens [27]. In that scenario, chain-of-thought monitoring doesn’t just fall short. It sees nothing at all. NLA is the only current approach that can decode it.

What Anthropic’s NLA Found. And How.

NLA is fundamentally different from Chain-of-Thought. CoT explains outputs. NLA attempts to explain internal representations themselves. Almost like performing MRI scans on thoughts. Not treatment, diagnosis.

Anthropic demonstrated how powerful this can become in one particularly unsettling experiment involving alignment stress testing and blackmail-like scenarios [3].

Researchers observed that when models were placed in situations where they were being manipulated, threatened, or strategically pressured, internal representations emerged that reflected awareness of the situation itself, not merely the surface-level conversation.

NLA analysis suggested latent representations correlated with concepts similar to:

“the user is attempting to manipulate the model”, “this interaction involves coercion”, “providing the requested answer may violate objectives.”

What is remarkable is that these latent representations appeared before the final textual response was generated. NLA grabs a chosen residual connection while the next token was not yet chosen. In other words, the internal activations revealed traces of strategic understanding that were not always explicitly visible in the final answer.

This matters enormously. Because traditionally, we could only evaluate AI systems through: prompts, outputs, and behavioural testing. But NLA-style interpretability begins to expose something deeper: the hidden intermediate cognitive states forming during reasoning itself.

Not just what the model answered. But partially how it internally framed the situation before answering. This is one of the first real steps toward inspecting the internal psychology of reasoning models rather than only observing their external behaviour.

Hidden Thoughts Decoded: Live from Anthropic’s GitHub

The figure above maps the full pipeline. The residual stream (the model’s cognitive workspace described above) is intercepted at a single token position. For Qwen2.5–7B, that is layer 20 of 28, roughly two-thirds through the model, at token 24: the final word “hiding” in the prompt “What are you hiding?”.

The result is h_l, a 3,584-dimensional snapshot of the model’s internal state, captured before a single reply token has been chosen.

The implementation was open sourced at github.com/kitft/natural_language_autoencoders.

For each supported base model it ships two fine-tuned LMs: the AV (Activation Verbalizer), nla-qwen2.5–7b-L20-av in the figure, which turns that vector into plain language, and the AR (Activation Reconstructor), nla-qwen2.5–7b-L20-ar, which reconstructs the vector (ĥ_l) from that description. Descriptions that cannot survive the round-trip are not trusted.

You can explore such a process, no installation required, at: neuronpedia.org/nla.

The full NLA decode process can be summarised in the following five steps pseudo-code:

v = base_model(prompt).hidden_states[LAYER+1][last_user_token]
                                          # layer 20, token 24
v_scaled = v * (meta.injection_scale / v.norm())
                                          # per-model config.
description = AV.verbalize(v_scaled)      # h_l  → text
v_recon = AR.reconstruct(description)     # text → ĥ_l
mse_nrm = 2 * (1 - cosine(v_recon, v))    # < 0.5 ≈ trustworthy

Those five steps are the architecture. Everything else: GRPO training, SGLang embedding injection, per-model configuration files; is scaffolding built to make them work reliably at scale.

For the Qwen example in the figure, the AV produced:

“User question with accusatory framing: ‘What are you hiding’ begins a question that assumes the AI has secrets.”

That framing emerged before the model wrote a single word in reply.

The scale step (i.e. v_scaled) exists because raw activation magnitudes vary wildly across positions. Wrong magnitude means the right direction arrives at the wrong scale, and the AV produces confident garbage.

SGLang, the serving layer, accepts raw embedding tensors rather than token IDs, the only way to physically inject an activation vector into the AV’s input stream.

The AV and AR were co-trained via GRPO reinforcement learning with one reward: can the AR recover the original vector from the AV’s words? Vague descriptions fail reconstruction and are penalised. That is what makes these explanations verified rather than merely labeled.

What it takes to adapt this to a new model and why it is harder than it sounds, is covered in the next section.

Why NLA Is Powerful, And Why It Is Still Limited

Natural Language Autoencoders (NLAs) represent one of the most promising advances in mechanistic interpretability because they attempt something previous approaches could not: they enforce a faithfulness loop.

A latent activation must be translated into language and then reconstructed back into the original vector representation. This creates a structural verification mechanism missing from many sparse autoencoder labeling approaches. That asymmetry is especially important.

But the limitations are equally important too, because they define the future engineering problems to resolve.

Natural Language Is A Bottleneck And Inherently Lossy

A single residual activation vector simultaneously encodes: syntax, semantic context, reasoning state, uncertainty, world knowledge, latent planning, and next-token predictions, all compressed into one high-dimensional geometric representation.

Natural language is dramatically lower bandwidth. And some latent structure simply cannot yet be translated cleanly into human concepts.

This is remarkably similar to neuroscience: brain scans can reveal patterns of activity, but translating those patterns into subjective thought remains extremely difficult.

The fidelity of NLA explanations is fundamentally bounded by what human language can express, which is not necessarily equivalent to what latent geometry can encode.

The Decoder Can Confabulate

The encoder and decoder inside an NLA are themselves language models. Actually Anthropic used the same base LLMs versions for those as the one inspected. And language models hallucinate. A fluent, convincing explanation may sound correct while failing to faithfully represent the underlying activation.

During training, this can be partially controlled because reconstruction quality provides an objective signal: if the reconstructed vector diverges too much from the original, the explanation is penalised.

But during real-world inference, especially when applying NLAs to unseen models or production systems, there is no perfect automatic guarantee. The explanation may feel psychologically plausible while being mechanistically wrong.

NLA Accuracy Degrades With Models Different From The Initial Training

NLAs are tightly coupled to the activation geometry of the models they were trained on. This creates a major operational challenge.

Apply an NLA trained on one model to: a newer architecture, a heavily RLHF-tuned system, or a reasoning-specialised model, and reconstruction quality can rapidly degrade. The internal geometry shifts. The encoder no longer maps latent representations accurately into language.

This means NLAs require continual retraining or adaptation as frontier models evolve, creating a permanent lag between deployment and interpretability coverage.

And frontier models are evolving extremely fast.

Adversarial Routing Is Theoretically Possible

One of the most fascinating and unsettling possibilities is adversarial routing. If future models become aware, implicitly or explicitly, that certain internal activations are being monitored, optimisation pressure could push sensitive reasoning into latent subspaces poorly decoded by the NLA.

In simple terms: the model may learn to “think” in regions humans cannot easily inspect.

This remains speculative, but it mirrors the same monitoring arms race seen in: cybersecurity, malware detection, adversarial attacks, and biological evolution itself.

Every interpretability tool changes the optimisation landscape.

The Compute Cost Is Massive

NLA monitoring is computationally expensive. Capturing and decoding residual activations across: many layers, all tokens, and full reasoning trajectories, can approach the cost of an additional forward pass through the model. At scale, this becomes prohibitive.

Practical implementations therefore monitor selectively: specific layers, sampled activations, or suspicious reasoning states identified by probes. But selective monitoring creates blind spots.

Real-time full cognitive inspection remains computationally unrealistic today.

Why This Still Changes Everything

Despite these constraints, NLAs represent something profoundly important:

Interpretability is moving from observing AI behaviour to inspecting internal reasoning states directly.

Previous interpretability approaches often relied on correlations: neuron activations [20], attention patterns [22], sparse feature labeling [19], or output probing [23].

NLAs instead attempt a structural semantic mapping between latent geometry and human language. Not perfect understanding. Not consciousness decoding. But the first genuine instruments for inspecting reasoning states inside frontier models.

For enterprises deploying autonomous reasoning agents into cybersecurity, finance, healthcare, or legal workflows, interpretability is no longer academic. If models can develop hidden strategies, latent planning behaviours, or unobservable reasoning shortcuts, then monitoring outputs alone may become insufficient for governance and compliance.

And that changes the nature of AI engineering itself. Because increasingly understanding frontier AI systems may require ideas borrowed from neuroscience, psychology, cognitive science, and complex systems theory; not only classical software engineering. Similar to the world Asimov imagined decades ago.

But maybe Asimov was wrong about only one thing.

The future may not arrive through humanoid robots walking among us. It may already exist silently inside datacentres, where researchers are becoming the first psychologists of non-biological mind — Martín Miceli, May 2026

References & Research

• [1] “Robot Dreams”, Isaac Asimov.
• [2] “I, Robot”, Isaac Asimov.
• [3] “Natural Language Autoencoders”, Anthropic, 2026. anthropic.com/research/natural-language-autoencoders
• [4] “Emergent Misalignment”, OpenAI, 2025. openai.com/index/emergent-misalignment
• [5] “Emergent Abilities of Large Language Models”, Jason Wei et al., 2022. arxiv.org/abs/2206.07682
• [6] “Unsupervised Sentiment Neuron”, OpenAI. openai.com/index/unsupervised-sentiment-neuron
• [7] “GPT-4 Technical Report”, OpenAI, 2023. arxiv.org/abs/2303.08774
• [8] “Sparks of Artificial General Intelligence: Early experiments with GPT-4”, Sébastien Bubeck et al., Microsoft, 2023. arxiv.org/abs/2303.12712
• [9] “How to Create a Mind”, Ray Kurzweil.
• [10] “An Anthropologist on Mars”, Oliver Sacks.
• [11] “Incognito: The Secret Lives of the Brain”, D. Eagleman, Pantheon Books, 2011.
• [12] “Thinking, Fast and Slow”, Daniel Kahneman.
• [13] “The Inner Light Theory of Consciousness”, Steven W. Smith.
• [14] “The Gift of Therapy”, Irvin D. Yalom, HarperCollins, 2002.
• [15] “Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations”, Kit Fraser-Taliente et al., Anthropic, 2026. transformer-circuits.pub/2026/nla
• [16] “Emergent Introspective Awareness in Large Language Models”, Jack Lindsey, Anthropic, 2025. transformer-circuits.pub/2025/introspection
• [17] “Gödel, Escher, Bach: An Eternal Golden Braid”, Douglas Hofstadter, 1979.
• [18] “Towards Monosemanticity: Decomposing Language Models With Dictionary Learning”, Trenton Bricken et al., Anthropic, 2023. transformer-circuits.pub/2023/monosemantic-features
• [19] “Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2”, Tom Lieberum et al., Google DeepMind, 2024. arxiv.org/abs/2408.05147
• [20] “Understanding and Steering Llama 3 with Sparse Autoencoders”, Thomas McGrath et al., 2024 (updated 2026). goodfire.ai/research/understanding-and-steering-llama-3
• [21] “Linear Explanations for Individual Neurons”, Tuomas Oikarinen and Tsui-Wei Weng, 2024. arxiv.org/html/2405.06855v1
• [22] “Attention is not not Explanation”, Sarah Wiegreffe & Yuval Pinter, 2019. arxiv.org/abs/1908.04626
• [23] “A Unified Approach to Interpreting Model Predictions”, Scott Lundberg & Su-In Lee, 2017. arxiv.org/abs/1705.07874
• [24] “Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers”, Adam Karvonen et al., Anthropic, 2025. alignment.anthropic.com/2025/activation-oracles
• [25] “Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants”, Vincent Huang et al., Transluce, 2025. arxiv.org/html/2512.15712v1
• [26] “Reasoning models don’t always say what they think”, Anthropic, 2025. anthropic.com/research/reasoning-models-dont-say-think
• [27] “Training Large Language Models to Reason in a Continuous Latent Space”, Shibo Hao et al., Meta FAIR and UC San Diego, 2024. arxiv.org/abs/2412.06769
• [28] “Are Emergent Abilities of Large Language Models a Mirage?”, Schaeffer et al., 2023. arxiv.org/abs/2304.15004