My model learned to think. It never learned to listen.

When I was four, I asked my dad:

“Why can’t autoregressive language models remain in continuous latent space between decoding steps?”

He looked down at me.

“What the fuck is an autoregressive language model?”

My first peer reviewer was unconvinced. This would become a theme.

Twenty years later, I spent 21 GPU-hours finding out.

The suspiciously good idea

Language models think in vectors but communicate in tokens. During ordinary chain-of-thought reasoning, every latent thought gets stamped down into a vocabulary token.

This seems silly. Being the bold-faced explorer I am, I set out to set the world right. No more silly, pre-emptive, lossy decoding.

Papers such as Coconut asked the obvious question: what if the model stayed in hidden-state space and decoded only when it had something worth saying?

I tried an aggressive version on Qwen2.5-1.5B-Instruct. Whenever the model became uncertain, it paused. I cloned its hidden state into several parallel streams, added different noise to each, and let them think without producing tokens. Then I compressed their work, inserted it into the context, and resumed generation.

Parallel subconscious reasoning. A global-workspace bottleneck. Several tiny minds working together inside my laptop.

That was the story, anyway. Mechanically, I was feeding a transformer’s output back into itself and hoping it would become clever.

The frozen model discovers /router

Before training, I built a logit lens: at each silent reasoning step, project the hidden state through the output head and inspect the tokens it most resembles. This does not make the model speak. It just lets us rummage through its bins.

At step zero, the states resembled useful tokens: “Step”, “answer”, digits. Two to four steps later they had drifted into whitespace, /router, Array, and random CJK fragments.

The trajectories explored roughly a thousand times the hidden-space volume of an ordinary token chain of thought. Unfortunately, the volume was empty.

Reach is not meaning.

The streams also converged. This initially sounded promising: perhaps disagreement meant uncertainty and convergence meant consensus. In reality, they were agreeing on junk. Every stream had fallen into the same content-free attractor.

I had invented collapse and nearly called it confidence.

Fine. Train it. (Ablation style)

Nobody expected a frozen language model to understand this strange new organ, so I trained it. At every evaluation I ran three versions:

1. Use the latent thoughts normally.
2. Replace every latent thought with zero.
3. Give each question another question’s latent thoughts.

If the first version does not beat the other two, the thoughts are decorative, whatever the loss curve says.

Then I climbed a ladder of increasingly expensive optimism.

Attempt one: boundary adapter

I added a 4.7M-parameter adapter because final-layer states are badly out of distribution as input embeddings.

Loss fell from 4.3 to 1.3. GSM8K rose from 16.8% to 18.4%. Then the zeroed version beat the normal one.

The adapter had learned to emit a useful constant bias vector: prompt tuning in a trench coat.

Attempt two: LoRA

I added 23M trainable parameters and the first stage of a Coconut-style curriculum. Loss reached 0.53 and GSM8K reached 26%.

The ablations showed no separation. I had successfully fine-tuned a model on maths text. The latent channel remained unread.

Attempt three: more curriculum

I hid more of the written chain of thought behind latent blocks. Accuracy fell to 17.6%.

Forcing a model to rely on a channel containing nothing turns out to remove information.

Attempt four: make cheating impossible

This was the serious run. I pulled each latent state toward the teacher’s corresponding written reasoning state, then masked 30% of the visible reasoning so the decoder would have to look somewhere else.

The distillation worked on its own terms. Loss fell from 1.98 to 0.42. Lens entropy dropped from 6.5 to 2.2 nats. The states became sharp, structured, and teacher-like. Content was definitely going in.

At step 800, the normal model beat both ablations for the first time. Finally, something had worked.

Then the final evaluation finished:

Configuration	Accuracy
Thoughts enabled	20%
Thoughts replaced with zeros	30%
Thoughts stolen from another question	20%

The model performed best when I surgically removed the feature I had spent two weeks building.

Across twelve ablation evaluations, zeroing or shuffling the thoughts matched or beat the real thoughts in eleven.

It had learned to write thoughts. It had not learned to read them.

I also tried going deeper

For completeness, I repeatedly looped a block of transformer layers during normal generation. One pass reproduced the stock model. Two passes were roughly neutral. Four passes dropped accuracy to 6%.

There is apparently a point between “consider it again” and “sustain a head injury.” Models such as Huginn learn depth recurrence during pretraining. You do not get it for free by turning the loop count up at inference.

What actually survived

Ordinary token chain of thought scored 30% while using one-sixth of the FLOPs of any latent configuration.

In hindsight, there are some obvious problems with the experiment:

1. I was fine-tuning a model that was not trained to use latent reasoning. The lift may not have been as small as I anticipated.
2. Collapsing a latent state into a rigid token prevents runaway dynamics. Without that constraint, the model just kind of roams around.
3. If this line of attack worked, I probably would not have been the first to notice.

But the result I trust most is about evaluation. All of these looked like progress:

• falling training loss;
• falling distillation loss;
• latent states becoming much sharper;
• a nine-point benchmark improvement;
• one glorious mid-run evaluation.

Every measurement was real and reproducible. None meant what I wanted it to mean. Two extremely stupid controls exposed the whole thing: replace the clever new mechanism with zeros, or replace it with somebody else’s output.

If an architecture claims that a model reasons in latent space, ask whether anyone zeroed the latents.

Caveats, before I become a paper reviewer

This was one 1.5B model, one task family, one seed per configuration, and parameter-efficient training on a laptop. Full fine-tuning is where the published parity claims live, and I could not afford it. Injecting every stream separately may also work better than my mean-and-variance bottleneck.

If someone runs either experiment, I would genuinely love to be wrong.

Code, training logs, run data, and the full technical report: github.com/Oli-26/LatentReasoningNoDecode