A Tiny LLM Learned to Reason on One CPU Core in 9 Hours (2026 Guide)

"You need racks of GPUs to train anything useful" — still true? Maybe not for much longer.

On September 7, 2026, Greg Diamos — MLPerf founding member, formerly Baidu SVAIL, now at Anthropic — published a paper that takes the opposite constraint to its limit: training a 3.65M-parameter language model on exactly one CPU core. In nine hours, basic reasoning behaviors emerged. The experiments were run by Claude Code, and the paper lists "Claude Opus 5" as co-author — by name, not as an acknowledgment.

The tweet went viral (2,517 likes / 2,602 bookmarks in ~10 hours). Here's the full breakdown.

What you'll learn

Why single-core? (roofline-first architecture design)
What emerged after 9 hours (actual numbers)
The paper's core: 4 failures invisible in the loss curve
What this means for local AI

The Experiment at a Glance

Pipeline of the one-core LLM training experiment (figure by cldnavi.com) — Figure: created by cldnavi.com

Hardware: one physical core of an Intel Xeon Silver 4514Y (Emerald Rapids), pinned with OMP_NUM_THREADS=1
Model: block-routed MoE with 3.65M active parameters (E=128 experts)
Training: 259M tokens in ~9 hours at 6,616 tok/s

The point is not "a small model runs on CPU." It's that the architecture is derived from a single-core roofline: weights sized to fit the 2MiB L2, matrix ops big enough to amortize the 1.4–1.5µs GEMM dispatch floor. Training throughput is an end-to-end forward+backward rate, not a micro-benchmark.

What Emerged After 9 Hours

Task	Accuracy	Chance
In-context induction	93%	7%
Positional shift	90%	7%
Two-digit addition	47%	23%

These were behaviors the team assumed needed a much larger budget. They appeared at 5% of the tokens of the longest run.

And the 4.91B-token run shows training loss still descending at the end — 1,481 tokens per active parameter, roughly 74× the Chinchilla ratio, with no floor in sight. At normal scales that ratio is unaffordable; at 3M parameters it's cheap. That's the inference-optimal regime.

Post-Training: 4 Rounds to a Working QA Model

The pretrained model couldn't generate — it entered an absorbing state within 5 free-running tokens. Four rounds of post-training, each addressing a failure the previous one exposed:

Round	Effect
1. Instruction SFT	stop-on-EOT 0→52.5%
2. QA SFT	EM 0→13.9%
3. Abstention rebalance	over-refusal 57.7→19.8%
4. Vocabulary fix	EM 15.3→18.2%, F1 19.6→23.2%

Final: 18.2% EM / 23.2% F1 on held-out extractive QA (25.0% on DROP). Weak in absolute terms; remarkable for the size.

The Core: 4 Failures the Loss Curve Never Reported

Expert collapse — 128 experts collapsed to one function while loss looked plausible. Caught via the RMSNorm gain participation ratio (1.0/384). Root cause: not enough tokens per expert (the budget law E ≤ Nk/(τPe)).
Zero-init insertion isn't function-preserving under top-k routing — a zero-gated expert still displaces the k-th ranked one.
The routing statistic dominates — windowed vs prefix mean was the single largest improvement (4.431→4.041), but their proposed explanation was refuted by their own shuffle ablation. They published the refutation alongside.
Untrained vocabulary rows win the argmax — tokens with corpus count 0 are never sampled as negatives, keep logit ≈0, while trained-but-wrong tokens get pushed to ≈−7.9. Result: 29% of DROP answers were just ' ballo' or 'Frequently'.

The lesson the authors emphasize: "The score said DROP F1 20.7% — a weak model. The generations said ballo — a bug." Those are different problems. Print the generations next to the score.

Why This Matters: The Data Is Doing the Work

Every corpus in the mixture (Nemotron family) is a large-model artifact — quality classification, rephrasing, generated reasoning traces. Training a tiny model on them is implicit distillation.

When models this small were last studied seriously, such corpora did not exist. Past results about what tiny models can't do were partly measuring a data distribution.

The consequence: hold the model and hardware fixed, and a one-core budget buys more capability each year as curation improves. The cheapest experiment is to re-run the same config on the next corpus release.

Questions Readers Ask

Q: Is 6,616 tok/s fast? Not GPU-fast, but for batch data processing on a core that costs nothing to borrow, it's a meaningful target — the whole point is the roofline of one core.

Q: Is the model usable? It's a research artifact: passage-grounded extraction works, arithmetic doesn't.

Q: Did Claude Code really do it all? Per the author-contribution section: Claude Opus 5 ran experiments, implemented the architecture, diagnosed failures, and drafted the paper; Diamos set direction, made design decisions, and reviewed.

Q: Can I try it? Yes — gdiamos/amx-reasoning-v1-instruct on Hugging Face, Apache-2.0. Note: AutoModelForCausalLM won't work; use the bundled example.py with the vocab mask and greedy decoding.

Summary

One core, 9 hours, 3.65M parameters → basic reasoning emerges earlier than assumed
The four failures are the real teaching material — none visible in the loss curve
Data curation quality is now an axis of progress independent of compute

The paper and code are fully public. Start from example.py.

Source: Gregory Diamos, Claude Opus 5 — Outrageously Small Neural Networks (Sep 7, 2026)

PIM-DRAM-LLM Guide 2026: Can Ordinary DRAM Run LLMs Fast? Processing-in-Memory Technology Explained