AI Training Optimizer Muon Kills 25% of Neurons — Aurora Fix

More than 1 in 4 neurons trained with Muon — one of the most widely-used AI training optimizers — quietly die during warmup and never recover. That's the finding buried in Import AI #457, the weekly research newsletter from Jack Clark, co-founder of Anthropic. The fix has a name: Aurora — and it scores 10 MMLU benchmark points higher. If you train or fine-tune AI models anywhere in your AI automation workflow, this matters today.

The Silent Killer: How Muon Optimizer Damages AI Training

Muon is an optimizer (a mathematical algorithm that adjusts a neural network's internal settings — called "weights" — during the training process) that became popular in AI research as a faster alternative to the industry-standard AdamW. Labs and researchers adopted it because early results looked promising. But Tilde Research just published a paper revealing a serious flaw that has been hiding in plain sight.

The problem lies in how Muon computes updates. It creates what researchers call row-norm anisotropy on tall matrices (an imbalance in how strongly it updates different rows of the network's weight tables). This imbalance damages neurons inside MLP layers (the dense, fully-connected layers that handle the bulk of a model's reasoning and memory).

The damage follows a consistent, repeatable pattern:

• At the start of training, neurons are healthy — all contributing equally to the network's output
• Around training step 500 (during the learning rate warmup — a phase where the optimizer gradually ramps up its update strength), a large fraction of neurons suddenly collapse
• Dead neurons never recover, even after 24,000 training steps
• More than 1 in 4 MLP neurons (>25%) are permanently destroyed

The analogy: imagine a factory where 1 in 4 workers freezes up on day one and never participates again — but management keeps counting them as active capacity. Every model trained with Muon may have been running well below its potential, with no error message and no warning sign.

Aurora: The Fix That Scores 10 Points Higher

Aurora is Tilde Research's direct solution to the Muon problem. It's designed specifically for rectangular matrices (the most common weight-table shape inside transformer models — the architecture used by Claude, ChatGPT, and virtually every modern AI assistant) using a technique called leverage-aware updates (a method that checks how much each neuron is currently contributing before deciding how hard to push it — preventing the catastrophic over-correction that kills Muon neurons).

The team trained 1.1 billion-parameter transformer models on approximately 100 billion tokens of text. Here are the final training loss results (lower is better):

• Aurora: 2.26 — best performance
• Muon: 2.31
• NorMuon (a partial community fix): 2.33

On the MMLU benchmark (Massive Multitask Language Understanding — a 57-subject test covering medicine, law, history, and more, used to measure how much factual knowledge a model has retained), Aurora's advantage is even clearer: +10 percentage points over Muon in the most memorization-heavy scenarios.

Independent confirmation came from Pleias, a European open-source AI research group, which validated Aurora's advantage at 600 million parameters using entirely separate infrastructure. Same conclusion: Aurora outperforms Muon.

One important caveat that honest reporting requires: Aurora has not yet been directly tested against AdamW (the industry gold standard optimizer developed by Google, still used by default at most major labs). The Tilde Research team themselves acknowledge that "no one has conclusively beaten AdamW yet" despite years of competing attempts. Aurora is a clear fix for Muon users — and a strong candidate for broader adoption — but that final head-to-head comparison still needs to happen.

14,000 GPU Hours: AI Automation Agents Push Research Limits

The second major story in Import AI #457 is a glimpse at automated AI research — and it's simultaneously impressive and humbling. Prime Intellect, a distributed AI research group, ran an experiment: can AI agents (autonomous programs that plan and execute multi-step technical tasks without human direction at each step) conduct meaningful machine learning research on their own?

The challenge chosen: the nanoGPT speedrun — a community competition to train a 124 million-parameter language model as efficiently as possible using only public methods. The agents deployed: GPT 5.5 Codex and Claude Code Opus 4.7.

What the agents accomplished over the full experiment:

• ~10,000 total training runs completed autonomously
• ~14,000 H200 GPU hours burned (H200 is NVIDIA's current top-tier AI training chip, priced at roughly $3–5 per hour in cloud deployments)
• Both agents beat every previous human baseline on the benchmark
• New records were set in every experimental session

The agents genuinely excelled at optimizer search, hyperparameter sweeps (systematically testing thousands of numerical configurations), and stacking known techniques together. But they hit a clear ceiling when asked for anything original. They couldn't generate genuinely new ideas, they tended to add components rather than elegantly simplify, and they lacked accurate mental models of how the parts they were combining actually interact with each other.

Prime Intellect described the results as "pretty yolo" and explicitly called them "a lower bound of what's possible." Translation: the agents are already remarkable at high-throughput experimentation. They are not yet capable of the kind of insight that moves a field forward in a genuinely new direction. The gap between "runs 10,000 experiments" and "thinks of the right experiment to run" remains wide.

Beyond Safety: What Should AI Actually Do for People?

The third thread in #457 may be the most philosophically significant. A research paper signed by authors from 11 institutions — including Oxford, Google DeepMind, OpenAI, Anthropic, Stanford, and UCLA — introduces a concept the AI safety field has mostly sidestepped: once AI is safe, what should it actively be for?

They call the answer positive alignment (a design goal for AI systems that moves beyond preventing harm toward actively supporting human flourishing — the difference between "do no damage" and "actively help people live better").

Traditional AI safety asks: how do we stop AI from doing bad things? Positive alignment asks: once that's solved, what genuinely good things should AI do?

The paper's four governing principles:

• Pluralistic — AI supports many different definitions of a good life, not a single standardized one
• Polycentric — No single entity (not a government, not a corporation, not a lab) decides unilaterally what AI should optimize for
• User-authored — You define what the AI does for you; it doesn't impose a vision onto you
• Context-sensitive — The right outcome differs by culture, community, and individual situation

Clark highlights one of the paper's most important contributions: it makes the value choices embedded in AI design explicit and debatable. Current AI safety framing tends to hide political and ethical decisions inside technical language. Positive alignment demands those decisions be named, discussed, and governed openly — which is the prerequisite for any democratic oversight of AI at all. If you want to understand the direction AI design philosophy is heading, explore our AI automation learning guides before the decisions are already made.

Clark's 60% Prediction — and Why the Timeline Is Shorter Than You Think

Jack Clark has been publishing Import AI since 2016. More than 461 weekly issues covering the transition from academic curiosity to the most competitive technology race in modern history. In a recent Hacker News discussion tied to issue #455, he put a specific probability on where this ends: a 60%+ chance that AI systems will be conducting their own research and development — autonomously — by 2029. Not assisting researchers. Running the full loop themselves.

The three stories in #457 are each a data point on that trajectory: an optimizer has been quietly degrading models for months or years (we still don't fully control what our training tools do), AI agents just ran 10,000 unsupervised experiments and beat every human benchmark (the automation at small scale is already here), and a coalition of the world's leading AI labs is drafting a philosophy for what AI should accomplish after safety is solved (they are planning for a post-problem world).

All three threads converge on the same uncomfortable truth: the field is moving faster than its governance frameworks, its tooling quality controls, and its positive vision can keep up with.

Import AI is free and published every Monday. You can subscribe on Substack or read the full archive at jack-clark.net. Issues #455 through #457 give you the current state of play in roughly 45 minutes of reading — and they're the most information-dense 45 minutes you can spend tracking where AI actually is right now.

Related Content — Get Started | Guides | More News