LLM Architecture Gallery: A Free Visual Catalog Comparing 43 AI Model Internals
Sebastian Raschka's LLM Architecture Gallery lets you visually compare the architectures of 43 AI models including ChatGPT, DeepSeek, and Llama — all for free. It covers key 2026 trends like MoE and MLA in an intuitive, color-coded format.
Llama, DeepSeek, Qwen, GPT — you've heard the names, but what actually makes them different? Sebastian Raschka, one of AI's most prominent educators, has released the LLM Architecture Gallery — a free visual catalog that lets you compare the internal architectures of 43 LLMs side by side. Within days of its March 14 launch, it earned 216 points on Hacker News and widespread attention in the developer and research community.
LLM Architecture Gallery — AI Model "Anatomy Charts" in One Place
Behind everyday AI services like ChatGPT, Claude, and Gemini are Large Language Models (LLMs). While they look similar on the surface, their internal designs — their "architectures" — vary significantly.
Raschka visualizes these differences with color-coded diagrams, making them intuitively understandable even for non-experts. If you're curious about AI basics, our AI fundamentals guide is a great starting point.
▲ 43 AI model architectures laid out for side-by-side comparison. Each model has distinct colors for quick visual differentiation.
From Dense to MoE — 43 Models Across 4 Architecture Types
The gallery's 43 models fall into four broad categories:
Dense Models
Llama 3, OLMo 2/3, Gemma 3, SmolLM3, etc.
The traditional approach — activate all parameters at once. Common in small to mid-size models.
Mixture of Experts (MoE)
DeepSeek V3, Llama 4 Maverick, Qwen3 235B, GPT-OSS, etc.
Activate only the relevant "experts" per query. For example, DeepSeek V3 has 671B total parameters but only activates 37B at a time, dramatically reducing compute costs.
Hybrid Attention
Qwen3 Next, Kimi Linear, Nemotron 3 series, etc.
Mix fast scanning with deep reading for long documents — balancing speed and accuracy.
Trillion-Scale Models
Kimi K2 (1T), GLM-5 (744B), Grok 2.5 (270B)
Models with hundreds of billions to a trillion parameters. This gallery is the first to compare their internals visually.
▲ Detailed structural comparison of major models including Llama, Qwen, SmolLM, DeepSeek, and Kimi K2.
7 Years of Transformer Evolution — A Surprising Discovery
The most interesting finding: line them all up, and they're more alike than you'd expect.
A Hacker News commenter noted: "Surprisingly, the main differences are just layer sizes." Raschka himself asks: "Has the fundamental Transformer architecture really changed since GPT seven years ago, or are we refining the same foundation?"
The implication is clear: AI's dramatic advances came from revolutions in data, training methods, and scale — not architecture. Like a great chef given better ingredients and a bigger kitchen.
3 Key LLM Design Trends for 2026
1. MoE Architecture Goes Mainstream
Triggered by DeepSeek V3 — activating only a subset of parameters means even trillion-scale models have small-model compute costs. This enabled open-source players to compete in the large model race.
2. MLA (Multi-head Latent Attention) Spreads
Compresses the KV cache (memory for remembering prior conversation) — pioneered by DeepSeek, now adopted by Kimi K2, GLM-5, and others. Enables longer conversations on the same hardware.
3. Hybrid Attention Experiments
For long documents, comparing every word to every other is too slow. Models like Qwen3 Next and Kimi Linear experiment with mixing "fast scan" and "deep read" modes.
▲ 10 open-source AI models released in early 2026 with architecture and benchmark comparisons.
Sebastian Raschka — Author of "Build an LLM From Scratch"
Sebastian Raschka is one of the most influential AI/ML educators. A former University of Wisconsin professor, he authored the bestselling "Build an LLM From Scratch." HN commenters called it "the best resource for understanding LLMs."
Browse the Gallery for Free
The gallery is completely free with no sign-up — just open it in your browser:
Click any diagram to see detailed fact sheets (model size, training method, paper links, etc.).
If you've wondered "what's actually different between all these AI models?" — this gallery provides the most intuitive answer. Also available as a poster (14,570×12,490 px, 56MB) for your office or study room.
For a systematic journey from AI basics to practical use, check out our free learning guide.
Related Content — More AI News | Free Learning Guide
Sources
Stay updated on AI news
Simple explanations of the latest AI developments