Computed from every card
Insights
How open model architectures have changed. Each chart has its data table underneath.
- Architectures
- 118
- Labs
- 44
- Concepts tracked
- 25
- Non-dense designs
- 75%
Releases per half-year
Stacked by decoder family
› Data table
| Period | Dense | Sparse MoE | Hybrid MoE | Dense hybrid | Recurrent |
|---|---|---|---|---|---|
| ≤2023 | 1 | 0 | 0 | 0 | 0 |
| 2024 H1 | 1 | 0 | 0 | 0 | 0 |
| 2024 H2 | 4 | 1 | 0 | 0 | 0 |
| 2025 H1 | 7 | 4 | 0 | 0 | 1 |
| 2025 H2 | 4 | 13 | 3 | 0 | 0 |
| 2026 H1 | 8 | 26 | 7 | 4 | 0 |
| 2026 H2 | 4 | 16 | 11 | 3 | 0 |
Decoder family share
Each half-year normalised to 100%
› Data table
| Period | Dense | Sparse MoE | Hybrid MoE | Dense hybrid | Recurrent |
|---|---|---|---|---|---|
| ≤2023 | 100% | 0% | 0% | 0% | 0% |
| 2024 H1 | 100% | 0% | 0% | 0% | 0% |
| 2024 H2 | 80% | 20% | 0% | 0% | 0% |
| 2025 H1 | 58% | 33% | 0% | 0% | 8% |
| 2025 H2 | 20% | 65% | 15% | 0% | 0% |
| 2026 H1 | 18% | 58% | 16% | 9% | 0% |
| 2026 H2 | 12% | 47% | 32% | 9% | 0% |
Most common concepts
Number of models using each design idea
› Data table
| Concept | Models |
|---|---|
| MoE | 79 |
| GQA | 78 |
| QK-Norm | 40 |
| SWA | 38 |
| MTP | 34 |
| MLA | 28 |
| Hybrid attention | 24 |
| Gated attention | 21 |
| NoPE | 18 |
| ShortConv | 10 |
| DSA | 10 |
| mHC | 6 |
| Latent MoE | 5 |
| MHA | 4 |
Sparsest MoE models
Lowest share of parameters active per token
› Data table
| Model | % active |
|---|---|
| DeepSeek V4.1-Flash | 1.4% |
| DeepSeek V4-Pro | 3.1% |
| Kimi K2 | 3.2% |
| Kimi K2.5 | 3.2% |
| Kimi K2.6 | 3.2% |
| Kimi K2.7 Code | 3.2% |
| Arcee AI Trinity Large 400B | 3.3% |
| Kimi K3 | 3.7% |
| Ember-1 | 3.7% |
| Qwen3 Next 80B-A3B | 3.8% |
| Xiaomi MiMo-V2.5-Pro 1.02T | 4.1% |
| Motif 3 Beta | 4.1% |
| Ling 3.0 Flash | 4.1% |
| Xiaomi MiMo-V2.6-Pro-RL 1.02T | 4.1% |
| Inkling | 4.2% |
Leanest KV cache at scale
KiB per token (bf16), models with ≥100B parameters
› Data table
| Model | KiB / token |
|---|---|
| DeepSeek V4.1-Flash | 3.1 |
| DeepSeek V4-Flash | 5.4 |
| DeepSeek V4-Pro | 7.7 |
| Ling 3.0 Flash | 7.9 |
| Nemotron 3 Super 120B-A12B | 8 |
| Nemotron 3.5 Super VL 120B-A12B | 8 |
| Ling 3.1 Flash | 10.1 |
| GLM-5.3-Flash | 11 |
| Ling 2.5 1T | 11.2 |
| Ling 2.6 1T | 11.2 |
| Nemotron 3 Ultra 550B-A55B | 12 |
| Mistral Small 4 | 22.5 |
| Qwen3.8-Flash-Next 125B-A6B | 24 |
| Kimi K3 | 27 |
| Ember-1 | 27 |
Most active labs
Models per organisation in the atlas
› Data table
| Lab | Models |
|---|---|
| Alibaba Cloud | 13 |
| 7 | |
| NVIDIA | 7 |
| Z.ai | 7 |
| DeepSeek | 6 |
| Meta | 6 |
| Moonshot AI | 6 |
| MiniMax | 5 |
| Xiaomi | 5 |
| InclusionAI | 4 |
| Liquid AI | 4 |
| Mistral AI | 4 |
| Ai2 | 3 |
| Cohere | 3 |
Where models come from
Country of the releasing organisation
› Data table
| Country | Models |
|---|---|
| China | 56 |
| United States | 46 |
| France | 4 |
| Canada | 3 |
| India | 2 |
| Other | 7 |
Longest context windows
Maximum supported tokens