Tencent, released 2026-08-28. Sparse MoE decoder, 770B total, 49B active, 1M context. Attention: Gated MLA with DeepSeek Sparse Attention and IndexCache.
Sparse MoEMistral AI 🇫🇷 · October 29, 2026
Mistral Large 4 (1T)
Mistral's trillion-parameter multimodal MoE combines MLA with sparse token selection and cross-layer KV sharing.
Key detail
Selects the top 2,048 tokens, reuses sparse indices across layers, and alternates three LeRoPE layers with one NoPE layer.Total params
1T
Active / token
52B
Context
1M
Fact sheet
- Scale
- 1T total, 52B active (5.2% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- MLA with modified DeepSeek Sparse Attention, index reuse, and KV sharing
- Layer mix
- 72 MLA/DSA
- KV cache / token (bf16)
- 80.75 KiB · Moderate
- License
- Mistral Ecosystem Open-Weight License
- Released
Efficiency
Parameters active per token
5.2%
52B of 1T weights run for each token.
AA Intelligence Index v5.0
Source26
- Knowledge Work
- 40.6
- Coding
- 18.9
- Science
- 13
- General
- 31.7
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Z.ai, released 2026-06-17. Sparse MoE decoder, 744B total, 40B active, 1M context. Attention: MLA with DeepSeek Sparse Attention and IndexShare.
Z.ai, released 2026-08-26. Hybrid MoE decoder, 320B total, 18B active, 1M context. Attention: 3:1 Kimi Delta Attention and MLA with DeepSeek Sparse Attention.
Z.ai, released 2026-02-11. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.