MiniMax, released 2026-06-13. Sparse MoE decoder, 428B total, 23B active, 1M context. Attention: GQA with QK-Norm and MiniMax Sparse Attention.
Sparse MoEMiniMax 🇨🇳 · October 19, 2026
MiniMax M3.1-Flash (428B)
MiniMax M3 successor with sparse attention in all 60 decoder layers, quantization-aware training, and a DSpark draft head for speculative decoding.
Key detail
All 60 layers use sparse attention; the first 3 retain dense FFNs, while the remaining 57 route to 4 of 128 experts plus 1 shared expert per token.Total params
428B
Active / token
23B
Context
1M
Fact sheet
- Scale
- 428B total, 23B active (5.4% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm and MiniMax Sparse Attention
- Layer mix
- 60 MiniMax Sparse Attention; 3 dense FFN + 57 MoE
- KV cache / token (bf16)
- 120 KiB · Moderate
- License
- MiniMax Community License
- Released
Efficiency
Parameters active per token
5.4%
23B of 428B weights run for each token.
AA Intelligence Index v5.0
Source28.9
- Knowledge Work
- 47.9
- Coding
- 22.8
- Science
- 14.4
- General
- 30.5
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
Reflection, released 2026-10-16. Sparse MoE decoder, 501B total, 23B active, 1M context. Attention: Gated GQA with QK-Norm and sliding-window/global attention.
Z.ai, released 2025-07-28. Sparse MoE decoder, 355B total, 32B active, 128K context. Attention: GQA with QK-Norm.