MiniMax, released 2026-10-19. Sparse MoE decoder, 428B total, 23B active, 1M context. Attention: GQA with QK-Norm and MiniMax Sparse Attention.
Sparse MoEMiniMax 🇨🇳 · June 13, 2026
MiniMax M3 (428B)
Native multimodal MiniMax model that adds sparse attention to an M2-style MoE backbone for 1M-token context.
Key detail
Uses three dense prefix layers, then 57 sparse-attention MoE layers with 128 routed experts and 4 routed plus 1 shared expert active per token.Total params
428B
Active / token
23B
Context
1M
Fact sheet
- Scale
- 428B total, 23B active (5.4% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm and MiniMax Sparse Attention
- Layer mix
- 3 full GQA + 57 MiniMax Sparse Attention
- KV cache / token (bf16)
- 120 KiB · Moderate
- License
- MiniMax Community License
- Released
Efficiency
Parameters active per token
5.4%
23B of 428B weights run for each token.
AA Intelligence Index v5.0
Source18.7
- Knowledge Work
- 23.9
- Coding
- 13.9
- Science
- 10.9
- General
- 26.1
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
Reflection, released 2026-10-16. Sparse MoE decoder, 501B total, 23B active, 1M context. Attention: Gated GQA with QK-Norm and sliding-window/global attention.
Z.ai, released 2025-07-28. Sparse MoE decoder, 355B total, 32B active, 128K context. Attention: GQA with QK-Norm.