Xiaomi, released 2026-09-21. Sparse MoE decoder, 1.02T total, 42B active, 1M context. Attention: GQA with 6:1 sliding-window/global attention.
Sparse MoEXiaomi 🇨🇳 · April 22, 2026
Xiaomi MiMo-V2.5-Pro (1.02T)
Trillion-parameter sparse MoE model for long-horizon agentic and software-engineering tasks.
Key detail
Scales the MiMo-V2.5 Pro stack to 70 layers, 384 routed experts, top-8 routing, and a 1M-token context window.Total params
1.02T
Active / token
42B
Context
1M
Fact sheet
- Scale
- 1.02T total, 42B active (4.1% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with 6:1 sliding-window/global attention
- Layer mix
- 60 sliding-window + 10 global
- KV cache / token (bf16)
- 350 KiB · Very high
- License
- MIT License
- Released
Efficiency
Parameters active per token
4.1%
42B of 1.02T weights run for each token.
AA Intelligence Index v4.3.2
Source26
- General
- 34.6
- Scientific
- 19.8
- Coding
- 25.3
- Agents
- 22.2
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Thinking Machines Lab, released 2026-07-15. Sparse MoE decoder, 975B total, 41B active, 1M context. Attention: GQA with QK-Norm and learned relative-position bias in a 5:1 sliding-window/global pattern.
Meta, released 2026-10-15. Sparse MoE decoder, 816B total, 42B active, 1M context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.
StepFun, released 2026-09-20. Sparse MoE decoder, 600B total, 27B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/sparse attention.