Thinking Machines Lab, released 2026-07-15. Sparse MoE decoder, 975B total, 41B active, 1M context. Attention: GQA with QK-Norm and learned relative-position bias in a 5:1 sliding-window/global pattern.
Sparse omnimodal MoEXiaomi 🇨🇳 · September 21, 2026
Xiaomi MiMo-V2.6-Pro-RL (1.02T)
Omnimodal sparse MoE model with the MiMo-V2.5-Pro text backbone, mixed-domain reinforcement learning, and a five-layer DFlash speculative decoder.
Key detail
Uses 384 routed experts with top-8 routing, one dense prefix layer, no shared experts, and a 128-token sliding window.Total params
1.02T
Active / token
42B
Context
1M
Fact sheet
- Scale
- 1.02T total, 42B active (4.1% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse omnimodal MoE
- Attention
- GQA with 6:1 sliding-window/global attention
- Layer mix
- 60 sliding-window + 10 global
- KV cache / token (bf16)
- 350 KiB · Very high
- License
- MIT License
- Released
Efficiency
Parameters active per token
4.1%
42B of 1.02T weights run for each token.
AA Intelligence Index v5.0
Source29.9
- Knowledge Work
- 45.9
- Coding
- 28.3
- Science
- 17
- General
- 28
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
StepFun, released 2026-09-20. Sparse MoE decoder, 600B total, 27B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/sparse attention.
Xiaomi, released 2026-09-21. Sparse MoE decoder, 309B total, 15B active, 1M context. Attention: GQA with sliding-window/global attention.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.