Xiaomi, released 2026-09-21. Sparse MoE decoder, 309B total, 15B active, 1M context. Attention: GQA with sliding-window/global attention.
Sparse MoENaiveAI 🇨🇳 · September 27, 2026
Naive-N0.5-Flash (309B)
MiMo-V2.5-derived sparse MoE that replaces its global attention layers with GQA-based DeepSeek Sparse Attention for a native 1M-token context.
Key detail
Pairs a 128-token local window with top-2,048 DSA selection, four DSA KV heads, and a 16-head indexer. Uses 256 routed experts with top-8 routing and no full-attention layers.Total params
309B
Active / token
16B
Context
1M
Fact sheet
- Scale
- 309B total, 15.5B active (5.0% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with sliding-window attention and DeepSeek Sparse Attention
- Layer mix
- 39 sliding-window + 9 GQA/DSA
- KV cache / token (bf16)
- 219.75 KiB · High
- License
- MIT License
- Released
Efficiency
Parameters active per token
5.0%
16B of 309B weights run for each token.
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.
StepFun, released 2026-02-01. Sparse MoE decoder, 196B total, 11B active, 256K context. Attention: GQA with 3:1 sliding-window attention.