MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.
Sparse MoEStepFun 🇨🇳 · February 1, 2026
Step 3.5 Flash (196B)
Throughput-oriented MoE model that stays competitive with much larger DeepSeek-style systems.
Key detail
Uses MTP-3 during both training and inference for unusually high throughput.Total params
196B
Active / token
11B
Context
256K
Fact sheet
- Scale
- 196B total, 11B active (5.6% active)
- Context window
- 262,144 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with 3:1 sliding-window attention
- Layer mix
- 36 sliding-window + 12 global
- KV cache / token (bf16)
- 192 KiB · High
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
5.6%
11B of 196B weights run for each token.
AA Intelligence Index v4.3.2
Source17
- Scientific
- 13.4
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Xiaomi, released 2026-09-21. Sparse MoE decoder, 309B total, 15B active, 1M context. Attention: GQA with sliding-window/global attention.
StepFun, released 2026-09-20. Sparse MoE decoder, 600B total, 27B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/sparse attention.
Thinking Machines Lab, released 2026-07-15. Sparse MoE decoder, 975B total, 41B active, 1M context. Attention: GQA with QK-Norm and learned relative-position bias in a 5:1 sliding-window/global pattern.