Z.ai, released 2026-02-11. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.
Sparse MoEDeepSeek 🇨🇳 · December 1, 2025
DeepSeek V3.2 (671B)
DeepSeek's successor keeps the V3 template but adds sparse attention to cut long-context costs.
Key detail
Adds sparse attention while retaining the DeepSeek V3-style MTP path.Total params
671B
Active / token
37B
Context
128K
Fact sheet
- Scale
- 671B total, 37B active (5.5% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- MLA with DeepSeek Sparse Attention
- Layer mix
- 61 MLA
- KV cache / token (bf16)
- 68.6 KiB · Low
- License
- MIT License
- Released
Efficiency
Parameters active per token
5.5%
37B of 671B weights run for each token.
AA Intelligence Index v4.3.2
Source16
- Scientific
- 6
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Z.ai, released 2026-04-07. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.
Tencent, released 2026-08-28. Sparse MoE decoder, 770B total, 49B active, 1M context. Attention: Gated MLA with DeepSeek Sparse Attention and IndexCache.
DeepSeek, released 2026-04-24. Sparse MoE decoder, 284B total, 13B active, 1M context. Attention: MLA-style CSA/HCA with mHC.