DeepSeek, released 2024-12-26. Sparse MoE decoder, 671B total, 37B active, 128K context. Attention: MLA.
Sparse MoEDeepSeek 🇨🇳 · January 20, 2025
DeepSeek R1 (671B)
Reasoning-tuned DeepSeek model built on the V3 architecture rather than a new base design.
Key detail
Architecture matches DeepSeek V3, including the MTP-1 path; the main change is the reasoning-oriented training recipe.Total params
671B
Active / token
37B
Context
128K
Fact sheet
- Scale
- 671B total, 37B active (5.5% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- MLA
- Layer mix
- 61 MLA
- KV cache / token (bf16)
- 68.6 KiB · Low
- License
- MIT License
- Released
Efficiency
Parameters active per token
5.5%
37B of 671B weights run for each token.
AA Intelligence Index v4.3.2
Source11.4
- General
- 22.2
- Scientific
- 4.5
- Coding
- 19.2
- Agents
- 0
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
DeepSeek, released 2025-12-01. Sparse MoE decoder, 671B total, 37B active, 128K context. Attention: MLA with DeepSeek Sparse Attention.
Z.ai, released 2026-02-11. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.
Z.ai, released 2026-04-07. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.