DeepSeek, released 2026-04-24. Sparse MoE decoder, 1.6T total, 49B active, 1M context. Attention: MLA-style CSA/HCA with mHC.
Sparse MoEDeepSeek 🇨🇳 · April 24, 2026
DeepSeek V4-Flash (284B)
DeepSeek's efficient V4 preview keeps the million-token architecture while reducing the MoE scale to 284B parameters and 13B active parameters.
Key detail
Uses 256 experts, 6 routed plus 1 shared expert per token, hash-based routing, compressed attention, and the V4 MTP path.Total params
284B
Active / token
13B
Context
1M
Fact sheet
- Scale
- 284B total, 13B active (4.6% active)
- Context window
- 1,048,576 tokens
- Decoder type
- Sparse MoE
- Attention
- MLA-style CSA/HCA with mHC
- Layer mix
- 43 CSA/HCA
- KV cache / token (bf16)
- 5.4 KiB · Very low
- License
- MIT License
- Released
Efficiency
Parameters active per token
4.6%
13B of 284B weights run for each token.
AA Intelligence Index v4.3.2
Source34.3
- General
- 31.8
- Scientific
- 27.6
- Coding
- 31.2
- Agents
- 43.5
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Tencent, released 2026-08-28. Sparse MoE decoder, 770B total, 49B active, 1M context. Attention: Gated MLA with DeepSeek Sparse Attention and IndexCache.
Z.ai, released 2026-08-26. Hybrid MoE decoder, 320B total, 18B active, 1M context. Attention: 3:1 Kimi Delta Attention and MLA with DeepSeek Sparse Attention.
Motif Technologies, released 2026-07-21. Sparse MoE decoder, 314B total, 13B active, 256K context. Attention: GDLA with MLA-style latent projections, RoPE, and 3:1 sliding-window/full attention.