DeepSeek, released 2024-12-26. Sparse MoE decoder, 671B total, 37B active, 128K context. Attention: MLA.
Sparse MoEMistral AI 🇫🇷 · December 2, 2025
Mistral Large 3 (673B)
Mistral's new flagship effectively adopts the DeepSeek architecture and retunes the expert sizes.
Key detail
Near-clone of DeepSeek V3 with larger experts, fewer routed experts, and multimodal support.Total params
673B
Active / token
41B
Context
256K
Fact sheet
- Scale
- 673B total, 41B active (6.1% active)
- Context window
- 262,144 tokens
- Decoder type
- Sparse MoE
- Attention
- MLA
- Layer mix
- 61 MLA
- KV cache / token (bf16)
- 68.6 KiB · Low
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
6.1%
41B of 673B weights run for each token.
AA Intelligence Index v4.3.2
Source9.3
- General
- 17.1
- Scientific
- 2.1
- Coding
- 18.3
- Agents
- 0.2
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
DeepSeek, released 2025-01-20. Sparse MoE decoder, 671B total, 37B active, 128K context. Attention: MLA.
DeepSeek, released 2025-12-01. Sparse MoE decoder, 671B total, 37B active, 128K context. Attention: MLA with DeepSeek Sparse Attention.
Z.ai, released 2026-02-11. Sparse MoE decoder, 744B total, 40B active, 198K context. Attention: MLA with DeepSeek Sparse Attention.