Google, released 2025-03-11. Dense decoder, 27B parameters, 128K context. Attention: GQA with QK-Norm and 5:1 sliding-window/global attention.
DenseMistral AI 🇫🇷 · March 18, 2025
Mistral Small 3.1 (24B)
Fast dense 24B model that drops the sliding-window setup used in older Mistral releases.
Key detail
Latency-focused design with a smaller KV cache and fewer layers than Gemma 3 27B.Parameters
24B
Context
128K
KV / token
160 KiB
Fact sheet
- Scale
- 24B parameters
- Context window
- 128,000 tokens
- Decoder type
- Dense
- Attention
- Standard GQA
- Layer mix
- 40 GQA
- KV cache / token (bf16)
- 160 KiB · Moderate
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
100%
Dense: every weight runs for every token.
AA Intelligence Index v4.3.2
Source7.1
- General
- 12.8
- Scientific
- 2.2
- Coding
- 13.9
- Agents
- 0.2
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Meta, released 2026-08-10. Dense decoder, 30B parameters, 128K context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.
Google, released 2026-04-02. Dense decoder, 31B parameters, 256K context. Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention.
Ai2, released 2025-11-20. Dense decoder, 32B parameters, 64K context. Attention: GQA with QK-Norm and 3:1 sliding-window/global attention.