Poolside, released 2026-04-28. Sparse MoE decoder, 33B total, 3B active, 128K context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.
Sparse MoESarvam AI 🇮🇳 · March 3, 2026
Sarvam (30B)
Reasoning-oriented Indian-language sparse MoE that keeps GQA at the smaller size.
Key detail
Large vocabulary and strong Indic language support paired with a reasoning-focused sparse MoE design.Total params
30B
Active / token
2.4B
Context
128K
Fact sheet
- Scale
- 30B total, 2.4B active (8% active)
- Context window
- 131,072 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 19 GQA
- KV cache / token (bf16)
- 19 KiB · Very low
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
8.0%
2.4B of 30B weights run for each token.
AA Intelligence Index v4.3.2
Source6.6
- Scientific
- 3.9
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Poolside, released 2026-07-02. Sparse MoE decoder, 33B total, 3B active, 256K context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.
Google, released 2026-04-02. Sparse MoE decoder, 25B total, 3.8B active, 256K context. Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention.
Recursive, released 2026-10-18. Sparse MoE decoder, 40B total, 2.5B active, 256K context. Attention: GQA with QK-Norm, learned attention sinks, and sliding-window/global attention.