Z.ai, released 2025-07-28. Sparse MoE decoder, 106B total, 12B active, 128K context. Attention: GQA.
Sparse MoEPrime Intellect 🇺🇸 · November 26, 2025
INTELLECT-3 (106B)
Large-scale RL post-training of GLM-4.5-Air that keeps the compact 106B sparse MoE backbone intact.
Key detail
Keeps the GLM-4.5-Air architecture unchanged and shifts the capability profile through SFT plus large-scale RL.Total params
106B
Active / token
12B
Context
128K
Fact sheet
- Scale
- 106B total, 12B active (11.3% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA
- Layer mix
- 46 GQA
- KV cache / token (bf16)
- 184 KiB · High
- License
- MIT License
- Released
Efficiency
Parameters active per token
11%
12B of 106B weights run for each token.
AA Intelligence Index v4.3.2
Source10.6
- Scientific
- 6.7
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Sarvam AI, released 2026-03-03. Sparse MoE decoder, 105B total, 10B active, 128K context. Attention: MLA with KV LayerNorm and NoPE + RoPE.
OpenAI, released 2025-08-04. Sparse MoE decoder, 117B total, 5.1B active, 128K context. Attention: GQA with alternating sliding-window and global layers.
Poolside, released 2026-07-21. Sparse MoE decoder, 118B total, 8B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.