StepFun, released 2026-02-01. Sparse MoE decoder, 196B total, 11B active, 256K context. Attention: GQA with 3:1 sliding-window attention.
Sparse MoEZ.ai 🇨🇳 · July 28, 2025
GLM-4.5-Air (106B)
Compact GLM-4.5 companion that keeps the same agent-oriented sparse MoE recipe at a smaller serving footprint.
Key detail
Shrinks the GLM-4.5 layout to 46 layers, keeps the dense warmup, and inherits the MTP training recipe.Total params
106B
Active / token
12B
Context
128K
Fact sheet
- Scale
- 106B total, 12B active (11.3% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA
- Layer mix
- 46 GQA
- KV cache / token (bf16)
- 184 KiB · High
- License
- MIT License
- Released
Efficiency
Parameters active per token
11%
12B of 106B weights run for each token.
AA Intelligence Index v4.3.2
Source11.1
- Scientific
- 3.5
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
MiniMax, released 2025-10-23. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm and partial RoPE.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.
MiniMax, released 2026-03-18. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.