Z.ai, released 2025-07-28. Sparse MoE decoder, 355B total, 32B active, 128K context. Attention: GQA with QK-Norm.
Sparse MoEZ.ai 🇨🇳 · December 22, 2025
GLM-4.7 (355B)
Immediate GLM predecessor that stays closer to the older GLM-4.5 style before the MLA shift.
Key detail
Serves as the pre-MLA, pre-sparse-attention baseline and uses an MTP-3-style setup.Total params
355B
Active / token
32B
Context
198K
Fact sheet
- Scale
- 355B total, 32B active (9% active)
- Context window
- 202,752 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 92 GQA
- KV cache / token (bf16)
- 368 KiB · Very high
- License
- MIT License
- Released
Efficiency
Parameters active per token
9.0%
32B of 355B weights run for each token.
AA Intelligence Index v4.3.2
Source17.4
- Scientific
- 3.2
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Tencent, released 2026-04-23. Sparse MoE decoder, 295B total, 21B active, 256K context. Attention: GQA with QK-Norm.
MiniMax, released 2025-10-23. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm and partial RoPE.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.