Z.ai, released 2025-12-22. Sparse MoE decoder, 355B total, 32B active, 198K context. Attention: GQA with QK-Norm.
Sparse MoEZ.ai 🇨🇳 · July 28, 2025
GLM-4.5 (355B)
Agent-oriented instruction/reasoning hybrid that borrows DeepSeek's dense-prefix MoE layout.
Key detail
Starts with three dense layers before MoE routing, keeps a shared expert, and uses MTP during training.Total params
355B
Active / token
32B
Context
128K
Fact sheet
- Scale
- 355B total, 32B active (9% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 92 GQA
- KV cache / token (bf16)
- 368 KiB · Very high
- License
- MIT License
- Released
Efficiency
Parameters active per token
9.0%
32B of 355B weights run for each token.
AA Intelligence Index v4.3.2
Source12.8
- Scientific
- 6.5
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Tencent, released 2026-04-23. Sparse MoE decoder, 295B total, 21B active, 256K context. Attention: GQA with QK-Norm.
MiniMax, released 2025-10-23. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm and partial RoPE.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.