MiniMax, released 2025-10-23. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm and partial RoPE.
Sparse MoEAlibaba Cloud 🇨🇳 · April 28, 2025
Qwen3 (235B-A22B)
Large sparse Qwen variant that stays very close to DeepSeek V3 while removing the shared expert.
Key detail
High-capacity MoE design optimized for serving efficiency without a shared expert.Total params
235B
Active / token
22B
Context
128K
Fact sheet
- Scale
- 235B total, 22B active (9.4% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 94 GQA
- KV cache / token (bf16)
- 188 KiB · High
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
9.4%
22B of 235B weights run for each token.
AA Intelligence Index v4.3.2
Source8.3
- Scientific
- 2.1
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.
MiniMax, released 2026-03-18. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.
Tencent, released 2026-04-23. Sparse MoE decoder, 295B total, 21B active, 256K context. Attention: GQA with QK-Norm.