InclusionAI, released 2026-08-04. Hybrid MoE decoder, 124B total, 5.1B active, 256K context. Attention: 5:1 Kimi Delta Attention and gated MLA.
Sparse hybridAlibaba Cloud 🇨🇳 · September 9, 2025
Qwen3 Next (80B-A3B)
Efficiency-focused Qwen refresh that swaps standard attention for a DeltaNet-attention hybrid.
Key detail
Adds many more experts, a shared expert, native 262k context, and MTP for speculative decoding.Total params
80B
Active / token
3B
Context
256K
Fact sheet
- Scale
- 80B total, 3B active (3.8% active)
- Context window
- 262,144 tokens
- Decoder type
- Sparse hybrid
- Attention
- 3:1 Gated DeltaNet and Gated Attention
- Layer mix
- 12 gated attention + 36 DeltaNet
- KV cache / token (bf16)
- 24 KiB · Very low
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
3.8%
3B of 80B weights run for each token.
AA Intelligence Index v4.3.2
Source9.6
- Scientific
- 3.8
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Alibaba Cloud, released 2026-08-26. Hybrid MoE decoder, 125B total, 6B active, 256K context. Attention: 3:1 Gated DeltaNet and Qwen Sparse Attention (QSA).
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
InclusionAI, released 2026-10-13. Hybrid MoE decoder, 560B total, 25B active, 1M context. Attention: 7:1 Kimi Delta Attention and gated MLA.