InclusionAI, released 2026-10-13. Hybrid MoE decoder, 560B total, 25B active, 1M context. Attention: 7:1 Kimi Delta Attention and gated MLA.
Sparse hybridInclusionAI 🇨🇳 · August 4, 2026
Ling 3.0 Flash (124B)
Hybrid MoE that alternates five Kimi Delta Attention layers with one gated MLA layer.
Key detail
Uses fine-grained diagonal KDA gating, top-8 routing over 512 experts plus one shared expert, and one MTP layer.Total params
124B
Active / token
5.1B
Context
256K
Fact sheet
- Scale
- 124B total, 5.1B active (4.1% active)
- Context window
- 262,144 tokens
- Decoder type
- Sparse hybrid
- Attention
- 5:1 Kimi Delta Attention and gated MLA
- Layer mix
- 35 Kimi Delta Attention + 7 gated MLA + 1 MTP layer
- KV cache / token (bf16)
- 7.9 KiB · Very low
- License
- MIT License
- Released
Efficiency
Parameters active per token
4.1%
5.1B of 124B weights run for each token.
AA Intelligence Index v4.3.2
Source20.1
- General
- 29.3
- Scientific
- 12.7
- Coding
- 21
- Agents
- 15.4
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Z.ai, released 2026-08-26. Hybrid MoE decoder, 320B total, 18B active, 1M context. Attention: 3:1 Kimi Delta Attention and MLA with DeepSeek Sparse Attention.
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
Moonshot AI, released 2026-07-27. Hybrid MoE decoder, 2.8T total, 104B active, 1M context. Attention: 3:1 Kimi Delta Attention and Gated MLA with NoPE.