Aleph Alpha, released 2026-10-03. Sparse MoE decoder, 78B total, 3.5B active, 256K context. Attention: GQA with QK-Norm and 4:1 sliding-window/global attention.
Sparse MoERecursive 🇺🇸 · October 18, 2026
Recursive Flint (40B-A2.5B)
Recursive's coding model uses two dense FFN layers followed by 47 MoE layers. The active-parameter count includes the output head.
Key detail
Uses top-6 routing over 256 experts plus one shared expert, with RoPE in local layers, NoPE in global layers, and a local window covering the current token and 512 preceding tokens.Total params
40B
Active / token
2.5B
Context
256K
Fact sheet
- Scale
- 39.8B total, 2.5B active (6.3% active)
- Context window
- 262,144 configured; 512k supported tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm, learned attention sinks, and sliding-window/global attention
- Layer mix
- 35 sliding-window + 14 global GQA
- KV cache / token (bf16)
- 98 KiB · Moderate
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
6.3%
2.5B of 40B weights run for each token.
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
Reflection, released 2026-10-16. Sparse MoE decoder, 501B total, 23B active, 1M context. Attention: Gated GQA with QK-Norm and sliding-window/global attention.
Meta, released 2026-10-15. Sparse MoE decoder, 816B total, 42B active, 1M context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.