Recursive, released 2026-10-18. Sparse MoE decoder, 40B total, 2.5B active, 256K context. Attention: GQA with QK-Norm, learned attention sinks, and sliding-window/global attention.
Sparse MoECohere 🇨🇦 · June 5, 2026
North Mini Code (30B-A3B)
Cohere's compact agentic coding MoE adapts the parallel Cohere2-MoE block to a 30B total, 3B active model.
Key detail
Runs attention and MoE in parallel, with RoPE in sliding-window layers, NoPE in global layers, and top-8 routing over 128 experts.Total params
30B
Active / token
3B
Context
256K
Fact sheet
- Scale
- 30B total, 3B active (10% active)
- Context window
- 256K input, 64K output tokens
- Decoder type
- Sparse MoE
- Attention
- 8:1 GQA with 3:1 sliding-window/global attention
- Layer mix
- 36 sliding-window + 13 global GQA/MoE
- KV cache / token (bf16)
- 98 KiB · Moderate
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
10%
3B of 30B weights run for each token.
AA Intelligence Index v4.3.2
Source9.9
- General
- 15.9
- Scientific
- 5.7
- Coding
- 19.6
- Agents
- 0.2
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Aleph Alpha, released 2026-10-03. Sparse MoE decoder, 78B total, 3.5B active, 256K context. Attention: GQA with QK-Norm and 4:1 sliding-window/global attention.
Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
Reflection, released 2026-10-16. Sparse MoE decoder, 501B total, 23B active, 1M context. Attention: Gated GQA with QK-Norm and sliding-window/global attention.