Arcee AI, released 2026-01-27. Sparse MoE decoder, 400B total, 13B active, 512K context. Attention: GQA with gated attention and 3:1 sliding-window/global attention.
Sparse MoEMeta 🇺🇸 · April 5, 2025
Llama 4 Maverick (400B)
Meta's large MoE follows the DeepSeek V3 playbook but with a more conventional attention stack.
Key detail
Alternates dense and MoE blocks and uses fewer, larger experts than DeepSeek V3.Total params
400B
Active / token
17B
Context
1M
Fact sheet
- Scale
- 400B total, 17B active (4.3% active)
- Context window
- 1,000,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA
- Layer mix
- 36 chunked + 12 full GQA
- KV cache / token (bf16)
- 192 KiB · High
- License
- Llama 4 Community License Agreement
- Released
Efficiency
Parameters active per token
4.3%
17B of 400B weights run for each token.
AA Intelligence Index v4.3.2
Source9.3
- General
- 18.8
- Scientific
- 2.5
- Coding
- 15.9
- Agents
- 0
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
MiniMax, released 2026-06-13. Sparse MoE decoder, 428B total, 23B active, 1M context. Attention: GQA with QK-Norm and MiniMax Sparse Attention.
MiniMax, released 2026-10-19. Sparse MoE decoder, 428B total, 23B active, 1M context. Attention: GQA with QK-Norm and MiniMax Sparse Attention.
Z.ai, released 2025-07-28. Sparse MoE decoder, 355B total, 32B active, 128K context. Attention: GQA with QK-Norm.