NVIDIA, released 2026-08-11. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with six GQA layers.
Hybrid MoENVIDIA 🇺🇸 · December 4, 2025
Nemotron 3 Nano (30B-A3B)
NVIDIA's Nano model is the most extreme transformer-state-space hybrid in the gallery.
Key detail
Interleaves Mamba-2 and MoE blocks, using attention only sparingly.Total params
30B
Active / token
3B
Context
1M
Fact sheet
- Scale
- 30B total, 3B active (10% active)
- Context window
- 1,000,000 tokens
- Decoder type
- Hybrid MoE
- Attention
- Mostly Mamba-2 with a few GQA layers
- Layer mix
- 6 GQA + 23 Mamba-2 + 23 MoE
- KV cache / token (bf16)
- 6 KiB · Very low
- License
- NVIDIA Nemotron Open Model License
- Released
Efficiency
Parameters active per token
10%
3B of 30B weights run for each token.
AA Intelligence Index v4.3.2
Source6.8
- Scientific
- 2.3
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
SOOFI Consortium, released 2026-07-10. Hybrid MoE decoder, 32B total, 3.2B active, 1M context. Attention: Mostly Mamba-2 with 6 GQA layers and no positional embeddings (NoPE).
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
NVIDIA, released 2026-03-11. Hybrid MoE decoder, 120B total, 12B active, 1M context. Attention: Mostly Mamba-2 with a few GQA layers.