NVIDIA, released 2026-10-15. Hybrid MoE decoder, 120B total, 12B active, 1M context. Attention: Mostly Mamba-2 with eight GQA layers.
Hybrid MoENVIDIA 🇺🇸 · March 11, 2026
Nemotron 3 Super (120B-A12B)
The Super variant scales up Nano and adds both latent experts and native speculative decoding support.
Key detail
Adds latent-space MoE and shared-weight MTP for fast inference.Total params
120B
Active / token
12B
Context
1M
Fact sheet
- Scale
- 120B total, 12B active (10% active)
- Context window
- 1,000,000 tokens
- Decoder type
- Hybrid MoE
- Attention
- Mostly Mamba-2 with a few GQA layers
- Layer mix
- 8 GQA + 40 Mamba-2 + 40 MoE
- KV cache / token (bf16)
- 8 KiB · Very low
- License
- NVIDIA Nemotron Open Model License
- Released
Efficiency
Parameters active per token
10%
12B of 120B weights run for each token.
AA Intelligence Index v4.3.2
Source12.8
- General
- 22.1
- Scientific
- 12
- Coding
- 18.1
- Agents
- 0.6
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Alibaba Cloud, released 2026-08-26. Hybrid MoE decoder, 125B total, 6B active, 256K context. Attention: 3:1 Gated DeltaNet and Qwen Sparse Attention (QSA).
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
NVIDIA, released 2026-08-11. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with six GQA layers.