Hybrid MoESOOFI Consortium 🇩🇪 · July 10, 2026

Soofi S (30B-A3B)

German-English base model that retrains the Nemotron 3 Nano architecture on 26.68T tokens with a fully documented data mixture.

Key detail

Reuses the Nemotron 3 Nano architecture without modification, isolating the effect of a German-English data and training recipe.

Total params

32B

Active / token

3.2B

Context

Fact sheet

Scale: 31.6B total, 3.2B active (10.1% active)
Context window: 1,048,576 tokens
Decoder type: Hybrid MoE
Attention: Mostly Mamba-2 with 6 GQA layers and no positional embeddings (NoPE)
Layer mix: 6 GQA + 23 Mamba-2 + 23 MoE
KV cache / token (bf16): 6 KiB · Very low
License: Custom license (preview)
Released: July 10, 2026

Parameters active per token

10%

3.2B of 32B weights run for each token.

KV cache per token (bf16)

6 KiB

Smaller than 96% of models in the atlas. How it’s calculated

Use the ← and → keys to step through releases.

Solar Open 2 (250B)
Upstage, released 2026-07-22. Hybrid MoE decoder, 250B total, 15B active, 1M context. Attention: 3:1 Kimi Delta Attention and gated GQA with NoPE.
Nemotron 3 Nano (30B-A3B)
NVIDIA, released 2025-12-04. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with a few GQA layers.
Nemotron 3.5 Lightning (30B-A3B)
NVIDIA, released 2026-08-11. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with six GQA layers.
Ornith 1.5 (35B-A3B)
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.