DenseGoogle 🇺🇸 · April 2, 2026

Gemma 4 (31B)

Dense Gemma 4 scales the family to a 256K-context multimodal checkpoint without changing the core local-global recipe much.

Key detail

Carries Gemma's unusual pre/post-norm stack into a larger 31B dense model with 256K context.

Parameters

31B

Context

256K

KV / token

840 KiB

Fact sheet

Scale: 30.7B parameters
Context window: 256,000 tokens
Decoder type: Dense
Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention
Layer mix: 50 sliding-window + 10 global
KV cache / token (bf16): 840 KiB · Very high
License: Apache License 2.0
Released: April 2, 2026

Parameters active per token

100%

Dense: every weight runs for every token.

KV cache per token (bf16)

840 KiB

Smaller than 1% of models in the atlas. How it’s calculated

13.9

Use the ← and → keys to step through releases.

Muse Glimmer (30B)
Meta, released 2026-08-10. Dense decoder, 30B parameters, 128K context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.
OLMo 3 (32B)
Ai2, released 2025-11-20. Dense decoder, 32B parameters, 64K context. Attention: GQA with QK-Norm and 3:1 sliding-window/global attention.
Gemma 3 (27B)
Google, released 2025-03-11. Dense decoder, 27B parameters, 128K context. Attention: GQA with QK-Norm and 5:1 sliding-window/global attention.
Gemma 4 (12B)
Google, released 2026-06-03. Dense decoder, 12B parameters, 128K context. Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention.