Meta, released 2026-08-10. Dense decoder, 30B parameters, 128K context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.
DenseIBM 🇺🇸 · April 29, 2026
Granite 4.1 (30B)
IBM's dense 30B Granite 4.1 model keeps a straightforward long-context GQA transformer stack and improves tool calling, instruction following, and chat behavior through post-training.
Key detail
Uses a 4096-wide dense stack, 32 query heads, 8 KV heads, SwiGLU MLPs, shared input/output embeddings, and Granite-specific MuP scaling factors.Parameters
30B
Context
128K
KV / token
256 KiB
Fact sheet
- Scale
- 30B parameters
- Context window
- 131,072 tokens
- Decoder type
- Dense
- Attention
- GQA with RoPE
- Layer mix
- 64 GQA
- KV cache / token (bf16)
- 256 KiB · High
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
100%
Dense: every weight runs for every token.
AA Intelligence Index v4.3.2
Source7.4
- Scientific
- 2.1
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Google, released 2026-04-02. Dense decoder, 31B parameters, 256K context. Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention.
Alibaba Cloud, released 2025-04-28. Dense decoder, 32B parameters, 128K context. Attention: GQA with QK-Norm.
Ai2, released 2025-11-20. Dense decoder, 32B parameters, 64K context. Attention: GQA with QK-Norm and 3:1 sliding-window/global attention.