Poolside, released 2026-07-21. Sparse MoE decoder, 118B total, 8B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.
Sparse MoEOpenAI 🇺🇸 · August 4, 2025
GPT-OSS (120B)
Larger gpt-oss variant keeps the same alternating-attention recipe as the 20B model.
Key detail
Shared architectural template scaled up for OpenAI's flagship open-weight release.Total params
117B
Active / token
5.1B
Context
128K
Fact sheet
- Scale
- 117B total, 5.1B active (4.4% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse MoE
- Attention
- GQA with alternating sliding-window and global layers
- Layer mix
- 18 sliding-window + 18 global
- KV cache / token (bf16)
- 72 KiB · Low
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
4.4%
5.1B of 117B weights run for each token.
AA Intelligence Index v4.3.2
Source11.6
- General
- 18.8
- Scientific
- 10.4
- Coding
- 17
- Agents
- 1.8
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Aleph Alpha, released 2026-10-03. Sparse MoE decoder, 78B total, 3.5B active, 256K context. Attention: GQA with QK-Norm and 4:1 sliding-window/global attention.
StepFun, released 2026-02-01. Sparse MoE decoder, 196B total, 11B active, 256K context. Attention: GQA with 3:1 sliding-window attention.
MiniMax, released 2026-02-12. Sparse MoE decoder, 230B total, 10B active, 192K context. Attention: GQA with QK-Norm.