Poolside, released 2026-07-02. Sparse MoE decoder, 33B total, 3B active, 256K context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.
Sparse MoEPoolside 🇺🇸 · April 28, 2026
Laguna XS.2 (33B)
Poolside's open-weight agentic coding MoE uses mixed sliding-window/global attention and activates only 3B of 33B parameters per token.
Key detail
Uses per-layer query-head counts, a 512-token SWA window, sigmoid MoE routing, and one shared expert alongside the top-8 routed experts.Total params
33B
Active / token
3B
Context
128K
Fact sheet
- Scale
- 33B total, 3B active (9.1% active)
- Context window
- 131,072 tokens
- Decoder type
- Sparse MoE
- Attention
- Gated GQA with QK-Norm and 3:1 sliding-window/global attention
- Layer mix
- 30 sliding-window + 10 global
- KV cache / token (bf16)
- 160 KiB · Moderate
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
9.1%
3B of 33B weights run for each token.
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Poolside, released 2026-07-21. Sparse MoE decoder, 118B total, 8B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/global attention.
Recursive, released 2026-10-18. Sparse MoE decoder, 40B total, 2.5B active, 256K context. Attention: GQA with QK-Norm, learned attention sinks, and sliding-window/global attention.
Google, released 2026-04-02. Sparse MoE decoder, 25B total, 3.8B active, 256K context. Attention: GQA with QK-Norm, unified K/V on global layers, p-RoPE on global layers, and 5:1 sliding-window/global attention.