NVIDIA, released 2025-12-04. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with a few GQA layers.
Sparse hybridLiquid AI 🇺🇸 · May 28, 2026
LFM2.5 (8B-A1B)
Liquid AI's sparse LFM2.5 model combines LIV convolution, GQA, and MoE routing with 4 of 32 experts active per token.
Key detail
Uses 24 layers, 32 MoE experts with 4 active per token, and leaves the first two layers dense before sparse routing begins.Total params
8.3B
Active / token
1.5B
Context
128K
Fact sheet
- Scale
- 8.3B total, 1.5B active (18.1% active)
- Context window
- 128,000 tokens
- Decoder type
- Sparse hybrid
- Attention
- LIV convolution blocks plus GQA and MoE
- Layer mix
- 18 LIV conv + 6 GQA + MoE FFNs
- KV cache / token (bf16)
- 12 KiB · Very low
- License
- LFM Open License v1.0
- Released
Efficiency
Parameters active per token
18%
1.5B of 8.3B weights run for each token.
AA Intelligence Index v4.3.2
Source7.2
- Scientific
- 3.4
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
NVIDIA, released 2026-08-11. Hybrid MoE decoder, 30B total, 3B active, 1M context. Attention: Mostly Mamba-2 with six GQA layers.
SOOFI Consortium, released 2026-07-10. Hybrid MoE decoder, 32B total, 3.2B active, 1M context. Attention: Mostly Mamba-2 with 6 GQA layers and no positional embeddings (NoPE).
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.