Dense hybridLiquid AI 🇺🇸 · October 27, 2026

LFM3 (1.7B)

Liquid AI's 1.73B dense hybrid combines short convolutions with sliding-window and global GQA in a 24-layer stack.

Key detail

Uses kernel-7 ShortConv, a 256-token attention window, per-head sigmoid gates, and sandwich RMSNorm. Global attention uses 50% partial RoPE.

Parameters

1.7B

Context

256K

KV / token

22 KiB

Fact sheet

Scale: 1.73B parameters
Context window: 262,144 tokens
Decoder type: Dense hybrid
Attention: ShortConv plus gated GQA with QK-Norm and sliding-window/global attention
Layer mix: 13 ShortConv + 5 sliding-window + 6 global GQA
KV cache / token (bf16): 22 KiB · Very low
License: LFM Open License v1.0
Released: October 27, 2026

Parameters active per token

100%

Dense: every weight runs for every token.

KV cache per token (bf16)

22 KiB

Smaller than 79% of models in the atlas. How it’s calculated

Use the ← and → keys to step through releases.

Dory1.0 (4B)
NVIDIA, released 2026-10-16. Dense decoder, 4B parameters, 256K context. Attention: Gated GQA; 3:1 sliding-window/full, RoPE.
Muse Glimmer (30B)
Meta, released 2026-08-10. Dense decoder, 30B parameters, 128K context. Attention: Gated GQA with scaleless QK-Norm, local-only RoPE, global NoPE, and 3:1 sliding-window/global attention.
Ornith 1.5 (35B-A3B)
Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
BTL-3 (27B)
Bad Theory Labs, released 2026-07-20. Dense hybrid decoder, 27B parameters, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.