Ornith, released 2026-08-23. Hybrid MoE decoder, 35B total, 3B active, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
Dense hybridAlibaba Cloud 🇨🇳 · August 14, 2026
Qwen3.8 (27B)
Dense 27B native vision-language model with a 64-layer Gated DeltaNet and Gated Attention language backbone; the figure omits the vision encoder and MTP head.
Key detail
Uses a 64-layer 3:1 Gated DeltaNet/Gated Attention decoder with partial RoPE and one MTP layer.Parameters
27B
Context
256K
KV / token
64 KiB
Fact sheet
- Scale
- 27B parameters
- Context window
- 262,144 tokens
- Decoder type
- Dense hybrid
- Attention
- 3:1 Gated DeltaNet and Gated Attention
- Layer mix
- 16 gated attention + 48 DeltaNet
- KV cache / token (bf16)
- 64 KiB · Low
- License
- Apache License 2.0
- Released
Efficiency
Parameters active per token
100%
Dense: every weight runs for every token.
AA Intelligence Index v5.0
Source20.1
- Knowledge Work
- 39.7
- Coding
- 11.2
- Science
- 1.9
- General
- 27.3
Architecture concepts
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Bad Theory Labs, released 2026-07-20. Dense hybrid decoder, 27B parameters, 256K context. Attention: 3:1 Gated DeltaNet and Gated Attention.
Alibaba Cloud, released 2026-08-26. Hybrid MoE decoder, 125B total, 6B active, 256K context. Attention: 3:1 Gated DeltaNet and Qwen Sparse Attention (QSA).
StepFun, released 2026-09-20. Sparse MoE decoder, 600B total, 27B active, 1M context. Attention: Gated GQA with QK-Norm and 3:1 sliding-window/sparse attention.