Ai2, released 2024-11-25. Dense decoder, 7B parameters, 4K context. Attention: MHA with QK-Norm.
RecurrentNXAI 🇦🇹 · March 17, 2025
xLSTM (7B)
Recurrent 7B language model that replaces self-attention with xLSTM blocks built around matrix memory.
Key detail
Stateful recurrent architecture aimed at fast long-context inference without an explicit context window.Parameters
7B
Context
—
KV / token
0 B
Fact sheet
- Scale
- 7B parameters
- Context window
- No explicit limit tokens
- Decoder type
- Recurrent
- Attention
- No self-attention; mLSTM recurrent layers with matrix memory
- Layer mix
- 32 mLSTM
- KV cache / token (bf16)
- 0 B · No cache
- License
- NXAI Community License Agreement
- Released
Efficiency
Parameters active per token
100%
Dense: every weight runs for every token.
KV cache per token (bf16)
0 B
Not reported for this model.
Sources & resources
Use the ← and → keys to step through releases.
Similar architectures
Ai2, released 2025-11-20. Dense decoder, 7B parameters, 64K context. Attention: MHA with QK-Norm and 3:1 sliding-window/global attention.
Meta, released 2024-04-18. Dense decoder, 8B parameters, 8K context. Attention: GQA with RoPE.
Alibaba Cloud, released 2025-04-28. Dense decoder, 8B parameters, 128K context. Attention: GQA with QK-Norm.