# Short Convolution (ShortConv)

> A tiny causal depthwise convolution (kernel ≈ 4) that mixes each channel over the last few tokens.

Category: Attention

Each channel is convolved over a handful of recent positions with its own small kernel, and future positions are never seen. It is extremely cheap and gives layers a built-in notion of local order.

Kimi Linear applies ShortConv to the queries, keys and values in its delta-attention layers; Inkling places several ShortConv modules in every block. LFM-style models use convolutions as the main token mixer.

**Trade-off:** Near-free local mixing that complements linear attention, but it cannot see far by itself.

## Models using ShortConv (10)

- [LFM3 (1.7B)](https://llm-atlas-7or.pages.dev/models/lfm3-1-7b.md) — Liquid AI, 2026-10-27
- [Ling 3.1 Flash (560B)](https://llm-atlas-7or.pages.dev/models/ling-3-1-flash.md) — InclusionAI, 2026-10-13
- [Ember-1 (2.8T)](https://llm-atlas-7or.pages.dev/models/ember-1.md) — Fireworks AI, 2026-09-23
- [GLM-5.3-Flash (320B)](https://llm-atlas-7or.pages.dev/models/glm-5-3-flash.md) — Z.ai, 2026-08-26
- [Ornith 1.5 (35B-A3B)](https://llm-atlas-7or.pages.dev/models/ornith-1-5-35b-a3b.md) — Ornith, 2026-08-23
- [Ling 3.0 Flash (124B)](https://llm-atlas-7or.pages.dev/models/ling-3-0-flash.md) — InclusionAI, 2026-08-04
- [Kimi K3 (2.8T)](https://llm-atlas-7or.pages.dev/models/kimi-k3.md) — Moonshot AI, 2026-07-27
- [Solar Open 2 (250B)](https://llm-atlas-7or.pages.dev/models/solar-open-2.md) — Upstage, 2026-07-22
- [Inkling (975B)](https://llm-atlas-7or.pages.dev/models/inkling.md) — Thinking Machines Lab, 2026-07-15
- [Kimi Linear (48B-A3B)](https://llm-atlas-7or.pages.dev/models/kimi-linear-48b-a3b.md) — Moonshot AI, 2025-10-30

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/shortconv/
Web page: https://llm-atlas-7or.pages.dev/concepts/shortconv/