# Per-Layer Embeddings (PLE)

> Each layer gets its own token-specific embedding signal from a large lookup table.

Category: Efficiency & sharing

Lookup tables are cheap to run and can live outside fast memory, so small models can store extra token-specific information per layer without making the transformer stack itself more expensive.

It is why Gemma 4 E2B is described as 2.3B effective parameters but 5.1B including embeddings. The “E” stands for effective.

**Trade-off:** More capacity for small on-device models at very low compute cost.

## Models using PLE (3)

- [Qwen3.8-Flash-Next (125B-A6B)](https://llm-atlas-7or.pages.dev/models/qwen3-8-flash-next-125b-a6b.md) — Alibaba Cloud, 2026-08-26
- [Gemma 4 (E2B)](https://llm-atlas-7or.pages.dev/models/gemma-4-e2b.md) — Google, 2026-04-02
- [Gemma 4 (E4B)](https://llm-atlas-7or.pages.dev/models/gemma-4-e4b.md) — Google, 2026-04-02

Full explainer with figures by Sebastian Raschka: https://sebastianraschka.com/llm-architecture-gallery/per-layer-embeddings/
Web page: https://llm-atlas-7or.pages.dev/concepts/per-layer-embeddings/