ByteDance releases cross-layer transcoder CLT
The story so far
ByteDance Seed released a cross-layer transcoder CLT designed for Qwen3-8B-Base, containing 36 layers × 32,768 potential variables, with 94.2B parameters, a total size of 377 GB, and supporting fp32 inference. The model encodes by jointly scoring the 360 highest activated potentials in all layers, and reconstructs the original MLP output during decoding. Model weights and code are open-sourced under the Apache License 2.0, with copyright held by ByteDance and/or its affiliated companies in 2026.
AI-written from the reports · Updated 14 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- ByteDance Seed · Hugging Face new modelsByteDance-Seed/Cross-Layer-TopK-CLT
ByteDance-Seed released the Cross-Layer Transcoder (CLT) for Qwen3-8B-Base, which includes 36 layers × 32,768 latent variables, with 94.2B parameters, a total size of 377 GB, supporting fp32 inference, encoding by jointly scoring the top 360 activated latent variables across all layers, and reconstructing the original MLP output during decoding.