Skip to content
Live
Trending storyPast story

ByteDance releases cross-layer transcoder CLT

1 reports1 outlets14 days agoUpdate

The story so far

AI overview

ByteDance Seed released a cross-layer transcoder CLT designed for Qwen3-8B-Base, containing 36 layers × 32,768 potential variables, with 94.2B parameters, a total size of 377 GB, and supporting fp32 inference. The model encodes by jointly scoring the 360 highest activated potentials in all layers, and reconstructs the original MLP output during decoding. Model weights and code are open-sourced under the Apache License 2.0, with copyright held by ByteDance and/or its affiliated companies in 2026.

AI-written from the reports · Updated 14 days ago

Coverage timeline

Follow the coverage to see every side of the story.

5 Nov
  1. ByteDance Seed · Hugging Face new models
    ByteDance-Seed/Cross-Layer-TopK-CLT

    ByteDance-Seed released the Cross-Layer Transcoder (CLT) for Qwen3-8B-Base, which includes 36 layers × 32,768 latent variables, with 94.2B parameters, a total size of 377 GB, supporting fp32 inference, encoding by jointly scoring the top 360 activated latent variables across all layers, and reconstructing the original MLP output during decoding.