Redistribution sensitivity score identifies feature redistribution patterns
The story so far
Hugging Face has released a paper on Representation Redistribution in Sparse Autoencoders, proposing the Redistribution sensitivity score (RSS) to identify repeatable feature redistribution patterns in pre-trained SAE models and quantify their impact on downstream tasks. The study covers 27 pre-trained SAE models across 8 language models, finding that feature changes do not always equate to failure. The code has been open-sourced to https://github.com/OpenSAE/Redistribution-Sensitivity-Score.
AI-written from the reports · Updated 22 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersHugging Face Daily Papers: Representation Redistribution in Sparse Autoencoders
Redistribution sensitivity score (RSS) identification generates repeatable feature redistribution patterns on 27 pre-trained SAE models (covering 8 language models), combined with clean margin to quantify its contribution to downstream reading.