HOPE Advanced Pruning Method Released
The story so far
Hugging Face releases the HOPE method, which considers the second-order interaction between experts to achieve better pruning effects in hybrid expert language models. At a 50% pruning rate, it improves performance by 6.1% on agentic coding tasks compared to the next best method. This method is validated on models like Qwen3.5-122B, with each MoE layer having 256 experts. After pruning, it reduces memory usage by approximately 250 GB and is compatible with quantization.
AI-written from the reports · Updated 9 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face BlogYour Experts Have Partners. There's HOPE for Keeping Them Together
The HOPE method achieves better pruning effects in mixed expert language models by considering the second-order interactions between experts. It improves performance by 6.1% over the next best method on the agentic coding task at a 50% pruning rate. This method is validated on models like Qwen3.5-122B, where each MoE layer has 256 experts. After pruning, it can reduce memory usage by about 250 GB and is compatible with quantization.