Apple proposed a new method to improve the efficiency of long-context retrieval.
The story so far
Apple Machine Learning Research proposes a Weighted State Souping method that combines cached State state encodings using query dependency weights, significantly improving long-context performance. This method is suitable for tasks such as single-hop and multi-hop document question answering, pinpoint retrieval, variable tracking, and tool use, and can be adapted into an efficient sparse variant to achieve near-constant encoding costs.
AI-written from the reports · Updated 7 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Apple Machine Learning ResearchWeighted State Souping for Efficient Long-Context Retrieval
Weighted State Souping proposes query dependency weighting to combine cached State state encodings, significantly improving long-context performance on tasks such as single-hop and multi-hop document question answering, pinpoint retrieval, variable tracking, and tool use. The method can be adapted into an efficient sparse variant, achieving near-constant encoding costs while maintaining (or sometimes improving) performance.
Heat over time
Not enough data yet to draw a trend.