BestRand method reduces benchmark test error
The story so far
qwen released the BestRand method, which confirms the high probability of strong performance of random sampling in benchmark tests through error distribution analysis, but there are rare large errors leading to instability. The method has been validated in over 70 text, multimodal, and agent benchmark tests, reducing the high error risk by 50.2% and average estimation error by 16.4% compared to competing methods, while only requiring 51.3% of the construction cost, and averaging a 21.8% reduction in OOD estimation error. The code and data have been publicly released at https://github.com/George-Ling3/BestRand.
AI-written from the reports · Updated 12 days ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face Daily PapersRandom Is Almost Enough? Stable and Low-Cost Benchmark Compression via BestRand
BestRand confirms high-probability strong performance of random sampling in benchmark tests through analysis of error distribution, but exists rare large errors leading to instability. The method is validated in over 70 text, multimodal, and agent benchmark tests, reducing high error risk by 50.2% compared to competing methods, decreasing average estimated error by 16.4%, while requiring only 51.3% of the construction cost, and averaging a 21.8% reduction in OOD estimated error.