Hugging Face releases predictive model
The story so far
Hugging Face, by analyzing 272 production environment supervised fine-tuning tasks, found that traditional FLOPs-based time-consuming prediction accuracy is only 31%, while its predictive model CatBoost reached 98% accuracy. The gap is due to FLOPs not considering overheads such as environment setup, evaluation, and checkpoints, as well as training signals, while the predictive model learns non-linear interactions using task configuration and metadata.
AI-written from the reports · Updated 2 hours ago
Coverage timeline
Follow the coverage to see every side of the story.
- Hugging Face BlogWhy your fine-tuning job takes longer than the FLOPs say
Hugging Face, by analyzing 272 production environment supervised fine-tuning tasks, found that the accuracy of traditional FLOPs-based duration prediction is only 31% (R²=0.31, MAE 12.41 hours), while its predictive model CatBoost achieved 98% accuracy (MAE 1.71 hours). The gap is due to FLOPs not considering overheads such as environment setup, evaluation, and checkpoints, as well as training signals, while the predictive model learns non-linear interactions using task configuration and metadata.
Heat over time
Not enough data yet to draw a trend.