Skip to content
Hot eventWatching

Dharma-AI previews model predicting fine-tuning job duration

1 reports1 reporting sourcesUpdated 1 d ago

Start here

AI digest

Dharma-AI published a blog preview of an upcoming paper describing a CatBoost regressor that predicts end-to-end fine-tuning time for generative language models. Trained on 272 production supervised fine-tuning (SFT) runs on NVIDIA H200 GPUs and using only pre-dispatch configuration features, the model reports R² = 0.98 and a mean absolute error of 1.71 hours, against R² = 0.31 and 12.41 hours for the standard FLOP-based estimate. The authors say it captures evaluation cadence, checkpoint I/O and per-GPU load that closed-form math misses. The results come with stated limits: the study covers SFT only, excluding RLHF, DPO and RLVR, and every run used H200s, so other GPU types are untested. Errors widen for jobs longer than 60 hours. The full paper is to appear on arXiv.

Generated from reports · updated 1 d ago

Report timeline

Follow the reports to see the event from different sides.

November 19
  1. Hugging Face Blog
    Why your fine-tuning job takes longer than the FLOPs say

    A CatBoost regressor trained on 272 production SFT runs on NVIDIA H200 GPUs predicts end-to-end fine-tuning time with R² = 0.98 and MAE of 1.71 h, versus R² = 0.31 and MAE 12.41 h for the standard FLOP-based estimate. Using only pre-dispatch configuration features, it captures evaluation cadence, checkpoint I/O and per-GPU load that closed-form math misses.

Heat for this event

Current, comparable range
5
Peak, comparable range
10November 19 04:00
Change in 24 hours
–

The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.