Add warmup logic when calculating eta #923
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #923 +/- ##
==========================================
- Coverage 86.49% 86.41% -0.09%
==========================================
Files 468 468
Lines 44046 44115 +69
==========================================
+ Hits 38098 38122 +24
- Misses 5948 5993 +45
☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
The estimation has a hard time stabilizing in the beginning (on ag-news-train I had 20 to 40 iterations where the estimation was changing all the time).
I don't think it's a matter of how much time you wait (I tried at 10 secs then deactivated this condition to wait 30 secs, only to get similar results), but rather the fact that when we start showing the time the iterations window on which it is based is very small because the warmup items are ignored. However I think the last part of the warmup could be used to stabilize the first iterations.
Also, not directly related to your PR, but there is no singular in the estimate (it writes "1 hours")
|
@louisfd Merging the current PR, since adding a minimum and maximum data points before showing the estimate is out of scope for this PR. |
Fix #912