-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some tags only contain one data point #31
Comments
Can you share the log files so I can take a closer look? |
Yes, you can download the logs from here. |
Hmmm... Could you post your versions for Tensorboard Reducer, TensorBoard, and PyTorch/TensorFlow? Running the command you posted above tb-reducer ant_sac/*seed*/log/serial/ -o out -r mean,std,min,max --lax-steps on the data you shared, my TB dashboard looks very different. Or maybe try reloading the dashboard a few times. I've known it to be a bit laggy in displaying all available data. |
As I said, it has problem with the tag |
Oops, my bad. Didn't read carefully enough. I'm not fully awake until I've had my 2nd tea or coffee. 😄 I took another look and I think the problem is some of the runs are empty, resulting in those steps being filtered out. If you re-run your command with tb-reducer ant_sac/*seed*/log/serial/ -o out -r mean,std,min,max --lax-steps --min-runs-per-step 1 --overwrite it does include many more steps in the reduction but it gives this jumbled mess which isn't very helpful either. I'll have to dig deeper to see why that happens but maybe you can try removing/excluding the empty event files as workaround in the meantime. |
Actually, all logs that match |
Sorry for the long radio silence.
Ah, that could be it. Glad you figured it out.
I haven't looked into that but happy to take a PR for this feature. You may be able to use the |
tb-reducer ant_sac/*seed*/log/serial/ -o out -r mean,std,min,max --lax-steps
Although I used
--lax-steps
flag, some tags only contain one data point. Others are fine.collector_step/reward_mean
is actually containing more data points thanevaluator_step/reward_mean
The text was updated successfully, but these errors were encountered: