Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Autocomplete for losses #2148

Merged
merged 2 commits into from
May 2, 2024
Merged

[Feature] Autocomplete for losses #2148

merged 2 commits into from
May 2, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented May 2, 2024 •
edited
Loading

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented May 2, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2148

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Unrelated Failure

As of commit 0d17e45 with merge base 7109a3f (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 2, 2024
@vmoens vmoens added the enhancement New feature or request label May 2, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented May 2, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 57.8266ms 55.3583ms 18.0642 Ops/s 18.0553 Ops/s $\color{#35bf28}+0.05\%$
test_sync 40.7981ms 31.2962ms 31.9527 Ops/s 32.5571 Ops/s $\color{#d91a1a}-1.86\%$
test_async 58.6591ms 28.9610ms 34.5291 Ops/s 34.5173 Ops/s $\color{#35bf28}+0.03\%$
test_simple 0.4379s 0.3644s 2.7442 Ops/s 2.8908 Ops/s $\textbf{\color{#d91a1a}-5.07\%}$
test_transformed 0.5004s 0.4981s 2.0077 Ops/s 2.0107 Ops/s $\color{#d91a1a}-0.15\%$
test_serial 1.3218s 1.2533s 0.7979 Ops/s 0.7962 Ops/s $\color{#35bf28}+0.22\%$
test_parallel 1.1133s 1.0556s 0.9473 Ops/s 0.9647 Ops/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-True-True-True-True] 0.1235ms 21.2397μs 47.0817 KOps/s 46.8375 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-True-True-True-False] 46.2260μs 12.9302μs 77.3386 KOps/s 77.4596 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-True-True-False-True] 85.5290μs 12.5068μs 79.9565 KOps/s 79.1050 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-True-True-False-False] 39.9650μs 7.5830μs 131.8744 KOps/s 131.6757 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-True-False-True-True] 54.3710μs 22.8452μs 43.7729 KOps/s 43.8532 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-True-False-True-False] 43.5010μs 14.3251μs 69.8074 KOps/s 69.6292 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-False-False-True] 47.7990μs 13.6606μs 73.2033 KOps/s 72.5898 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-False-False] 51.5660μs 8.7890μs 113.7792 KOps/s 113.0797 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-False-True-True-True] 59.8620μs 24.1104μs 41.4759 KOps/s 41.4174 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-True-True-False] 44.7030μs 15.7121μs 63.6451 KOps/s 64.2199 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-True-False-True] 78.5800μs 13.8198μs 72.3599 KOps/s 72.0232 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-False-True-False-False] 51.4660μs 8.7989μs 113.6503 KOps/s 112.2198 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-False-False-True-True] 63.3580μs 25.2709μs 39.5712 KOps/s 40.1738 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-False-False-True-False] 77.9860μs 16.9131μs 59.1258 KOps/s 60.3708 KOps/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[True-False-False-False-True] 56.2860μs 14.9771μs 66.7685 KOps/s 67.0700 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-False-False-False] 39.1230μs 10.0587μs 99.4165 KOps/s 99.6943 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-True-True-True-True] 60.5530μs 24.0326μs 41.6102 KOps/s 41.6696 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-True-True-True-False] 43.3510μs 15.8310μs 63.1673 KOps/s 64.4308 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[False-True-True-False-True] 47.5380μs 15.8357μs 63.1485 KOps/s 62.3156 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-True-True-False-False] 37.1890μs 10.1008μs 99.0025 KOps/s 99.1528 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-False-True-True] 64.8110μs 25.0972μs 39.8451 KOps/s 39.7391 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-False-True-False] 66.7640μs 16.7588μs 59.6702 KOps/s 59.6966 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-True-False-False-True] 47.5280μs 17.0216μs 58.7488 KOps/s 59.0944 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-False-False-False] 52.4580μs 11.2417μs 88.9547 KOps/s 88.4682 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-True-True-True] 83.3960μs 26.4588μs 37.7947 KOps/s 37.8578 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-False-True-True-False] 84.2240μs 17.9182μs 55.8093 KOps/s 55.3539 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-False-True-False-True] 74.3180μs 17.0752μs 58.5644 KOps/s 58.3488 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-False-True-False-False] 42.7800μs 11.1286μs 89.8588 KOps/s 88.7180 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-False-False-True-True] 50.4840μs 27.8039μs 35.9662 KOps/s 35.8566 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-True-False] 66.4040μs 19.0908μs 52.3813 KOps/s 52.4631 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-False-False-False-True] 63.9390μs 18.0071μs 55.5335 KOps/s 55.0197 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-False-False-False-False] 40.9670μs 12.2678μs 81.5145 KOps/s 81.4368 KOps/s $\color{#35bf28}+0.10\%$
test_values[generalized_advantage_estimate-True-True] 10.4600ms 9.8217ms 101.8159 Ops/s 103.0225 Ops/s $\color{#d91a1a}-1.17\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.8316ms 36.4676ms 27.4216 Ops/s 27.5094 Ops/s $\color{#d91a1a}-0.32\%$
test_values[td0_return_estimate-False-False] 0.2630ms 0.1863ms 5.3680 KOps/s 5.5018 KOps/s $\color{#d91a1a}-2.43\%$
test_values[td1_return_estimate-False-False] 26.5741ms 24.1389ms 41.4269 Ops/s 41.7056 Ops/s $\color{#d91a1a}-0.67\%$
test_values[vec_td1_return_estimate-False-False] 39.9447ms 36.5287ms 27.3757 Ops/s 27.4772 Ops/s $\color{#d91a1a}-0.37\%$
test_values[td_lambda_return_estimate-True-False] 37.7534ms 34.4872ms 28.9962 Ops/s 29.0306 Ops/s $\color{#d91a1a}-0.12\%$
test_values[vec_td_lambda_return_estimate-True-False] 39.7250ms 36.6535ms 27.2825 Ops/s 27.2961 Ops/s $\color{#d91a1a}-0.05\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7703ms 8.4322ms 118.5924 Ops/s 120.2052 Ops/s $\color{#d91a1a}-1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.6976ms 2.0596ms 485.5318 Ops/s 525.5289 Ops/s $\textbf{\color{#d91a1a}-7.61\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4775ms 0.3655ms 2.7357 KOps/s 2.7085 KOps/s $\color{#35bf28}+1.00\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 51.5365ms 47.6570ms 20.9833 Ops/s 20.6180 Ops/s $\color{#35bf28}+1.77\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7475ms 3.0836ms 324.2947 Ops/s 320.7182 Ops/s $\color{#35bf28}+1.12\%$
test_dqn_speed 2.1798ms 1.3876ms 720.6476 Ops/s 711.5946 Ops/s $\color{#35bf28}+1.27\%$
test_ddpg_speed 3.6836ms 2.9805ms 335.5144 Ops/s 335.0968 Ops/s $\color{#35bf28}+0.12\%$
test_sac_speed 11.1456ms 9.3293ms 107.1895 Ops/s 105.3942 Ops/s $\color{#35bf28}+1.70\%$
test_redq_speed 15.3938ms 14.0698ms 71.0744 Ops/s 69.5377 Ops/s $\color{#35bf28}+2.21\%$
test_redq_deprec_speed 18.9905ms 14.9932ms 66.6969 Ops/s 66.7545 Ops/s $\color{#d91a1a}-0.09\%$
test_td3_speed 19.0759ms 9.2319ms 108.3204 Ops/s 107.9635 Ops/s $\color{#35bf28}+0.33\%$
test_cql_speed 44.7555ms 39.9840ms 25.0100 Ops/s 25.0604 Ops/s $\color{#d91a1a}-0.20\%$
test_a2c_speed 9.5197ms 8.2383ms 121.3847 Ops/s 118.6209 Ops/s $\color{#35bf28}+2.33\%$
test_ppo_speed 9.9484ms 8.5177ms 117.4025 Ops/s 114.0912 Ops/s $\color{#35bf28}+2.90\%$
test_reinforce_speed 8.4996ms 7.1261ms 140.3289 Ops/s 139.1455 Ops/s $\color{#35bf28}+0.85\%$
test_iql_speed 36.8495ms 34.9059ms 28.6484 Ops/s 28.5643 Ops/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7877ms 2.4338ms 410.8745 Ops/s 383.5534 Ops/s $\textbf{\color{#35bf28}+7.12\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0234ms 0.5283ms 1.8929 KOps/s 1.8835 KOps/s $\color{#35bf28}+0.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6384ms 0.5006ms 1.9978 KOps/s 1.9878 KOps/s $\color{#35bf28}+0.50\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9107ms 2.4540ms 407.4910 Ops/s 386.1322 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9604ms 0.5155ms 1.9398 KOps/s 1.9163 KOps/s $\color{#35bf28}+1.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8135ms 0.4922ms 2.0315 KOps/s 2.0120 KOps/s $\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7824ms 1.2428ms 804.6045 Ops/s 791.4724 Ops/s $\color{#35bf28}+1.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3704ms 1.1825ms 845.6564 Ops/s 826.6091 Ops/s $\color{#35bf28}+2.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2561ms 2.5867ms 386.5958 Ops/s 349.3074 Ops/s $\textbf{\color{#35bf28}+10.67\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8431ms 0.6401ms 1.5623 KOps/s 1.5474 KOps/s $\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1294s 0.7195ms 1.3899 KOps/s 1.6086 KOps/s $\textbf{\color{#d91a1a}-13.60\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7622ms 2.4589ms 406.6843 Ops/s 381.6914 Ops/s $\textbf{\color{#35bf28}+6.55\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6250ms 0.5179ms 1.9307 KOps/s 1.8983 KOps/s $\color{#35bf28}+1.71\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8574ms 0.5017ms 1.9932 KOps/s 1.9818 KOps/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4962ms 2.6012ms 384.4390 Ops/s 380.8450 Ops/s $\color{#35bf28}+0.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1292s 0.6924ms 1.4442 KOps/s 1.9048 KOps/s $\textbf{\color{#d91a1a}-24.18\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5929ms 0.4878ms 2.0499 KOps/s 2.0219 KOps/s $\color{#35bf28}+1.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0705ms 2.7220ms 367.3719 Ops/s 368.0541 Ops/s $\color{#d91a1a}-0.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0570ms 0.6553ms 1.5261 KOps/s 1.5603 KOps/s $\color{#d91a1a}-2.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.8096ms 0.6236ms 1.6036 KOps/s 1.6156 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1303s 8.9441ms 111.8051 Ops/s 109.6205 Ops/s $\color{#35bf28}+1.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.7974ms 12.8825ms 77.6250 Ops/s 76.2642 Ops/s $\color{#35bf28}+1.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6740ms 1.1236ms 890.0305 Ops/s 842.0117 Ops/s $\textbf{\color{#35bf28}+5.70\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1203s 6.1762ms 161.9132 Ops/s 157.9578 Ops/s $\color{#35bf28}+2.50\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.3110ms 12.8250ms 77.9726 Ops/s 65.4476 Ops/s $\textbf{\color{#35bf28}+19.14\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.6481ms 1.1385ms 878.3279 Ops/s 794.9719 Ops/s $\textbf{\color{#35bf28}+10.49\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1333s 9.2589ms 108.0038 Ops/s 141.4789 Ops/s $\textbf{\color{#d91a1a}-23.66\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.4438ms 13.5117ms 74.0100 Ops/s 74.9413 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5636ms 1.6083ms 621.7577 Ops/s 678.6465 Ops/s $\textbf{\color{#d91a1a}-8.38\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented May 2, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1159s 0.1155s 8.6588 Ops/s 8.7980 Ops/s $\color{#d91a1a}-1.58\%$
test_sync 0.1022s 0.1016s 9.8463 Ops/s 9.7139 Ops/s $\color{#35bf28}+1.36\%$
test_async 0.1872s 94.7670ms 10.5522 Ops/s 10.5609 Ops/s $\color{#d91a1a}-0.08\%$
test_single_pixels 0.2049s 0.1360s 7.3518 Ops/s 7.9952 Ops/s $\textbf{\color{#d91a1a}-8.05\%}$
test_sync_pixels 0.1407s 85.0629ms 11.7560 Ops/s 11.7055 Ops/s $\color{#35bf28}+0.43\%$
test_async_pixels 0.1603s 68.4773ms 14.6034 Ops/s 13.0387 Ops/s $\textbf{\color{#35bf28}+12.00\%}$
test_simple 0.7615s 0.7586s 1.3182 Ops/s 1.3133 Ops/s $\color{#35bf28}+0.38\%$
test_transformed 1.0134s 1.0103s 0.9898 Ops/s 0.9928 Ops/s $\color{#d91a1a}-0.30\%$
test_serial 2.5074s 2.4432s 0.4093 Ops/s 0.4155 Ops/s $\color{#d91a1a}-1.50\%$
test_parallel 2.3700s 2.2895s 0.4368 Ops/s 0.4370 Ops/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-True-True-True-True] 0.1077ms 33.5540μs 29.8027 KOps/s 30.2163 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-True-True-True-False] 0.1746ms 19.6804μs 50.8120 KOps/s 50.2883 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-True-True-False-True] 0.1468ms 18.8432μs 53.0697 KOps/s 52.9637 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-True-False-False] 0.1809ms 11.2520μs 88.8728 KOps/s 89.8760 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-True-False-True-True] 51.9930μs 35.1114μs 28.4808 KOps/s 28.6341 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-False-True-False] 0.2083ms 21.4398μs 46.6423 KOps/s 45.9331 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-True-False-False-True] 0.2019ms 20.4632μs 48.8681 KOps/s 48.6552 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-True-False-False-False] 33.4820μs 12.9500μs 77.2203 KOps/s 75.5015 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-False-True-True-True] 58.8220μs 35.8199μs 27.9174 KOps/s 26.8844 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[True-False-True-True-False] 53.8220μs 23.2354μs 43.0378 KOps/s 42.2543 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-False-True-False-True] 46.4320μs 20.4737μs 48.8433 KOps/s 48.8659 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-True-False-False] 28.5210μs 13.0323μs 76.7325 KOps/s 75.7710 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-False-False-True-True] 72.1930μs 38.0717μs 26.2662 KOps/s 26.0232 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-False-False-True-False] 41.2420μs 24.9095μs 40.1453 KOps/s 39.0062 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-False-False-True] 50.6620μs 22.1193μs 45.2094 KOps/s 44.7798 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-False-False-False-False] 37.1010μs 14.7246μs 67.9137 KOps/s 66.7076 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-True-True-True-True] 67.6930μs 36.1123μs 27.6914 KOps/s 26.8805 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-True-True-True-False] 47.2020μs 23.2685μs 42.9765 KOps/s 42.5573 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-True-True-False-True] 54.1920μs 24.4477μs 40.9036 KOps/s 39.9654 KOps/s $\color{#35bf28}+2.35\%$
test_step_mdp_speed[False-True-True-False-False] 31.5020μs 14.8334μs 67.4152 KOps/s 65.5958 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[False-True-False-True-True] 63.0220μs 38.0249μs 26.2985 KOps/s 25.8670 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-True-False-True-False] 44.2820μs 24.9284μs 40.1148 KOps/s 38.9248 KOps/s $\color{#35bf28}+3.06\%$
test_step_mdp_speed[False-True-False-False-True] 41.9820μs 26.0645μs 38.3664 KOps/s 37.4748 KOps/s $\color{#35bf28}+2.38\%$
test_step_mdp_speed[False-True-False-False-False] 38.8720μs 16.3824μs 61.0410 KOps/s 59.3007 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[False-False-True-True-True] 58.8320μs 40.0005μs 24.9997 KOps/s 24.7139 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-True-True-False] 50.5020μs 26.9803μs 37.0641 KOps/s 35.9482 KOps/s $\color{#35bf28}+3.10\%$
test_step_mdp_speed[False-False-True-False-True] 42.0720μs 25.9027μs 38.6060 KOps/s 38.0885 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-False-True-False-False] 34.0720μs 16.3883μs 61.0190 KOps/s 60.2034 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-False-False-True-True] 70.9220μs 42.8215μs 23.3528 KOps/s 23.3120 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-False-True-False] 0.1253ms 29.0778μs 34.3905 KOps/s 34.0303 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-False-False-True] 47.5220μs 27.4648μs 36.4103 KOps/s 35.3879 KOps/s $\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-False-False-False-False] 42.7020μs 17.8872μs 55.9059 KOps/s 53.7346 KOps/s $\color{#35bf28}+4.04\%$
test_values[generalized_advantage_estimate-True-True] 25.8777ms 25.2247ms 39.6436 Ops/s 40.0148 Ops/s $\color{#d91a1a}-0.93\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.0997ms 3.2282ms 309.7682 Ops/s 313.3989 Ops/s $\color{#d91a1a}-1.16\%$
test_values[td0_return_estimate-False-False] 88.4730μs 63.2899μs 15.8003 KOps/s 15.8735 KOps/s $\color{#d91a1a}-0.46\%$
test_values[td1_return_estimate-False-False] 53.7242ms 53.3101ms 18.7582 Ops/s 19.0160 Ops/s $\color{#d91a1a}-1.36\%$
test_values[vec_td1_return_estimate-False-False] 2.1321ms 1.7566ms 569.2948 Ops/s 570.1438 Ops/s $\color{#d91a1a}-0.15\%$
test_values[td_lambda_return_estimate-True-False] 85.0051ms 84.3927ms 11.8494 Ops/s 11.9408 Ops/s $\color{#d91a1a}-0.77\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1149ms 1.7526ms 570.5878 Ops/s 572.1585 Ops/s $\color{#d91a1a}-0.27\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0101ms 23.5133ms 42.5290 Ops/s 42.5910 Ops/s $\color{#d91a1a}-0.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8782ms 0.7003ms 1.4280 KOps/s 1.4363 KOps/s $\color{#d91a1a}-0.58\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7998ms 0.6439ms 1.5530 KOps/s 1.5609 KOps/s $\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6183ms 1.4542ms 687.6816 Ops/s 689.9999 Ops/s $\color{#d91a1a}-0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9762ms 0.6628ms 1.5088 KOps/s 1.5151 KOps/s $\color{#d91a1a}-0.41\%$
test_dqn_speed 80.0105ms 1.6165ms 618.6191 Ops/s 697.4400 Ops/s $\textbf{\color{#d91a1a}-11.30\%}$
test_ddpg_speed 3.3309ms 3.0151ms 331.6665 Ops/s 335.6037 Ops/s $\color{#d91a1a}-1.17\%$
test_sac_speed 10.0439ms 8.6974ms 114.9771 Ops/s 117.1546 Ops/s $\color{#d91a1a}-1.86\%$
test_redq_speed 12.7153ms 10.7755ms 92.8030 Ops/s 94.1523 Ops/s $\color{#d91a1a}-1.43\%$
test_redq_deprec_speed 12.4525ms 11.8098ms 84.6755 Ops/s 84.8771 Ops/s $\color{#d91a1a}-0.24\%$
test_td3_speed 17.7805ms 8.7061ms 114.8622 Ops/s 118.9994 Ops/s $\color{#d91a1a}-3.48\%$
test_cql_speed 27.6993ms 26.9324ms 37.1300 Ops/s 37.4074 Ops/s $\color{#d91a1a}-0.74\%$
test_a2c_speed 6.5137ms 5.8156ms 171.9510 Ops/s 169.3101 Ops/s $\color{#35bf28}+1.56\%$
test_ppo_speed 6.2716ms 6.0771ms 164.5523 Ops/s 162.2054 Ops/s $\color{#35bf28}+1.45\%$
test_reinforce_speed 5.0805ms 4.7517ms 210.4514 Ops/s 209.6567 Ops/s $\color{#35bf28}+0.38\%$
test_iql_speed 21.4310ms 20.7007ms 48.3076 Ops/s 48.8427 Ops/s $\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1630ms 2.8483ms 351.0886 Ops/s 352.0204 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7683ms 0.6060ms 1.6503 KOps/s 1.6404 KOps/s $\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.2647ms 0.5881ms 1.7005 KOps/s 1.7028 KOps/s $\color{#d91a1a}-0.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1373ms 2.8781ms 347.4549 Ops/s 355.6716 Ops/s $\color{#d91a1a}-2.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7813ms 0.6026ms 1.6596 KOps/s 1.6720 KOps/s $\color{#d91a1a}-0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.2889ms 0.5835ms 1.7139 KOps/s 1.7295 KOps/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7392ms 1.5899ms 628.9867 Ops/s 637.1114 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6706ms 1.5199ms 657.9563 Ops/s 660.7262 Ops/s $\color{#d91a1a}-0.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2731ms 2.9921ms 334.2096 Ops/s 339.6574 Ops/s $\color{#d91a1a}-1.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9869ms 0.7357ms 1.3592 KOps/s 1.3688 KOps/s $\color{#d91a1a}-0.70\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1234s 0.8292ms 1.2061 KOps/s 1.4177 KOps/s $\textbf{\color{#d91a1a}-14.93\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7473ms 2.8558ms 350.1639 Ops/s 354.4018 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7780ms 0.6106ms 1.6377 KOps/s 1.6421 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7845ms 0.5875ms 1.7022 KOps/s 1.6892 KOps/s $\color{#35bf28}+0.77\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1924ms 2.8911ms 345.8863 Ops/s 348.2751 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3780ms 0.6103ms 1.6385 KOps/s 1.6481 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1276s 0.7900ms 1.2658 KOps/s 1.7318 KOps/s $\textbf{\color{#d91a1a}-26.91\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1821ms 2.9779ms 335.8031 Ops/s 340.1658 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9314ms 0.7364ms 1.3580 KOps/s 1.3526 KOps/s $\color{#35bf28}+0.40\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9101ms 0.7127ms 1.4032 KOps/s 1.3934 KOps/s $\color{#35bf28}+0.70\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1242s 7.3870ms 135.3737 Ops/s 100.3305 Ops/s $\textbf{\color{#35bf28}+34.93\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.9275ms 15.5286ms 64.3972 Ops/s 65.7089 Ops/s $\color{#d91a1a}-2.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.4590ms 1.3381ms 747.3418 Ops/s 841.5042 Ops/s $\textbf{\color{#d91a1a}-11.19\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1231s 9.6723ms 103.3876 Ops/s 136.9235 Ops/s $\textbf{\color{#d91a1a}-24.49\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6293ms 15.6119ms 64.0537 Ops/s 65.8945 Ops/s $\color{#d91a1a}-2.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4508ms 1.2941ms 772.7426 Ops/s 855.5136 Ops/s $\textbf{\color{#d91a1a}-9.68\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1237s 7.6470ms 130.7706 Ops/s 99.0797 Ops/s $\textbf{\color{#35bf28}+31.99\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.2098ms 15.8001ms 63.2907 Ops/s 64.1440 Ops/s $\color{#d91a1a}-1.33\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.9755ms 1.7978ms 556.2433 Ops/s 593.5808 Ops/s $\textbf{\color{#d91a1a}-6.29\%}$

@vmoens vmoens mentioned this pull request May 2, 2024
Closed
@vmoens vmoens linked an issue May 2, 2024 that may be closed by this pull request
[Feature Request] Handling of unserializable policies #2082
Closed
@vmoens vmoens merged commit 6822468 into main May 2, 2024
54 of 58 checks passed
@vmoens vmoens deleted the loss-refactor branch May 2, 2024 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Feature Request] Handling of unserializable policies
2 participants
@vmoens @facebook-github-bot