Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] More robust _StepMDP and multi-purpose envs #2038

Merged
merged 4 commits into from
Mar 25, 2024
Merged

[BugFix] More robust _StepMDP and multi-purpose envs #2038

merged 4 commits into from
Mar 25, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 25, 2024

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 25, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2038

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 4 New Failures

As of commit 236f41a with merge base e835770 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 25, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 25, 2024 •
edited
Loading

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}0$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1049s 0.1033s 9.6836 Ops/s 9.6424 Ops/s $\color{#35bf28}+0.43\%$
test_sync 94.0050ms 91.9011ms 10.8813 Ops/s 10.9873 Ops/s $\color{#d91a1a}-0.96\%$
test_async 0.1757s 87.7504ms 11.3960 Ops/s 11.0182 Ops/s $\color{#35bf28}+3.43\%$
test_single_pixels 0.1134s 0.1118s 8.9450 Ops/s 8.8634 Ops/s $\color{#35bf28}+0.92\%$
test_sync_pixels 69.0376ms 67.4582ms 14.8240 Ops/s 14.7417 Ops/s $\color{#35bf28}+0.56\%$
test_async_pixels 0.1150s 56.5931ms 17.6700 Ops/s 17.3929 Ops/s $\color{#35bf28}+1.59\%$
test_simple 0.7408s 0.6723s 1.4873 Ops/s 1.4842 Ops/s $\color{#35bf28}+0.21\%$
test_transformed 0.9544s 0.8868s 1.1277 Ops/s 1.1240 Ops/s $\color{#35bf28}+0.33\%$
test_serial 2.1384s 2.0707s 0.4829 Ops/s 0.4750 Ops/s $\color{#35bf28}+1.67\%$
test_parallel 1.9307s 1.8692s 0.5350 Ops/s 0.5311 Ops/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-True-True-True] 91.7840μs 33.5762μs 29.7830 KOps/s 30.4842 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-True-True-True-False] 46.4920μs 20.0275μs 49.9315 KOps/s 50.2976 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[True-True-True-False-True] 36.9010μs 18.9271μs 52.8344 KOps/s 53.2819 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-True-False-False] 33.3220μs 11.2168μs 89.1520 KOps/s 87.7933 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[True-True-False-True-True] 55.9220μs 34.7777μs 28.7540 KOps/s 28.9492 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-True-False-True-False] 41.0920μs 21.5020μs 46.5072 KOps/s 46.0578 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-True-False-False-True] 36.3220μs 20.2595μs 49.3597 KOps/s 48.6550 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-True-False-False-False] 38.2920μs 13.0199μs 76.8058 KOps/s 75.8106 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-False-True-True-True] 53.6830μs 36.8103μs 27.1663 KOps/s 27.2141 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-True-True-False] 53.8230μs 23.2587μs 42.9946 KOps/s 42.3522 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-True-False-True] 36.6830μs 20.2860μs 49.2952 KOps/s 49.3981 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-False-False] 33.5310μs 13.0880μs 76.4059 KOps/s 75.5055 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-False-False-True-True] 70.6230μs 38.6034μs 25.9044 KOps/s 26.0489 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-False-False-True-False] 50.5530μs 25.1962μs 39.6886 KOps/s 39.5553 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-False-False-False-True] 42.8620μs 22.1560μs 45.1345 KOps/s 45.3724 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-False-False-False] 30.8420μs 15.0277μs 66.5437 KOps/s 66.8306 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-True-True-True-True] 58.7330μs 36.8298μs 27.1520 KOps/s 27.2510 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-True-True-True-False] 39.4910μs 23.5414μs 42.4784 KOps/s 41.7389 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-True-True-False-True] 50.3720μs 24.5178μs 40.7867 KOps/s 40.9719 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-True-False-False] 34.3920μs 15.1471μs 66.0192 KOps/s 66.3734 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-True-False-True-True] 67.0830μs 38.9734μs 25.6585 KOps/s 26.1255 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[False-True-False-True-False] 85.6230μs 25.4941μs 39.2248 KOps/s 39.2086 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-True-False-False-True] 50.7220μs 26.7167μs 37.4298 KOps/s 38.0326 KOps/s $\color{#d91a1a}-1.58\%$
test_step_mdp_speed[False-True-False-False-False] 32.8510μs 16.9259μs 59.0810 KOps/s 59.1860 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-False-True-True-True] 65.6830μs 40.0089μs 24.9944 KOps/s 24.6969 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-False-True-True-False] 49.1030μs 27.2220μs 36.7350 KOps/s 36.3613 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-False-True-False-True] 46.9820μs 26.4409μs 37.8203 KOps/s 38.2037 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-False-True-False-False] 33.0410μs 16.8063μs 59.5015 KOps/s 59.6310 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-False-True-True] 59.7030μs 41.5975μs 24.0399 KOps/s 24.1115 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-False-True-False] 53.7430μs 28.9203μs 34.5778 KOps/s 33.9456 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[False-False-False-False-True] 46.8920μs 28.0290μs 35.6773 KOps/s 36.0194 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-False-False-False] 37.9020μs 18.6178μs 53.7122 KOps/s 53.9746 KOps/s $\color{#d91a1a}-0.49\%$
test_values[generalized_advantage_estimate-True-True] 24.8756ms 24.2474ms 41.2415 Ops/s 42.0306 Ops/s $\color{#d91a1a}-1.88\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.5886ms 3.2376ms 308.8689 Ops/s 312.0929 Ops/s $\color{#d91a1a}-1.03\%$
test_values[td0_return_estimate-False-False] 92.3540μs 65.0699μs 15.3681 KOps/s 14.9904 KOps/s $\color{#35bf28}+2.52\%$
test_values[td1_return_estimate-False-False] 53.6757ms 53.1784ms 18.8046 Ops/s 18.8269 Ops/s $\color{#d91a1a}-0.12\%$
test_values[vec_td1_return_estimate-False-False] 2.0581ms 1.7615ms 567.7126 Ops/s 565.7796 Ops/s $\color{#35bf28}+0.34\%$
test_values[td_lambda_return_estimate-True-False] 85.3928ms 84.8698ms 11.7828 Ops/s 11.7638 Ops/s $\color{#35bf28}+0.16\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0785ms 1.7569ms 569.1825 Ops/s 566.1742 Ops/s $\color{#35bf28}+0.53\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.8424ms 23.6654ms 42.2559 Ops/s 42.4736 Ops/s $\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8805ms 0.6960ms 1.4368 KOps/s 1.4094 KOps/s $\color{#35bf28}+1.94\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7218ms 0.6486ms 1.5417 KOps/s 1.5339 KOps/s $\color{#35bf28}+0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4923ms 1.4486ms 690.3023 Ops/s 685.7536 Ops/s $\color{#35bf28}+0.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9255ms 0.6663ms 1.5007 KOps/s 1.4884 KOps/s $\color{#35bf28}+0.82\%$
test_dqn_speed 1.8596ms 1.4681ms 681.1556 Ops/s 674.0885 Ops/s $\color{#35bf28}+1.05\%$
test_ddpg_speed 3.0742ms 2.7495ms 363.6960 Ops/s 358.6013 Ops/s $\color{#35bf28}+1.42\%$
test_sac_speed 9.5250ms 8.1208ms 123.1410 Ops/s 122.2201 Ops/s $\color{#35bf28}+0.75\%$
test_redq_speed 11.1968ms 10.1927ms 98.1095 Ops/s 96.8487 Ops/s $\color{#35bf28}+1.30\%$
test_redq_deprec_speed 12.0109ms 11.1878ms 89.3830 Ops/s 89.9349 Ops/s $\color{#d91a1a}-0.61\%$
test_td3_speed 8.2646ms 7.9953ms 125.0742 Ops/s 123.4111 Ops/s $\color{#35bf28}+1.35\%$
test_cql_speed 26.5075ms 25.4359ms 39.3145 Ops/s 39.6247 Ops/s $\color{#d91a1a}-0.78\%$
test_a2c_speed 6.2044ms 5.5718ms 179.4761 Ops/s 183.0444 Ops/s $\color{#d91a1a}-1.95\%$
test_ppo_speed 6.3686ms 5.9314ms 168.5936 Ops/s 169.2438 Ops/s $\color{#d91a1a}-0.38\%$
test_reinforce_speed 4.7758ms 4.5130ms 221.5813 Ops/s 222.7095 Ops/s $\color{#d91a1a}-0.51\%$
test_iql_speed 19.7093ms 19.2379ms 51.9807 Ops/s 51.5779 Ops/s $\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0079ms 2.8961ms 345.2944 Ops/s 344.7843 Ops/s $\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4016ms 0.5404ms 1.8506 KOps/s 1.8212 KOps/s $\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7094ms 0.5222ms 1.9150 KOps/s 1.9043 KOps/s $\color{#35bf28}+0.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0267ms 2.8660ms 348.9130 Ops/s 343.3407 Ops/s $\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6521ms 0.5339ms 1.8729 KOps/s 1.8389 KOps/s $\color{#35bf28}+1.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.2180ms 0.5172ms 1.9336 KOps/s 1.9025 KOps/s $\color{#35bf28}+1.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6543ms 1.4408ms 694.0362 Ops/s 686.6013 Ops/s $\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6011ms 1.3621ms 734.1774 Ops/s 723.0687 Ops/s $\color{#35bf28}+1.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2043ms 3.0219ms 330.9170 Ops/s 329.1378 Ops/s $\color{#35bf28}+0.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3490ms 0.6681ms 1.4967 KOps/s 1.4763 KOps/s $\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7685ms 0.6504ms 1.5375 KOps/s 1.5357 KOps/s $\color{#35bf28}+0.12\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0006ms 2.8862ms 346.4790 Ops/s 349.0869 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1735ms 0.5431ms 1.8414 KOps/s 1.8256 KOps/s $\color{#35bf28}+0.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6301ms 0.5167ms 1.9354 KOps/s 1.9179 KOps/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1221ms 2.9285ms 341.4685 Ops/s 342.4620 Ops/s $\color{#d91a1a}-0.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6374ms 0.5364ms 1.8643 KOps/s 1.8484 KOps/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3281ms 0.5191ms 1.9264 KOps/s 1.9239 KOps/s $\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0944ms 3.0091ms 332.3214 Ops/s 330.6185 Ops/s $\color{#35bf28}+0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4121ms 0.6706ms 1.4912 KOps/s 1.4697 KOps/s $\color{#35bf28}+1.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8207ms 0.6444ms 1.5518 KOps/s 1.5261 KOps/s $\color{#35bf28}+1.68\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1224s 9.3644ms 106.7875 Ops/s 105.8628 Ops/s $\color{#35bf28}+0.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.9854ms 14.7872ms 67.6261 Ops/s 66.2605 Ops/s $\color{#35bf28}+2.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3455ms 1.1640ms 859.1137 Ops/s 845.5056 Ops/s $\color{#35bf28}+1.61\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1154s 7.0449ms 141.9459 Ops/s 141.5560 Ops/s $\color{#35bf28}+0.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.1059ms 14.7108ms 67.9774 Ops/s 66.4633 Ops/s $\color{#35bf28}+2.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.3784ms 1.2906ms 774.8632 Ops/s 771.2838 Ops/s $\color{#35bf28}+0.46\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1182s 9.6829ms 103.2754 Ops/s 103.0753 Ops/s $\color{#35bf28}+0.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.5024ms 15.1373ms 66.0620 Ops/s 65.4315 Ops/s $\color{#35bf28}+0.96\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.4376ms 1.6140ms 619.5725 Ops/s 624.1590 Ops/s $\color{#d91a1a}-0.73\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 25, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.9355ms 53.1957ms 18.7985 Ops/s 16.6726 Ops/s $\textbf{\color{#35bf28}+12.75\%}$
test_sync 38.8864ms 30.4625ms 32.8272 Ops/s 34.0202 Ops/s $\color{#d91a1a}-3.51\%$
test_async 60.7161ms 29.1856ms 34.2635 Ops/s 36.4559 Ops/s $\textbf{\color{#d91a1a}-6.01\%}$
test_simple 0.3233s 0.3196s 3.1292 Ops/s 2.9542 Ops/s $\textbf{\color{#35bf28}+5.93\%}$
test_transformed 0.5254s 0.4722s 2.1176 Ops/s 2.0855 Ops/s $\color{#35bf28}+1.54\%$
test_serial 1.2279s 1.1737s 0.8520 Ops/s 0.8331 Ops/s $\color{#35bf28}+2.27\%$
test_parallel 1.0885s 1.0228s 0.9778 Ops/s 0.9541 Ops/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[True-True-True-True-True] 0.1294ms 21.0083μs 47.6002 KOps/s 48.4481 KOps/s $\color{#d91a1a}-1.75\%$
test_step_mdp_speed[True-True-True-True-False] 40.5850μs 13.0467μs 76.6478 KOps/s 78.1582 KOps/s $\color{#d91a1a}-1.93\%$
test_step_mdp_speed[True-True-True-False-True] 54.7320μs 12.1927μs 82.0162 KOps/s 82.3049 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-True-False-False] 35.8970μs 7.5134μs 133.0957 KOps/s 134.0961 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-True-True] 60.0210μs 22.3705μs 44.7018 KOps/s 44.7859 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-True-False-True-False] 46.6670μs 14.1608μs 70.6174 KOps/s 71.0960 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-True-False-False-True] 34.9350μs 13.5153μs 73.9900 KOps/s 74.1186 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-True-False-False-False] 37.6700μs 8.7800μs 113.8954 KOps/s 116.8981 KOps/s $\color{#d91a1a}-2.57\%$
test_step_mdp_speed[True-False-True-True-True] 77.6850μs 23.6004μs 42.3722 KOps/s 42.6693 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-False-True-True-False] 47.5480μs 15.3208μs 65.2709 KOps/s 66.2455 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[True-False-True-False-True] 48.2300μs 13.5295μs 73.9123 KOps/s 74.7240 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-False-True-False-False] 25.9490μs 8.7290μs 114.5609 KOps/s 115.9324 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-False-False-True-True] 69.9000μs 24.9736μs 40.0422 KOps/s 40.5378 KOps/s $\color{#d91a1a}-1.22\%$
test_step_mdp_speed[True-False-False-True-False] 38.5220μs 16.7716μs 59.6247 KOps/s 61.3722 KOps/s $\color{#d91a1a}-2.85\%$
test_step_mdp_speed[True-False-False-False-True] 36.3880μs 14.6556μs 68.2334 KOps/s 68.5361 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-False-False-False-False] 39.7440μs 9.9658μs 100.3436 KOps/s 102.2478 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-True-True-True-True] 58.3890μs 23.7038μs 42.1874 KOps/s 42.1650 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-True-True-False] 34.7850μs 15.4594μs 64.6854 KOps/s 65.5664 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-True-True-False-True] 33.6030μs 15.8258μs 63.1880 KOps/s 63.1112 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-True-True-False-False] 44.2220μs 10.0471μs 99.5316 KOps/s 101.1640 KOps/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[False-True-False-True-True] 45.7050μs 25.4153μs 39.3464 KOps/s 39.7934 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-True-False-True-False] 46.6070μs 16.7382μs 59.7436 KOps/s 60.3557 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-False-False-True] 41.5570μs 16.9523μs 58.9889 KOps/s 58.5117 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-False-False-False] 36.3780μs 11.1689μs 89.5345 KOps/s 91.0810 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[False-False-True-True-True] 83.7460μs 26.4246μs 37.8435 KOps/s 38.3965 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-False-True-True-False] 53.8600μs 18.0978μs 55.2553 KOps/s 56.8278 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-False-True-False-True] 46.2270μs 17.2297μs 58.0393 KOps/s 59.3818 KOps/s $\color{#d91a1a}-2.26\%$
test_step_mdp_speed[False-False-True-False-False] 46.9180μs 11.1982μs 89.3001 KOps/s 90.1413 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-False-False-True-True] 74.1480μs 27.1636μs 36.8140 KOps/s 37.1004 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-False-True-False] 41.5470μs 19.0068μs 52.6126 KOps/s 53.5022 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-False-False-True] 47.7390μs 17.9882μs 55.5920 KOps/s 55.6551 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-False-False-False] 69.3390μs 12.0700μs 82.8499 KOps/s 82.2820 KOps/s $\color{#35bf28}+0.69\%$
test_values[generalized_advantage_estimate-True-True] 9.4653ms 9.1498ms 109.2915 Ops/s 108.0976 Ops/s $\color{#35bf28}+1.10\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.4438ms 35.0680ms 28.5160 Ops/s 30.0648 Ops/s $\textbf{\color{#d91a1a}-5.15\%}$
test_values[td0_return_estimate-False-False] 0.2309ms 0.1668ms 5.9964 KOps/s 5.8935 KOps/s $\color{#35bf28}+1.75\%$
test_values[td1_return_estimate-False-False] 25.9559ms 22.7148ms 44.0242 Ops/s 43.3096 Ops/s $\color{#35bf28}+1.65\%$
test_values[vec_td1_return_estimate-False-False] 36.7969ms 35.3431ms 28.2941 Ops/s 30.0573 Ops/s $\textbf{\color{#d91a1a}-5.87\%}$
test_values[td_lambda_return_estimate-True-False] 35.8349ms 32.8142ms 30.4747 Ops/s 30.1608 Ops/s $\color{#35bf28}+1.04\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.7300ms 35.3132ms 28.3180 Ops/s 29.6412 Ops/s $\color{#d91a1a}-4.46\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1036ms 7.9847ms 125.2400 Ops/s 122.0650 Ops/s $\color{#35bf28}+2.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3214ms 1.9881ms 503.0022 Ops/s 529.6523 Ops/s $\textbf{\color{#d91a1a}-5.03\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6605ms 0.3696ms 2.7056 KOps/s 2.8336 KOps/s $\color{#d91a1a}-4.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9722ms 46.2441ms 21.6244 Ops/s 25.1750 Ops/s $\textbf{\color{#d91a1a}-14.10\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1196ms 3.0036ms 332.9317 Ops/s 332.1622 Ops/s $\color{#35bf28}+0.23\%$
test_dqn_speed 6.9567ms 1.3432ms 744.4980 Ops/s 679.2825 Ops/s $\textbf{\color{#35bf28}+9.60\%}$
test_ddpg_speed 2.9168ms 2.6584ms 376.1694 Ops/s 367.0707 Ops/s $\color{#35bf28}+2.48\%$
test_sac_speed 9.1796ms 8.1047ms 123.3853 Ops/s 120.8467 Ops/s $\color{#35bf28}+2.10\%$
test_redq_speed 14.1809ms 13.1130ms 76.2603 Ops/s 75.7231 Ops/s $\color{#35bf28}+0.71\%$
test_redq_deprec_speed 15.2862ms 12.9048ms 77.4906 Ops/s 76.0213 Ops/s $\color{#35bf28}+1.93\%$
test_td3_speed 15.9282ms 8.1393ms 122.8604 Ops/s 120.6226 Ops/s $\color{#35bf28}+1.86\%$
test_cql_speed 37.6220ms 35.9539ms 27.8134 Ops/s 27.6288 Ops/s $\color{#35bf28}+0.67\%$
test_a2c_speed 8.4262ms 7.2403ms 138.1165 Ops/s 136.3923 Ops/s $\color{#35bf28}+1.26\%$
test_ppo_speed 8.1673ms 7.5200ms 132.9796 Ops/s 129.5897 Ops/s $\color{#35bf28}+2.62\%$
test_reinforce_speed 7.4889ms 6.4907ms 154.0677 Ops/s 152.5187 Ops/s $\color{#35bf28}+1.02\%$
test_iql_speed 33.4603ms 32.1549ms 31.0995 Ops/s 30.6975 Ops/s $\color{#35bf28}+1.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2265ms 2.1759ms 459.5893 Ops/s 464.0231 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6246ms 0.4900ms 2.0408 KOps/s 2.0369 KOps/s $\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.0167ms 0.4693ms 2.1308 KOps/s 2.1410 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4401ms 2.2380ms 446.8344 Ops/s 443.4997 Ops/s $\color{#35bf28}+0.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8455ms 0.4869ms 2.0539 KOps/s 2.0285 KOps/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7667ms 0.4585ms 2.1809 KOps/s 2.1366 KOps/s $\color{#35bf28}+2.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7037ms 1.2050ms 829.8561 Ops/s 813.2539 Ops/s $\color{#35bf28}+2.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5750ms 1.1403ms 876.9298 Ops/s 866.5896 Ops/s $\color{#35bf28}+1.19\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4322ms 2.3593ms 423.8466 Ops/s 432.2925 Ops/s $\color{#d91a1a}-1.95\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8814ms 0.5995ms 1.6682 KOps/s 1.6385 KOps/s $\color{#35bf28}+1.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1014s 0.6596ms 1.5160 KOps/s 1.6988 KOps/s $\textbf{\color{#d91a1a}-10.76\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3712ms 2.1743ms 459.9132 Ops/s 451.7111 Ops/s $\color{#35bf28}+1.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6990ms 0.4966ms 2.0135 KOps/s 2.0183 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8917ms 0.4712ms 2.1221 KOps/s 2.1226 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5378ms 2.2547ms 443.5127 Ops/s 447.2491 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 95.4602ms 0.6100ms 1.6394 KOps/s 2.0127 KOps/s $\textbf{\color{#d91a1a}-18.55\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5999ms 0.4622ms 2.1638 KOps/s 2.1263 KOps/s $\color{#35bf28}+1.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7651ms 2.4572ms 406.9735 Ops/s 438.4984 Ops/s $\textbf{\color{#d91a1a}-7.19\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0146ms 0.6071ms 1.6472 KOps/s 1.6399 KOps/s $\color{#35bf28}+0.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.9031ms 0.5852ms 1.7089 KOps/s 1.7205 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 96.1284ms 7.2795ms 137.3716 Ops/s 133.3559 Ops/s $\color{#35bf28}+3.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 13.9556ms 11.7763ms 84.9165 Ops/s 83.4801 Ops/s $\color{#35bf28}+1.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.4919ms 1.0207ms 979.7446 Ops/s 923.7687 Ops/s $\textbf{\color{#35bf28}+6.06\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 96.7414ms 5.4594ms 183.1713 Ops/s 182.3703 Ops/s $\color{#35bf28}+0.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.5962ms 11.7905ms 84.8139 Ops/s 71.7349 Ops/s $\textbf{\color{#35bf28}+18.23\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.6461ms 1.0401ms 961.4720 Ops/s 901.1889 Ops/s $\textbf{\color{#35bf28}+6.69\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 96.1566ms 7.5569ms 132.3291 Ops/s 167.8351 Ops/s $\textbf{\color{#d91a1a}-21.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.2943ms 12.3375ms 81.0536 Ops/s 81.8346 Ops/s $\color{#d91a1a}-0.95\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0654ms 1.3534ms 738.9037 Ops/s 695.5383 Ops/s $\textbf{\color{#35bf28}+6.23\%}$

@vmoens vmoens added the bug Something isn't working label Mar 25, 2024
@vmoens vmoens merged commit 1fcd3e3 into main Mar 25, 2024
63 of 67 checks passed
vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[BugFix] More robust _StepMDP and multi-purpose envs (#2038)
b9d0bd6 
(cherry picked from commit 1fcd3e3)
@vmoens vmoens deleted the fix-stepmdp branch April 3, 2024 06:04
vmoens added a commit that referenced this pull request Apr 3, 2024
@vmoens
[BugFix] More robust _StepMDP and multi-purpose envs (#2038)
5bf7192 
(cherry picked from commit 1fcd3e3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot