Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Gym async with _reset full of True #2145

Merged
merged 6 commits into from
May 1, 2024
Merged

[BugFix] Gym async with _reset full of True #2145

merged 6 commits into from
May 1, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented May 1, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented May 1, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2145

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 22 Unrelated Failures

As of commit abe3cc2 with merge base 711a4ee (image):

NEW FAILURE - The following job has failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 1, 2024
@vmoens vmoens added bug Something isn't working Environments Adds or modifies an environment wrapper labels May 1, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented May 1, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 56.9094ms 54.6744ms 18.2901 Ops/s 18.0894 Ops/s $\color{#35bf28}+1.11\%$
test_sync 35.2690ms 29.8149ms 33.5403 Ops/s 33.1662 Ops/s $\color{#35bf28}+1.13\%$
test_async 58.4768ms 28.5169ms 35.0670 Ops/s 35.2188 Ops/s $\color{#d91a1a}-0.43\%$
test_simple 0.4040s 0.3499s 2.8580 Ops/s 2.9045 Ops/s $\color{#d91a1a}-1.60\%$
test_transformed 0.5475s 0.5020s 1.9921 Ops/s 2.0239 Ops/s $\color{#d91a1a}-1.57\%$
test_serial 1.3032s 1.2343s 0.8102 Ops/s 0.8055 Ops/s $\color{#35bf28}+0.58\%$
test_parallel 1.0643s 1.0224s 0.9781 Ops/s 0.9782 Ops/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-True-True-True-True] 0.1414ms 21.8236μs 45.8219 KOps/s 45.8573 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[True-True-True-True-False] 60.7440μs 13.3642μs 74.8266 KOps/s 75.4205 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-True-False-True] 37.5300μs 12.9654μs 77.1282 KOps/s 78.8972 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-True-True-False-False] 29.1650μs 7.8829μs 126.8568 KOps/s 129.9793 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[True-True-False-True-True] 93.9350μs 22.5851μs 44.2769 KOps/s 43.2574 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[True-True-False-True-False] 39.5540μs 14.4132μs 69.3809 KOps/s 69.1391 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-True-False-False-True] 35.6060μs 14.0963μs 70.9405 KOps/s 72.2300 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[True-True-False-False-False] 35.6870μs 9.0440μs 110.5701 KOps/s 112.8420 KOps/s $\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-False-True-True-True] 58.6600μs 24.4773μs 40.8542 KOps/s 40.6117 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-True-True-False] 51.1660μs 16.0786μs 62.1944 KOps/s 62.0840 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-False-True-False-True] 53.0900μs 14.0817μs 71.0142 KOps/s 70.7124 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[True-False-True-False-False] 0.1276ms 9.0676μs 110.2831 KOps/s 111.9972 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[True-False-False-True-True] 52.4780μs 25.5973μs 39.0666 KOps/s 39.1209 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-False-False-True-False] 42.4690μs 17.0348μs 58.7032 KOps/s 58.3920 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-False-False-False-True] 80.8230μs 15.2414μs 65.6108 KOps/s 66.0549 KOps/s $\color{#d91a1a}-0.67\%$
test_step_mdp_speed[True-False-False-False-False] 38.5020μs 10.0326μs 99.6750 KOps/s 98.7772 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-True-True-True] 69.3630μs 23.9240μs 41.7990 KOps/s 41.2717 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-True-True-True-False] 59.3740μs 15.7452μs 63.5114 KOps/s 63.5813 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-True-True-False-True] 0.1003ms 16.0730μs 62.2161 KOps/s 61.5550 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-True-False-False] 57.6780μs 10.1612μs 98.4132 KOps/s 98.7793 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-True-False-True-True] 55.4030μs 25.4282μs 39.3264 KOps/s 38.8422 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-False-True-False] 62.6870μs 16.9172μs 59.1113 KOps/s 58.6422 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-True-False-False-True] 62.7370μs 17.5167μs 57.0883 KOps/s 57.5478 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-True-False-False-False] 32.4800μs 11.4728μs 87.1628 KOps/s 88.4726 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-False-True-True-True] 66.4640μs 27.0038μs 37.0318 KOps/s 36.8527 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-False-True-True-False] 45.1240μs 18.3507μs 54.4937 KOps/s 54.8727 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-True-False-True] 63.2280μs 17.4349μs 57.3564 KOps/s 57.5623 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-True-False-False] 50.3140μs 11.5065μs 86.9074 KOps/s 88.5916 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-False-False-True-True] 42.3190μs 28.1574μs 35.5146 KOps/s 35.5644 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-False-False-True-False] 43.2300μs 19.3070μs 51.7947 KOps/s 52.1675 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-False-False-True] 44.6830μs 18.3583μs 54.4714 KOps/s 54.8876 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-False-False-False-False] 32.9820μs 12.3872μs 80.7282 KOps/s 81.1137 KOps/s $\color{#d91a1a}-0.48\%$
test_values[generalized_advantage_estimate-True-True] 12.5287ms 9.3245ms 107.2443 Ops/s 104.4969 Ops/s $\color{#35bf28}+2.63\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.2160ms 35.3299ms 28.3047 Ops/s 30.0347 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_values[td0_return_estimate-False-False] 0.2269ms 0.1645ms 6.0774 KOps/s 5.9961 KOps/s $\color{#35bf28}+1.36\%$
test_values[td1_return_estimate-False-False] 23.6001ms 23.0192ms 43.4420 Ops/s 43.1001 Ops/s $\color{#35bf28}+0.79\%$
test_values[vec_td1_return_estimate-False-False] 36.8841ms 35.4004ms 28.2482 Ops/s 29.9471 Ops/s $\textbf{\color{#d91a1a}-5.67\%}$
test_values[td_lambda_return_estimate-True-False] 36.4336ms 33.5590ms 29.7982 Ops/s 29.7542 Ops/s $\color{#35bf28}+0.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.3234ms 35.2753ms 28.3484 Ops/s 29.9595 Ops/s $\textbf{\color{#d91a1a}-5.38\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2326ms 8.0747ms 123.8431 Ops/s 119.1530 Ops/s $\color{#35bf28}+3.94\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2092ms 2.0279ms 493.1099 Ops/s 527.8814 Ops/s $\textbf{\color{#d91a1a}-6.59\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5549ms 0.3542ms 2.8236 KOps/s 2.8131 KOps/s $\color{#35bf28}+0.37\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.0565ms 43.4527ms 23.0135 Ops/s 21.6480 Ops/s $\textbf{\color{#35bf28}+6.31\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6309ms 3.0161ms 331.5509 Ops/s 329.2697 Ops/s $\color{#35bf28}+0.69\%$
test_dqn_speed 7.4873ms 1.3784ms 725.4908 Ops/s 727.1890 Ops/s $\color{#d91a1a}-0.23\%$
test_ddpg_speed 3.2365ms 2.8915ms 345.8425 Ops/s 342.6844 Ops/s $\color{#35bf28}+0.92\%$
test_sac_speed 9.6321ms 8.5147ms 117.4435 Ops/s 104.0062 Ops/s $\textbf{\color{#35bf28}+12.92\%}$
test_redq_speed 13.9086ms 13.0555ms 76.5963 Ops/s 74.6973 Ops/s $\color{#35bf28}+2.54\%$
test_redq_deprec_speed 88.6498ms 14.1505ms 70.6689 Ops/s 74.6721 Ops/s $\textbf{\color{#d91a1a}-5.36\%}$
test_td3_speed 9.2139ms 8.4421ms 118.4543 Ops/s 116.9817 Ops/s $\color{#35bf28}+1.26\%$
test_cql_speed 39.8965ms 36.7683ms 27.1973 Ops/s 27.2126 Ops/s $\color{#d91a1a}-0.06\%$
test_a2c_speed 8.9118ms 7.4594ms 134.0594 Ops/s 133.4168 Ops/s $\color{#35bf28}+0.48\%$
test_ppo_speed 8.6493ms 7.6789ms 130.2263 Ops/s 128.4871 Ops/s $\color{#35bf28}+1.35\%$
test_reinforce_speed 7.4956ms 6.6489ms 150.4016 Ops/s 148.5453 Ops/s $\color{#35bf28}+1.25\%$
test_iql_speed 33.8420ms 32.9189ms 30.3776 Ops/s 30.1195 Ops/s $\color{#35bf28}+0.86\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8896ms 2.0625ms 484.8501 Ops/s 472.1228 Ops/s $\color{#35bf28}+2.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4870ms 0.5005ms 1.9979 KOps/s 2.0125 KOps/s $\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7649ms 0.4768ms 2.0974 KOps/s 2.0999 KOps/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4137ms 2.0643ms 484.4354 Ops/s 473.4455 Ops/s $\color{#35bf28}+2.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6144ms 0.4886ms 2.0467 KOps/s 2.0340 KOps/s $\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.3220ms 0.4727ms 2.1153 KOps/s 2.1327 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8035ms 1.2530ms 798.0639 Ops/s 807.1490 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.2904ms 1.1829ms 845.3972 Ops/s 852.3145 Ops/s $\color{#d91a1a}-0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2376ms 2.2202ms 450.4006 Ops/s 443.5010 Ops/s $\color{#35bf28}+1.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6277ms 0.6511ms 1.5359 KOps/s 1.6147 KOps/s $\color{#d91a1a}-4.88\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8868ms 0.5963ms 1.6769 KOps/s 1.6856 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1331ms 2.1391ms 467.4851 Ops/s 462.3329 Ops/s $\color{#35bf28}+1.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0286ms 0.5042ms 1.9835 KOps/s 1.9937 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.6924ms 0.5079ms 1.9690 KOps/s 2.0763 KOps/s $\textbf{\color{#d91a1a}-5.17\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1935ms 2.1145ms 472.9222 Ops/s 472.0049 Ops/s $\color{#35bf28}+0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6003ms 0.4940ms 2.0243 KOps/s 2.0391 KOps/s $\color{#d91a1a}-0.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.5896ms 0.4814ms 2.0772 KOps/s 2.1135 KOps/s $\color{#d91a1a}-1.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3960ms 2.1808ms 458.5375 Ops/s 443.5262 Ops/s $\color{#35bf28}+3.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0000ms 0.6223ms 1.6068 KOps/s 1.6239 KOps/s $\color{#d91a1a}-1.05\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6999ms 0.5907ms 1.6928 KOps/s 1.6789 KOps/s $\color{#35bf28}+0.83\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1102s 7.8276ms 127.7537 Ops/s 166.3904 Ops/s $\textbf{\color{#d91a1a}-23.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.7811ms 12.0612ms 82.9108 Ops/s 79.0409 Ops/s $\color{#35bf28}+4.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5876ms 1.0794ms 926.4132 Ops/s 970.1710 Ops/s $\color{#d91a1a}-4.51\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1005s 5.5809ms 179.1835 Ops/s 178.0883 Ops/s $\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1066s 13.9265ms 71.8055 Ops/s 70.2412 Ops/s $\color{#35bf28}+2.23\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.6401ms 1.1015ms 907.8495 Ops/s 779.5545 Ops/s $\textbf{\color{#35bf28}+16.46\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1015s 6.0200ms 166.1123 Ops/s 167.6849 Ops/s $\color{#d91a1a}-0.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.8976ms 12.4066ms 80.6022 Ops/s 80.0421 Ops/s $\color{#35bf28}+0.70\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.1051ms 1.4309ms 698.8514 Ops/s 740.1471 Ops/s $\textbf{\color{#d91a1a}-5.58\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented May 1, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1143s 0.1139s 8.7818 Ops/s 8.5099 Ops/s $\color{#35bf28}+3.20\%$
test_sync 0.1024s 0.1016s 9.8468 Ops/s 9.8226 Ops/s $\color{#35bf28}+0.25\%$
test_async 0.1919s 95.4573ms 10.4759 Ops/s 10.3362 Ops/s $\color{#35bf28}+1.35\%$
test_single_pixels 0.1283s 0.1261s 7.9281 Ops/s 7.9365 Ops/s $\color{#d91a1a}-0.11\%$
test_sync_pixels 0.1367s 85.1630ms 11.7422 Ops/s 11.4932 Ops/s $\color{#35bf28}+2.17\%$
test_async_pixels 0.1582s 77.6318ms 12.8813 Ops/s 13.0072 Ops/s $\color{#d91a1a}-0.97\%$
test_simple 0.8223s 0.7635s 1.3098 Ops/s 1.3012 Ops/s $\color{#35bf28}+0.66\%$
test_transformed 1.0636s 1.0064s 0.9937 Ops/s 0.9847 Ops/s $\color{#35bf28}+0.92\%$
test_serial 2.4625s 2.4046s 0.4159 Ops/s 0.4113 Ops/s $\color{#35bf28}+1.11\%$
test_parallel 2.3596s 2.3199s 0.4311 Ops/s 0.4347 Ops/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-True-True-True] 0.1002ms 32.6045μs 30.6706 KOps/s 29.8229 KOps/s $\color{#35bf28}+2.84\%$
test_step_mdp_speed[True-True-True-True-False] 0.2123ms 19.9735μs 50.0664 KOps/s 49.2648 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-False-True] 0.2053ms 18.9505μs 52.7691 KOps/s 51.9330 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-True-True-False-False] 28.1620μs 11.3991μs 87.7263 KOps/s 86.6857 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-False-True-True] 0.2214ms 34.9463μs 28.6153 KOps/s 28.3305 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[True-True-False-True-False] 0.2036ms 21.5624μs 46.3770 KOps/s 45.1074 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-True-False-False-True] 0.2072ms 20.9106μs 47.8226 KOps/s 47.2368 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-False-False-False] 29.6510μs 13.3821μs 74.7266 KOps/s 74.0243 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-False-True-True-True] 0.2215ms 36.8430μs 27.1422 KOps/s 26.9094 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-False-True-True-False] 0.2109ms 23.5838μs 42.4019 KOps/s 41.6963 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-True-False-True] 40.7020μs 20.6724μs 48.3737 KOps/s 47.2938 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-False-True-False-False] 0.1946ms 13.2598μs 75.4158 KOps/s 74.0639 KOps/s $\color{#35bf28}+1.83\%$
test_step_mdp_speed[True-False-False-True-True] 0.2200ms 38.8552μs 25.7366 KOps/s 25.7123 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[True-False-False-True-False] 0.2056ms 25.5030μs 39.2111 KOps/s 38.4718 KOps/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-False-False-False-True] 49.3330μs 22.6361μs 44.1772 KOps/s 43.6296 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-False-False-False-False] 0.2014ms 15.1299μs 66.0944 KOps/s 64.9180 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[False-True-True-True-True] 0.2199ms 37.1757μs 26.8993 KOps/s 26.5484 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-True-True-True-False] 0.2126ms 23.7550μs 42.0963 KOps/s 41.7280 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-True-True-False-True] 51.5430μs 24.9511μs 40.0785 KOps/s 40.3171 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-True-False-False] 0.2035ms 15.3067μs 65.3310 KOps/s 65.6918 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-True-False-True-True] 0.2286ms 38.6545μs 25.8702 KOps/s 25.3945 KOps/s $\color{#35bf28}+1.87\%$
test_step_mdp_speed[False-True-False-True-False] 48.5430μs 25.3664μs 39.4222 KOps/s 38.4169 KOps/s $\color{#35bf28}+2.62\%$
test_step_mdp_speed[False-True-False-False-True] 48.3430μs 26.6782μs 37.4838 KOps/s 37.4608 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-False-False-False] 0.2005ms 16.8904μs 59.2051 KOps/s 58.4133 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-False-True-True-True] 0.2220ms 41.0141μs 24.3819 KOps/s 24.2912 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-False-True-True-False] 0.2082ms 27.3084μs 36.6187 KOps/s 35.9540 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[False-False-True-False-True] 42.6530μs 26.8545μs 37.2377 KOps/s 36.6856 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-False-True-False-False] 0.1967ms 17.2280μs 58.0451 KOps/s 59.0225 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-False-True-True] 65.1540μs 42.9494μs 23.2832 KOps/s 23.0102 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-False-False-True-False] 0.2202ms 29.5867μs 33.7990 KOps/s 33.4508 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-False-False-True] 0.2119ms 28.3877μs 35.2265 KOps/s 35.2494 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-False-False-False-False] 44.2930μs 18.9168μs 52.8631 KOps/s 53.2856 KOps/s $\color{#d91a1a}-0.79\%$
test_values[generalized_advantage_estimate-True-True] 24.2078ms 23.8072ms 42.0042 Ops/s 42.7681 Ops/s $\color{#d91a1a}-1.79\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.0950ms 3.2436ms 308.2956 Ops/s 298.2044 Ops/s $\color{#35bf28}+3.38\%$
test_values[td0_return_estimate-False-False] 91.4260μs 62.1652μs 16.0862 KOps/s 15.6785 KOps/s $\color{#35bf28}+2.60\%$
test_values[td1_return_estimate-False-False] 51.6049ms 51.0475ms 19.5896 Ops/s 19.9379 Ops/s $\color{#d91a1a}-1.75\%$
test_values[vec_td1_return_estimate-False-False] 2.0223ms 1.7476ms 572.1982 Ops/s 571.6551 Ops/s $\color{#35bf28}+0.10\%$
test_values[td_lambda_return_estimate-True-False] 82.6576ms 81.7539ms 12.2318 Ops/s 12.4545 Ops/s $\color{#d91a1a}-1.79\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0724ms 1.7443ms 573.2874 Ops/s 573.5107 Ops/s $\color{#d91a1a}-0.04\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.9638ms 22.7604ms 43.9359 Ops/s 45.3020 Ops/s $\color{#d91a1a}-3.02\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8614ms 0.6819ms 1.4666 KOps/s 1.4623 KOps/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6880ms 0.6340ms 1.5774 KOps/s 1.5780 KOps/s $\color{#d91a1a}-0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 2.2060ms 1.4410ms 693.9394 Ops/s 693.9727 Ops/s $-0.00\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9268ms 0.6509ms 1.5364 KOps/s 1.5409 KOps/s $\color{#d91a1a}-0.29\%$
test_dqn_speed 1.9924ms 1.4213ms 703.5567 Ops/s 714.7386 Ops/s $\color{#d91a1a}-1.56\%$
test_ddpg_speed 3.0299ms 2.8902ms 345.9993 Ops/s 346.9932 Ops/s $\color{#d91a1a}-0.29\%$
test_sac_speed 8.7593ms 8.2991ms 120.4953 Ops/s 120.4371 Ops/s $\color{#35bf28}+0.05\%$
test_redq_speed 11.4902ms 10.0073ms 99.9266 Ops/s 98.5743 Ops/s $\color{#35bf28}+1.37\%$
test_redq_deprec_speed 97.1701ms 12.3823ms 80.7601 Ops/s 86.7098 Ops/s $\textbf{\color{#d91a1a}-6.86\%}$
test_td3_speed 8.3284ms 8.2493ms 121.2229 Ops/s 121.7684 Ops/s $\color{#d91a1a}-0.45\%$
test_cql_speed 27.2008ms 25.4449ms 39.3006 Ops/s 39.1957 Ops/s $\color{#35bf28}+0.27\%$
test_a2c_speed 6.0763ms 5.6462ms 177.1108 Ops/s 176.9258 Ops/s $\color{#35bf28}+0.10\%$
test_ppo_speed 6.4053ms 5.9178ms 168.9823 Ops/s 167.9283 Ops/s $\color{#35bf28}+0.63\%$
test_reinforce_speed 4.8409ms 4.6795ms 213.6986 Ops/s 216.8771 Ops/s $\color{#d91a1a}-1.47\%$
test_iql_speed 20.1339ms 19.4947ms 51.2961 Ops/s 50.2837 Ops/s $\color{#35bf28}+2.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0444ms 2.8180ms 354.8603 Ops/s 356.8930 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7154ms 0.5873ms 1.7026 KOps/s 1.6780 KOps/s $\color{#35bf28}+1.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6409ms 0.5676ms 1.7619 KOps/s 1.7584 KOps/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0243ms 2.8020ms 356.8853 Ops/s 361.2623 Ops/s $\color{#d91a1a}-1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7213ms 0.5788ms 1.7278 KOps/s 1.7100 KOps/s $\color{#35bf28}+1.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6763ms 0.5592ms 1.7883 KOps/s 1.7673 KOps/s $\color{#35bf28}+1.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5914ms 1.4817ms 674.9005 Ops/s 669.5940 Ops/s $\color{#35bf28}+0.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5827ms 1.4114ms 708.5092 Ops/s 700.3791 Ops/s $\color{#35bf28}+1.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0403ms 2.8901ms 346.0061 Ops/s 341.1733 Ops/s $\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4258ms 0.7153ms 1.3981 KOps/s 1.3869 KOps/s $\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9003ms 0.6884ms 1.4526 KOps/s 1.4452 KOps/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9925ms 2.7984ms 357.3471 Ops/s 358.9469 Ops/s $\color{#d91a1a}-0.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7163ms 0.5900ms 1.6950 KOps/s 1.6835 KOps/s $\color{#35bf28}+0.68\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.9001ms 0.5763ms 1.7351 KOps/s 1.7657 KOps/s $\color{#d91a1a}-1.73\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9889ms 2.8257ms 353.8900 Ops/s 354.2879 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5266ms 0.5824ms 1.7170 KOps/s 1.7165 KOps/s $\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7274ms 0.5602ms 1.7851 KOps/s 1.7857 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0103ms 2.9349ms 340.7313 Ops/s 339.0669 Ops/s $\color{#35bf28}+0.49\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8641ms 0.7139ms 1.4008 KOps/s 1.3936 KOps/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.8991ms 0.6975ms 1.4336 KOps/s 1.4465 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1353s 7.4186ms 134.7963 Ops/s 105.9179 Ops/s $\textbf{\color{#35bf28}+27.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3516ms 14.7424ms 67.8315 Ops/s 67.3548 Ops/s $\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.2187ms 1.1460ms 872.6003 Ops/s 747.2154 Ops/s $\textbf{\color{#35bf28}+16.78\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1190s 9.3105ms 107.4055 Ops/s 141.8167 Ops/s $\textbf{\color{#d91a1a}-24.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.0875ms 14.8591ms 67.2988 Ops/s 67.5764 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.2548ms 1.1484ms 870.7758 Ops/s 837.2518 Ops/s $\color{#35bf28}+4.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1150s 7.3447ms 136.1521 Ops/s 133.6693 Ops/s $\color{#35bf28}+1.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.7321ms 15.2149ms 65.7251 Ops/s 57.1969 Ops/s $\textbf{\color{#35bf28}+14.91\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.8148ms 1.6393ms 610.0233 Ops/s 653.0739 Ops/s $\textbf{\color{#d91a1a}-6.59\%}$

@vmoens vmoens merged commit 7109a3f into main May 1, 2024
17 of 19 checks passed
@vmoens vmoens deleted the fix-non-returning-transform branch May 1, 2024 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Environments Adds or modifies an environment wrapper
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot