Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Fix policy in getting started #2429

Merged
merged 1 commit into from
Sep 10, 2024
Merged

[Doc] Fix policy in getting started #2429

merged 1 commit into from
Sep 10, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 10, 2024 •
edited
Loading

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 10, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2429

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 10 Unrelated Failures

As of commit ee4f256 with merge base 0ad8e59 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 10, 2024
@vmoens vmoens added the documentation Improvements or additions to documentation label Sep 10, 2024
@vmoens vmoens merged commit 8b1195a into main Sep 10, 2024
50 of 58 checks passed
@vmoens vmoens deleted the correct-dqn-tuto branch September 10, 2024 10:47
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.1812ms 59.7405ms 16.7391 Ops/s 16.6138 Ops/s $\color{#35bf28}+0.75\%$
test_sync 39.2980ms 33.1622ms 30.1548 Ops/s 31.6576 Ops/s $\color{#d91a1a}-4.75\%$
test_async 67.2407ms 30.9097ms 32.3523 Ops/s 32.7608 Ops/s $\color{#d91a1a}-1.25\%$
test_simple 0.5042s 0.4238s 2.3595 Ops/s 2.4791 Ops/s $\color{#d91a1a}-4.82\%$
test_transformed 0.6546s 0.5837s 1.7133 Ops/s 1.7045 Ops/s $\color{#35bf28}+0.51\%$
test_serial 1.3566s 1.2887s 0.7760 Ops/s 0.7717 Ops/s $\color{#35bf28}+0.55\%$
test_parallel 1.1929s 1.1217s 0.8915 Ops/s 0.8745 Ops/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-True-True-True-True] 0.2383ms 27.5277μs 36.3271 KOps/s 35.8260 KOps/s $\color{#35bf28}+1.40\%$
test_step_mdp_speed[True-True-True-True-False] 46.7270μs 16.1902μs 61.7659 KOps/s 60.8865 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-True-True-False-True] 59.2510μs 15.9464μs 62.7101 KOps/s 62.9846 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-True-False-False] 35.4760μs 9.3546μs 106.8991 KOps/s 105.0717 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[True-True-False-True-True] 75.7820μs 29.4055μs 34.0072 KOps/s 33.6104 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-True-False-True-False] 49.5620μs 17.8144μs 56.1344 KOps/s 55.5690 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[True-True-False-False-True] 50.0630μs 17.5463μs 56.9921 KOps/s 57.1941 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-False-False-False] 39.2430μs 10.8798μs 91.9133 KOps/s 90.1404 KOps/s $\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-False-True-True-True] 71.2830μs 31.1767μs 32.0752 KOps/s 31.7869 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-True-False] 77.6520μs 19.1109μs 52.3262 KOps/s 50.0977 KOps/s $\color{#35bf28}+4.45\%$
test_step_mdp_speed[True-False-True-False-True] 49.7230μs 17.5657μs 56.9292 KOps/s 55.9921 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-True-False-False] 36.0770μs 10.9957μs 90.9449 KOps/s 89.8301 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-False-True-True] 71.0420μs 32.7423μs 30.5416 KOps/s 30.0238 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-False-False-True-False] 46.9570μs 21.0979μs 47.3981 KOps/s 42.9984 KOps/s $\textbf{\color{#35bf28}+10.23\%}$
test_step_mdp_speed[True-False-False-False-True] 60.7430μs 19.1607μs 52.1902 KOps/s 52.0274 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-False-False] 34.6450μs 12.3173μs 81.1867 KOps/s 78.8671 KOps/s $\color{#35bf28}+2.94\%$
test_step_mdp_speed[False-True-True-True-True] 60.5530μs 31.0225μs 32.2347 KOps/s 31.5645 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-True-True-True-False] 43.0300μs 19.4760μs 51.3452 KOps/s 49.8387 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-True-True-False-True] 47.5580μs 19.8661μs 50.3370 KOps/s 49.4958 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-True-True-False-False] 45.6750μs 12.0845μs 82.7505 KOps/s 81.7052 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-True-False-True-True] 64.5500μs 32.3683μs 30.8945 KOps/s 30.0220 KOps/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-True-False-True-False] 44.6540μs 20.9911μs 47.6392 KOps/s 46.3485 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-False-False-True] 3.2663ms 21.3695μs 46.7956 KOps/s 45.9714 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-True-False-False-False] 39.2530μs 13.6778μs 73.1110 KOps/s 72.6232 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-True-True] 74.9800μs 34.1625μs 29.2719 KOps/s 29.1157 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-False-True-True-False] 47.0180μs 22.6960μs 44.0607 KOps/s 43.7149 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-False-True-False-True] 50.3030μs 21.8119μs 45.8466 KOps/s 46.3840 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-False-True-False-False] 39.8940μs 13.6446μs 73.2892 KOps/s 71.3836 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-False-False-True-True] 88.5350μs 35.3716μs 28.2713 KOps/s 27.7496 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-False-True-False] 65.0010μs 23.8538μs 41.9220 KOps/s 40.5797 KOps/s $\color{#35bf28}+3.31\%$
test_step_mdp_speed[False-False-False-False-True] 55.7140μs 22.7547μs 43.9470 KOps/s 43.0832 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[False-False-False-False-False] 56.5760μs 15.0021μs 66.6572 KOps/s 64.9241 KOps/s $\color{#35bf28}+2.67\%$
test_values[generalized_advantage_estimate-True-True] 10.8157ms 9.5584ms 104.6197 Ops/s 105.8131 Ops/s $\color{#d91a1a}-1.13\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.8706ms 33.5120ms 29.8400 Ops/s 27.5610 Ops/s $\textbf{\color{#35bf28}+8.27\%}$
test_values[td0_return_estimate-False-False] 0.2449ms 0.1678ms 5.9585 KOps/s 5.3551 KOps/s $\textbf{\color{#35bf28}+11.27\%}$
test_values[td1_return_estimate-False-False] 27.5835ms 24.0152ms 41.6404 Ops/s 42.0472 Ops/s $\color{#d91a1a}-0.97\%$
test_values[vec_td1_return_estimate-False-False] 35.2563ms 33.5199ms 29.8331 Ops/s 27.5484 Ops/s $\textbf{\color{#35bf28}+8.29\%}$
test_values[td_lambda_return_estimate-True-False] 42.9721ms 34.9812ms 28.5868 Ops/s 29.5525 Ops/s $\color{#d91a1a}-3.27\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.5507ms 33.6017ms 29.7604 Ops/s 27.5258 Ops/s $\textbf{\color{#35bf28}+8.12\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.8810ms 8.3874ms 119.2262 Ops/s 119.6219 Ops/s $\color{#d91a1a}-0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4570ms 1.9751ms 506.3085 Ops/s 499.6850 Ops/s $\color{#35bf28}+1.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4579ms 0.3542ms 2.8236 KOps/s 2.8527 KOps/s $\color{#d91a1a}-1.02\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9219ms 46.1160ms 21.6844 Ops/s 22.6951 Ops/s $\color{#d91a1a}-4.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1382ms 3.0321ms 329.8077 Ops/s 330.6377 Ops/s $\color{#d91a1a}-0.25\%$
test_dqn_speed 6.3002ms 1.3199ms 757.6150 Ops/s 766.1864 Ops/s $\color{#d91a1a}-1.12\%$
test_ddpg_speed 3.5426ms 2.7486ms 363.8168 Ops/s 369.4359 Ops/s $\color{#d91a1a}-1.52\%$
test_sac_speed 8.4559ms 7.9926ms 125.1157 Ops/s 125.0840 Ops/s $\color{#35bf28}+0.03\%$
test_redq_speed 15.0229ms 12.8384ms 77.8916 Ops/s 75.7964 Ops/s $\color{#35bf28}+2.76\%$
test_redq_deprec_speed 13.6396ms 12.8264ms 77.9641 Ops/s 76.5766 Ops/s $\color{#35bf28}+1.81\%$
test_td3_speed 8.2352ms 8.0126ms 124.8032 Ops/s 122.4115 Ops/s $\color{#35bf28}+1.95\%$
test_cql_speed 36.6003ms 35.3845ms 28.2609 Ops/s 28.1559 Ops/s $\color{#35bf28}+0.37\%$
test_a2c_speed 8.5519ms 7.3034ms 136.9218 Ops/s 136.4245 Ops/s $\color{#35bf28}+0.36\%$
test_ppo_speed 8.6091ms 7.6417ms 130.8606 Ops/s 131.9118 Ops/s $\color{#d91a1a}-0.80\%$
test_reinforce_speed 7.9729ms 6.6654ms 150.0290 Ops/s 150.9582 Ops/s $\color{#d91a1a}-0.62\%$
test_iql_speed 33.5109ms 31.8169ms 31.4298 Ops/s 30.8737 Ops/s $\color{#35bf28}+1.80\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.3463ms 4.8527ms 206.0692 Ops/s 203.5287 Ops/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9353ms 0.4719ms 2.1193 KOps/s 2.1337 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8055ms 0.4504ms 2.2200 KOps/s 2.1931 KOps/s $\color{#35bf28}+1.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2278ms 4.7563ms 210.2454 Ops/s 201.6702 Ops/s $\color{#35bf28}+4.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8608ms 0.4662ms 2.1452 KOps/s 2.1100 KOps/s $\color{#35bf28}+1.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7826ms 0.4437ms 2.2539 KOps/s 2.2208 KOps/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4131ms 1.5928ms 627.8356 Ops/s 633.5013 Ops/s $\color{#d91a1a}-0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2904ms 1.5034ms 665.1680 Ops/s 666.4884 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1756ms 4.9369ms 202.5546 Ops/s 195.1383 Ops/s $\color{#35bf28}+3.80\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9097ms 0.6097ms 1.6401 KOps/s 1.6451 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9637ms 0.5816ms 1.7193 KOps/s 1.7229 KOps/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.0220ms 4.8871ms 204.6203 Ops/s 200.2203 Ops/s $\color{#35bf28}+2.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8743ms 0.4771ms 2.0960 KOps/s 2.1074 KOps/s $\color{#d91a1a}-0.54\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6117ms 0.4479ms 2.2327 KOps/s 2.2471 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 8.2236ms 4.8208ms 207.4330 Ops/s 207.5674 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1455s 0.6554ms 1.5257 KOps/s 2.1211 KOps/s $\textbf{\color{#d91a1a}-28.07\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6042ms 0.4412ms 2.2665 KOps/s 2.2434 KOps/s $\color{#35bf28}+1.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.3331ms 5.0059ms 199.7647 Ops/s 199.5582 Ops/s $\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7340ms 0.6036ms 1.6568 KOps/s 1.6405 KOps/s $\color{#35bf28}+0.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9209ms 0.5903ms 1.6940 KOps/s 1.7253 KOps/s $\color{#d91a1a}-1.81\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1254s 6.3525ms 157.4180 Ops/s 150.5130 Ops/s $\color{#35bf28}+4.59\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.9090ms 12.9053ms 77.4875 Ops/s 77.0556 Ops/s $\color{#35bf28}+0.56\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.9482ms 1.2365ms 808.7443 Ops/s 813.9804 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1255s 8.6658ms 115.3963 Ops/s 146.0588 Ops/s $\textbf{\color{#d91a1a}-20.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.5735ms 12.9965ms 76.9439 Ops/s 77.2374 Ops/s $\color{#d91a1a}-0.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.9361ms 1.2418ms 805.3024 Ops/s 737.2787 Ops/s $\textbf{\color{#35bf28}+9.23\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1202s 6.4156ms 155.8693 Ops/s 108.9017 Ops/s $\textbf{\color{#35bf28}+43.13\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.6108ms 13.0478ms 76.6415 Ops/s 74.4520 Ops/s $\color{#35bf28}+2.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0781ms 1.3812ms 724.0302 Ops/s 694.2103 Ops/s $\color{#35bf28}+4.30\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1026s 0.1022s 9.7839 Ops/s 9.0263 Ops/s $\textbf{\color{#35bf28}+8.39\%}$
test_sync 93.4917ms 89.4488ms 11.1796 Ops/s 11.1938 Ops/s $\color{#d91a1a}-0.13\%$
test_async 0.1674s 84.5243ms 11.8309 Ops/s 11.8260 Ops/s $\color{#35bf28}+0.04\%$
test_single_pixels 0.1091s 0.1087s 9.1956 Ops/s 9.2841 Ops/s $\color{#d91a1a}-0.95\%$
test_sync_pixels 72.1116ms 70.9791ms 14.0887 Ops/s 13.9813 Ops/s $\color{#35bf28}+0.77\%$
test_async_pixels 0.1277s 67.4079ms 14.8351 Ops/s 14.9403 Ops/s $\color{#d91a1a}-0.70\%$
test_simple 0.7254s 0.7236s 1.3819 Ops/s 1.3538 Ops/s $\color{#35bf28}+2.08\%$
test_transformed 0.9528s 0.9523s 1.0501 Ops/s 1.0390 Ops/s $\color{#35bf28}+1.07\%$
test_serial 2.1589s 2.0853s 0.4796 Ops/s 0.4862 Ops/s $\color{#d91a1a}-1.37\%$
test_parallel 1.9566s 1.8699s 0.5348 Ops/s 0.5366 Ops/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-True-True] 0.1954ms 37.1383μs 26.9263 KOps/s 26.7407 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-True-True-False] 67.1220μs 21.2074μs 47.1533 KOps/s 45.9365 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-True-True-False-True] 60.9910μs 20.9287μs 47.7813 KOps/s 46.6741 KOps/s $\color{#35bf28}+2.37\%$
test_step_mdp_speed[True-True-True-False-False] 0.1267ms 12.1183μs 82.5198 KOps/s 81.6488 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-False-True-True] 93.4620μs 38.8455μs 25.7430 KOps/s 24.9290 KOps/s $\color{#35bf28}+3.27\%$
test_step_mdp_speed[True-True-False-True-False] 55.5810μs 23.2908μs 42.9354 KOps/s 42.3482 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-False-True] 53.1810μs 23.1690μs 43.1612 KOps/s 42.3778 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-False-False-False] 46.0110μs 14.2134μs 70.3562 KOps/s 68.7144 KOps/s $\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-False-True-True-True] 68.8510μs 41.4925μs 24.1007 KOps/s 23.4903 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-False-True-True-False] 55.9220μs 25.6208μs 39.0308 KOps/s 38.6205 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-True-False-True] 53.3310μs 23.0159μs 43.4481 KOps/s 42.0592 KOps/s $\color{#35bf28}+3.30\%$
test_step_mdp_speed[True-False-True-False-False] 40.3910μs 14.3095μs 69.8837 KOps/s 69.7184 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-False-True-True] 0.1105ms 43.0334μs 23.2378 KOps/s 22.7061 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-False-False-True-False] 56.4810μs 27.4932μs 36.3726 KOps/s 35.4714 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-False-False-False-True] 0.1024ms 25.3386μs 39.4655 KOps/s 39.2339 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-False-False-False] 0.1089ms 16.1658μs 61.8590 KOps/s 60.4942 KOps/s $\color{#35bf28}+2.26\%$
test_step_mdp_speed[False-True-True-True-True] 69.2810μs 41.6380μs 24.0165 KOps/s 23.7421 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-True-True-True-False] 77.4020μs 25.8532μs 38.6799 KOps/s 38.0157 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-True-True-False-True] 63.9910μs 26.0644μs 38.3665 KOps/s 37.4294 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-True-True-False-False] 41.1710μs 15.7951μs 63.3108 KOps/s 61.8645 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[False-True-False-True-True] 73.8820μs 43.4103μs 23.0360 KOps/s 22.5889 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[False-True-False-True-False] 78.2520μs 27.5604μs 36.2839 KOps/s 35.3337 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[False-True-False-False-True] 3.6576ms 28.3524μs 35.2704 KOps/s 34.6735 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[False-True-False-False-False] 84.7120μs 17.8559μs 56.0040 KOps/s 54.9328 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-False-True-True-True] 77.0920μs 45.8889μs 21.7917 KOps/s 21.7855 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-False-True-True-False] 57.3010μs 29.7167μs 33.6511 KOps/s 33.5296 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-True-False-True] 63.2010μs 28.3549μs 35.2672 KOps/s 35.7628 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[False-False-True-False-False] 48.2710μs 17.7794μs 56.2448 KOps/s 56.5821 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-False-False-True-True] 76.0020μs 47.0274μs 21.2642 KOps/s 21.4284 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-False-True-False] 59.3510μs 31.3890μs 31.8583 KOps/s 31.8379 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-False-False-False-True] 0.1416ms 28.4235μs 35.1821 KOps/s 33.9265 KOps/s $\color{#35bf28}+3.70\%$
test_step_mdp_speed[False-False-False-False-False] 47.0710μs 19.4318μs 51.4620 KOps/s 51.5255 KOps/s $\color{#d91a1a}-0.12\%$
test_values[generalized_advantage_estimate-True-True] 24.6669ms 24.1295ms 41.4430 Ops/s 39.9959 Ops/s $\color{#35bf28}+3.62\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1166s 3.2119ms 311.3402 Ops/s 317.7863 Ops/s $\color{#d91a1a}-2.03\%$
test_values[td0_return_estimate-False-False] 93.9720μs 64.9850μs 15.3882 KOps/s 15.5393 KOps/s $\color{#d91a1a}-0.97\%$
test_values[td1_return_estimate-False-False] 55.1804ms 54.4611ms 18.3617 Ops/s 18.3640 Ops/s $\color{#d91a1a}-0.01\%$
test_values[vec_td1_return_estimate-False-False] 1.2887ms 1.0656ms 938.4279 Ops/s 944.7112 Ops/s $\color{#d91a1a}-0.67\%$
test_values[td_lambda_return_estimate-True-False] 91.7697ms 86.6982ms 11.5343 Ops/s 11.3191 Ops/s $\color{#35bf28}+1.90\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2603ms 1.0540ms 948.7686 Ops/s 946.5330 Ops/s $\color{#35bf28}+0.24\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.5999ms 24.4404ms 40.9159 Ops/s 41.6389 Ops/s $\color{#d91a1a}-1.74\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9504ms 0.7141ms 1.4004 KOps/s 1.4237 KOps/s $\color{#d91a1a}-1.63\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7832ms 0.6466ms 1.5466 KOps/s 1.5572 KOps/s $\color{#d91a1a}-0.68\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6075ms 1.4551ms 687.2451 Ops/s 689.5839 Ops/s $\color{#d91a1a}-0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8151ms 0.6632ms 1.5078 KOps/s 1.5168 KOps/s $\color{#d91a1a}-0.59\%$
test_dqn_speed 7.7221ms 1.3372ms 747.8246 Ops/s 734.3361 Ops/s $\color{#35bf28}+1.84\%$
test_ddpg_speed 2.9864ms 2.6672ms 374.9187 Ops/s 369.7995 Ops/s $\color{#35bf28}+1.38\%$
test_sac_speed 8.2441ms 7.7594ms 128.8755 Ops/s 128.2790 Ops/s $\color{#35bf28}+0.46\%$
test_redq_speed 0.1086s 11.1841ms 89.4124 Ops/s 98.7446 Ops/s $\textbf{\color{#d91a1a}-9.45\%}$
test_redq_deprec_speed 11.3115ms 10.7626ms 92.9140 Ops/s 93.3343 Ops/s $\color{#d91a1a}-0.45\%$
test_td3_speed 8.0288ms 7.8098ms 128.0449 Ops/s 129.9864 Ops/s $\color{#d91a1a}-1.49\%$
test_cql_speed 25.6433ms 25.0394ms 39.9370 Ops/s 40.4376 Ops/s $\color{#d91a1a}-1.24\%$
test_a2c_speed 6.1443ms 5.6057ms 178.3894 Ops/s 184.5544 Ops/s $\color{#d91a1a}-3.34\%$
test_ppo_speed 6.9374ms 5.8827ms 169.9897 Ops/s 175.2679 Ops/s $\color{#d91a1a}-3.01\%$
test_reinforce_speed 4.9025ms 4.5582ms 219.3845 Ops/s 223.7628 Ops/s $\color{#d91a1a}-1.96\%$
test_iql_speed 20.0517ms 19.3478ms 51.6854 Ops/s 52.3533 Ops/s $\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.6182ms 6.3531ms 157.4042 Ops/s 156.0640 Ops/s $\color{#35bf28}+0.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8919ms 0.2440ms 4.0981 KOps/s 4.2492 KOps/s $\color{#d91a1a}-3.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7317ms 0.2206ms 4.5334 KOps/s 4.7146 KOps/s $\color{#d91a1a}-3.84\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.6406ms 6.2728ms 159.4195 Ops/s 158.0474 Ops/s $\color{#35bf28}+0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6510ms 0.3070ms 3.2570 KOps/s 4.2563 KOps/s $\textbf{\color{#d91a1a}-23.48\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6085ms 0.2964ms 3.3741 KOps/s 4.7630 KOps/s $\textbf{\color{#d91a1a}-29.16\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6029ms 1.3244ms 755.0385 Ops/s 808.2129 Ops/s $\textbf{\color{#d91a1a}-6.58\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5226ms 1.2791ms 781.8028 Ops/s 875.0953 Ops/s $\textbf{\color{#d91a1a}-10.66\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7216ms 6.5033ms 153.7681 Ops/s 152.6442 Ops/s $\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.6868ms 0.4554ms 2.1961 KOps/s 2.6564 KOps/s $\textbf{\color{#d91a1a}-17.33\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.6104ms 0.4399ms 2.2734 KOps/s 2.8218 KOps/s $\textbf{\color{#d91a1a}-19.43\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.6551ms 6.4157ms 155.8676 Ops/s 155.2198 Ops/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6128ms 0.2369ms 4.2220 KOps/s 4.2445 KOps/s $\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.3974ms 0.2176ms 4.5963 KOps/s 4.6910 KOps/s $\color{#d91a1a}-2.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7067ms 6.2656ms 159.6019 Ops/s 157.5481 Ops/s $\color{#35bf28}+1.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6945ms 0.2360ms 4.2366 KOps/s 4.2550 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.3751ms 0.2150ms 4.6503 KOps/s 4.6920 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7119ms 6.4922ms 154.0307 Ops/s 152.9101 Ops/s $\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7778ms 0.4942ms 2.0234 KOps/s 2.6215 KOps/s $\textbf{\color{#d91a1a}-22.81\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6924ms 0.4790ms 2.0876 KOps/s 2.7626 KOps/s $\textbf{\color{#d91a1a}-24.43\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1582s 8.2888ms 120.6453 Ops/s 121.6589 Ops/s $\color{#d91a1a}-0.83\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 21.4330ms 16.3234ms 61.2619 Ops/s 62.5349 Ops/s $\color{#d91a1a}-2.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1258ms 0.9549ms 1.0473 KOps/s 281.4150 Ops/s $\textbf{\color{#35bf28}+272.14\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1382s 7.8508ms 127.3748 Ops/s 129.8496 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 21.4204ms 16.1617ms 61.8748 Ops/s 62.6190 Ops/s $\color{#d91a1a}-1.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1359ms 1.0005ms 999.4554 Ops/s 1.0495 KOps/s $\color{#d91a1a}-4.77\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1331s 7.9503ms 125.7815 Ops/s 127.0364 Ops/s $\color{#d91a1a}-0.99\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1450s 18.9589ms 52.7456 Ops/s 61.7615 Ops/s $\textbf{\color{#d91a1a}-14.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.5556ms 1.2548ms 796.9197 Ops/s 881.3512 Ops/s $\textbf{\color{#d91a1a}-9.58\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot