Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Examples] Move examples to sota-implementations #2016

Merged
merged 3 commits into from
Mar 18, 2024
Merged

[Examples] Move examples to sota-implementations #2016

merged 3 commits into from
Mar 18, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 18, 2024 •
edited
Loading

I chose the name implementations over algorithms because algorithms can make people think that this is somewhat the official way we recommend to use the library.
These codes have a bunch of opinionated choices (e.g. in the way the experiments are configured) from the people that have helped coding them, and they sometimes don't use the entirety of the features they could from the lib.

Therefore, I wouldn't want users to think that the way to use the library is to copy-paste these examples.

cc @albertbou92 @BY571 @matteobettini @skandermoalla

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 18, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2016

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit 9ffeea2 with merge base 29d9a5b (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 18, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 18, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 55.9195ms 55.5346ms 18.0068 Ops/s 17.0450 Ops/s $\textbf{\color{#35bf28}+5.64\%}$
test_sync 36.9993ms 31.7950ms 31.4515 Ops/s 30.3325 Ops/s $\color{#35bf28}+3.69\%$
test_async 53.0661ms 27.2438ms 36.7057 Ops/s 34.4595 Ops/s $\textbf{\color{#35bf28}+6.52\%}$
test_simple 0.4166s 0.3543s 2.8228 Ops/s 2.8646 Ops/s $\color{#d91a1a}-1.46\%$
test_transformed 0.5308s 0.4802s 2.0826 Ops/s 2.1355 Ops/s $\color{#d91a1a}-2.48\%$
test_serial 1.2523s 1.2089s 0.8272 Ops/s 0.8219 Ops/s $\color{#35bf28}+0.65\%$
test_parallel 1.0991s 1.0558s 0.9472 Ops/s 0.9543 Ops/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-True-True-True] 53.1290μs 21.5193μs 46.4700 KOps/s 46.4903 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-True-True-False] 37.0190μs 13.2449μs 75.5005 KOps/s 77.4824 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[True-True-True-False-True] 44.5530μs 12.5662μs 79.5786 KOps/s 79.6848 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-False-False] 33.5430μs 7.6985μs 129.8952 KOps/s 132.4992 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[True-True-False-True-True] 54.4320μs 22.8070μs 43.8462 KOps/s 43.9209 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-True-False-True-False] 38.2520μs 14.5070μs 68.9320 KOps/s 70.5373 KOps/s $\color{#d91a1a}-2.28\%$
test_step_mdp_speed[True-True-False-False-True] 34.4750μs 13.8943μs 71.9720 KOps/s 73.1702 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[True-True-False-False-False] 39.1130μs 9.0529μs 110.4622 KOps/s 114.0958 KOps/s $\color{#d91a1a}-3.18\%$
test_step_mdp_speed[True-False-True-True-True] 0.1527ms 24.7348μs 40.4288 KOps/s 41.3535 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-False-True-True-False] 0.3339ms 16.5008μs 60.6030 KOps/s 63.6363 KOps/s $\color{#d91a1a}-4.77\%$
test_step_mdp_speed[True-False-True-False-True] 45.6550μs 13.7953μs 72.4885 KOps/s 71.8331 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-False-False] 43.3410μs 9.0510μs 110.4855 KOps/s 112.5443 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[True-False-False-True-True] 54.0210μs 25.3266μs 39.4842 KOps/s 35.9447 KOps/s $\textbf{\color{#35bf28}+9.85\%}$
test_step_mdp_speed[True-False-False-True-False] 42.2300μs 16.9444μs 59.0167 KOps/s 60.0104 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[True-False-False-False-True] 53.7810μs 14.8360μs 67.4036 KOps/s 66.5877 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-False-False-False-False] 36.9290μs 10.2177μs 97.8698 KOps/s 100.4916 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-True-True-True-True] 0.2162ms 24.3102μs 41.1349 KOps/s 41.4302 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-True-True-True-False] 43.6420μs 15.8737μs 62.9973 KOps/s 64.6521 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-True-True-False-True] 54.6230μs 16.1171μs 62.0459 KOps/s 61.8539 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-True-True-False-False] 35.0550μs 10.3125μs 96.9693 KOps/s 100.2814 KOps/s $\color{#d91a1a}-3.30\%$
test_step_mdp_speed[False-True-False-True-True] 40.0750μs 25.9086μs 38.5972 KOps/s 39.2623 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[False-True-False-True-False] 51.0260μs 16.9836μs 58.8802 KOps/s 60.2053 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[False-True-False-False-True] 70.2720μs 17.2500μs 57.9711 KOps/s 57.3738 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-True-False-False-False] 56.6670μs 11.4918μs 87.0185 KOps/s 89.7415 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-False-True-True-True] 61.0350μs 26.7652μs 37.3620 KOps/s 37.8383 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-False-True-True-False] 43.9430μs 18.5397μs 53.9382 KOps/s 55.7717 KOps/s $\color{#d91a1a}-3.29\%$
test_step_mdp_speed[False-False-True-False-True] 55.3540μs 17.2529μs 57.9614 KOps/s 58.3909 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-True-False-False] 32.9320μs 11.4367μs 87.4381 KOps/s 90.1659 KOps/s $\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-False-False-True-True] 56.9670μs 27.7088μs 36.0896 KOps/s 36.4404 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-False-False-True-False] 80.1660μs 19.3769μs 51.6078 KOps/s 52.9095 KOps/s $\color{#d91a1a}-2.46\%$
test_step_mdp_speed[False-False-False-False-True] 55.9350μs 18.2567μs 54.7744 KOps/s 55.3863 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-False-False-False-False] 0.1798ms 13.2461μs 75.4942 KOps/s 82.6828 KOps/s $\textbf{\color{#d91a1a}-8.69\%}$
test_values[generalized_advantage_estimate-True-True] 10.6917ms 9.3802ms 106.6079 Ops/s 105.9847 Ops/s $\color{#35bf28}+0.59\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.5150ms 35.7666ms 27.9590 Ops/s 29.7554 Ops/s $\textbf{\color{#d91a1a}-6.04\%}$
test_values[td0_return_estimate-False-False] 0.2174ms 0.1767ms 5.6588 KOps/s 5.3537 KOps/s $\textbf{\color{#35bf28}+5.70\%}$
test_values[td1_return_estimate-False-False] 25.0383ms 23.3720ms 42.7862 Ops/s 42.9779 Ops/s $\color{#d91a1a}-0.45\%$
test_values[vec_td1_return_estimate-False-False] 37.1661ms 36.0361ms 27.7499 Ops/s 29.6441 Ops/s $\textbf{\color{#d91a1a}-6.39\%}$
test_values[td_lambda_return_estimate-True-False] 34.2336ms 33.8210ms 29.5674 Ops/s 29.8889 Ops/s $\color{#d91a1a}-1.08\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.9717ms 35.8649ms 27.8824 Ops/s 29.7563 Ops/s $\textbf{\color{#d91a1a}-6.30\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2071ms 8.0806ms 123.7529 Ops/s 124.1508 Ops/s $\color{#d91a1a}-0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3791ms 1.7909ms 558.3762 Ops/s 523.8667 Ops/s $\textbf{\color{#35bf28}+6.59\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4372ms 0.3538ms 2.8261 KOps/s 2.8904 KOps/s $\color{#d91a1a}-2.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.2365ms 45.3647ms 22.0436 Ops/s 24.7494 Ops/s $\textbf{\color{#d91a1a}-10.93\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6168ms 3.0405ms 328.8910 Ops/s 326.5061 Ops/s $\color{#35bf28}+0.73\%$
test_dqn_speed 7.1471ms 1.3723ms 728.7082 Ops/s 736.5421 Ops/s $\color{#d91a1a}-1.06\%$
test_ddpg_speed 3.1373ms 2.6937ms 371.2306 Ops/s 372.6962 Ops/s $\color{#d91a1a}-0.39\%$
test_sac_speed 9.8827ms 8.5055ms 117.5714 Ops/s 118.9879 Ops/s $\color{#d91a1a}-1.19\%$
test_redq_speed 14.3801ms 13.4286ms 74.4679 Ops/s 74.8018 Ops/s $\color{#d91a1a}-0.45\%$
test_redq_deprec_speed 16.1078ms 13.9382ms 71.7455 Ops/s 72.2565 Ops/s $\color{#d91a1a}-0.71\%$
test_td3_speed 16.6821ms 8.5488ms 116.9758 Ops/s 119.7827 Ops/s $\color{#d91a1a}-2.34\%$
test_cql_speed 40.7966ms 37.5262ms 26.6480 Ops/s 27.2648 Ops/s $\color{#d91a1a}-2.26\%$
test_a2c_speed 8.3583ms 7.5856ms 131.8279 Ops/s 131.4129 Ops/s $\color{#35bf28}+0.32\%$
test_ppo_speed 9.5450ms 7.9531ms 125.7374 Ops/s 123.0788 Ops/s $\color{#35bf28}+2.16\%$
test_reinforce_speed 7.7859ms 6.6853ms 149.5830 Ops/s 148.5973 Ops/s $\color{#35bf28}+0.66\%$
test_iql_speed 34.7283ms 33.2457ms 30.0790 Ops/s 30.2577 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6450ms 2.4159ms 413.9233 Ops/s 422.7347 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1983ms 0.5168ms 1.9351 KOps/s 1.9593 KOps/s $\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7061ms 0.4893ms 2.0437 KOps/s 2.0676 KOps/s $\color{#d91a1a}-1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.6549ms 2.4232ms 412.6848 Ops/s 403.6833 Ops/s $\color{#35bf28}+2.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6764ms 0.5044ms 1.9826 KOps/s 1.9924 KOps/s $\color{#d91a1a}-0.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7605ms 0.4854ms 2.0600 KOps/s 2.0873 KOps/s $\color{#d91a1a}-1.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9177ms 1.2917ms 774.1482 Ops/s 784.9162 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.3052ms 1.2378ms 807.8959 Ops/s 829.1202 Ops/s $\color{#d91a1a}-2.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6843ms 2.5479ms 392.4749 Ops/s 394.7346 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1038s 0.7143ms 1.4000 KOps/s 1.6127 KOps/s $\textbf{\color{#d91a1a}-13.19\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8159ms 0.5998ms 1.6672 KOps/s 1.6910 KOps/s $\color{#d91a1a}-1.41\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.6207ms 2.4524ms 407.7564 Ops/s 417.8342 Ops/s $\color{#d91a1a}-2.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1335ms 0.5134ms 1.9480 KOps/s 1.9526 KOps/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7208ms 0.4908ms 2.0377 KOps/s 2.0755 KOps/s $\color{#d91a1a}-1.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6681ms 2.4579ms 406.8479 Ops/s 409.4032 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7997ms 0.5086ms 1.9661 KOps/s 1.8573 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.7848ms 0.4927ms 2.0298 KOps/s 2.1124 KOps/s $\color{#d91a1a}-3.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6835ms 2.5289ms 395.4353 Ops/s 399.2756 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8319ms 0.6293ms 1.5890 KOps/s 1.6180 KOps/s $\color{#d91a1a}-1.79\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7595ms 0.6054ms 1.6517 KOps/s 1.6885 KOps/s $\color{#d91a1a}-2.18\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1142s 7.8103ms 128.0358 Ops/s 127.9662 Ops/s $\color{#35bf28}+0.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.4494ms 12.0217ms 83.1828 Ops/s 80.8320 Ops/s $\color{#35bf28}+2.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0830ms 1.0319ms 969.0953 Ops/s 971.0461 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 89.6332ms 5.4719ms 182.7511 Ops/s 133.4040 Ops/s $\textbf{\color{#35bf28}+36.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1049s 13.9155ms 71.8622 Ops/s 80.3269 Ops/s $\textbf{\color{#d91a1a}-10.54\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.6232ms 1.0756ms 929.7317 Ops/s 970.3675 Ops/s $\color{#d91a1a}-4.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 98.6543ms 5.9883ms 166.9922 Ops/s 173.5516 Ops/s $\color{#d91a1a}-3.78\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.3869ms 12.4988ms 80.0074 Ops/s 73.8204 Ops/s $\textbf{\color{#35bf28}+8.38\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.0363ms 1.4969ms 668.0437 Ops/s 723.0660 Ops/s $\textbf{\color{#d91a1a}-7.61\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 18, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1021s 0.1010s 9.8987 Ops/s 9.2015 Ops/s $\textbf{\color{#35bf28}+7.58\%}$
test_sync 91.6832ms 88.6655ms 11.2783 Ops/s 11.2621 Ops/s $\color{#35bf28}+0.14\%$
test_async 0.1755s 88.5473ms 11.2934 Ops/s 12.5296 Ops/s $\textbf{\color{#d91a1a}-9.87\%}$
test_single_pixels 0.1817s 0.1201s 8.3289 Ops/s 8.8909 Ops/s $\textbf{\color{#d91a1a}-6.32\%}$
test_sync_pixels 69.5616ms 68.6141ms 14.5743 Ops/s 14.6756 Ops/s $\color{#d91a1a}-0.69\%$
test_async_pixels 0.1218s 55.4289ms 18.0411 Ops/s 17.8425 Ops/s $\color{#35bf28}+1.11\%$
test_simple 0.7308s 0.6725s 1.4871 Ops/s 1.4736 Ops/s $\color{#35bf28}+0.91\%$
test_transformed 0.9289s 0.8747s 1.1432 Ops/s 1.1276 Ops/s $\color{#35bf28}+1.38\%$
test_serial 2.1292s 2.0724s 0.4825 Ops/s 0.4767 Ops/s $\color{#35bf28}+1.23\%$
test_parallel 1.8986s 1.8396s 0.5436 Ops/s 0.5514 Ops/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-True-True-True-True] 0.1006ms 32.1438μs 31.1102 KOps/s 30.4386 KOps/s $\color{#35bf28}+2.21\%$
test_step_mdp_speed[True-True-True-True-False] 40.1400μs 19.2247μs 52.0164 KOps/s 51.7677 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-True-True-False-True] 54.2010μs 18.2359μs 54.8370 KOps/s 53.4472 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-True-False-False] 28.4600μs 10.9591μs 91.2487 KOps/s 90.1721 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-True-False-True-True] 50.0510μs 34.1713μs 29.2643 KOps/s 28.6650 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-True-False-True-False] 39.8110μs 21.4447μs 46.6316 KOps/s 46.6557 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-True-False-False-True] 46.8910μs 20.1357μs 49.6630 KOps/s 48.5282 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-True-False-False-False] 30.0500μs 12.8650μs 77.7301 KOps/s 77.7320 KOps/s $-0.00\%$
test_step_mdp_speed[True-False-True-True-True] 0.1257ms 35.4873μs 28.1791 KOps/s 27.5514 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-False-True-True-False] 41.9400μs 22.8553μs 43.7534 KOps/s 43.1082 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-False-True-False-True] 38.1210μs 20.0663μs 49.8348 KOps/s 48.1898 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[True-False-True-False-False] 29.5000μs 12.8332μs 77.9228 KOps/s 76.9270 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-False-False-True-True] 66.1110μs 37.3565μs 26.7691 KOps/s 26.0157 KOps/s $\color{#35bf28}+2.90\%$
test_step_mdp_speed[True-False-False-True-False] 40.2700μs 24.9742μs 40.0413 KOps/s 40.0609 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-False-False-True] 61.0810μs 21.7298μs 46.0197 KOps/s 44.7264 KOps/s $\color{#35bf28}+2.89\%$
test_step_mdp_speed[True-False-False-False-False] 32.3110μs 14.7290μs 67.8933 KOps/s 68.2159 KOps/s $\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-True-True-True-True] 88.2210μs 35.9873μs 27.7876 KOps/s 27.0998 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[False-True-True-True-False] 50.7610μs 23.2210μs 43.0644 KOps/s 42.6436 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-True-True-False-True] 50.1210μs 24.2500μs 41.2370 KOps/s 40.9267 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-True-False-False] 29.7400μs 14.8913μs 67.1532 KOps/s 67.9235 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-True-False-True-True] 72.9910μs 37.7143μs 26.5151 KOps/s 25.6909 KOps/s $\color{#35bf28}+3.21\%$
test_step_mdp_speed[False-True-False-True-False] 48.2110μs 24.9536μs 40.0745 KOps/s 39.6779 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-False-True] 43.4600μs 25.6828μs 38.9366 KOps/s 37.8991 KOps/s $\color{#35bf28}+2.74\%$
test_step_mdp_speed[False-True-False-False-False] 33.8500μs 16.5663μs 60.3637 KOps/s 60.6277 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-False-True-True-True] 72.1210μs 39.2559μs 25.4739 KOps/s 24.8622 KOps/s $\color{#35bf28}+2.46\%$
test_step_mdp_speed[False-False-True-True-False] 62.0110μs 26.9757μs 37.0704 KOps/s 36.6530 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-False-True-False-True] 0.1092ms 25.4975μs 39.2195 KOps/s 38.2303 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[False-False-True-False-False] 34.0610μs 16.0476μs 62.3146 KOps/s 60.3506 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[False-False-False-True-True] 70.8810μs 40.5858μs 24.6392 KOps/s 24.0452 KOps/s $\color{#35bf28}+2.47\%$
test_step_mdp_speed[False-False-False-True-False] 54.9210μs 28.5845μs 34.9840 KOps/s 34.6906 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-False-False-False-True] 47.7610μs 27.5097μs 36.3508 KOps/s 35.7302 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-False-False-False-False] 50.1110μs 18.1614μs 55.0619 KOps/s 54.8860 KOps/s $\color{#35bf28}+0.32\%$
test_values[generalized_advantage_estimate-True-True] 25.5054ms 25.0716ms 39.8858 Ops/s 39.5626 Ops/s $\color{#35bf28}+0.82\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.9500ms 3.2133ms 311.2077 Ops/s 290.7841 Ops/s $\textbf{\color{#35bf28}+7.02\%}$
test_values[td0_return_estimate-False-False] 92.2520μs 65.3374μs 15.3052 KOps/s 15.6837 KOps/s $\color{#d91a1a}-2.41\%$
test_values[td1_return_estimate-False-False] 54.5513ms 53.3993ms 18.7269 Ops/s 18.8266 Ops/s $\color{#d91a1a}-0.53\%$
test_values[vec_td1_return_estimate-False-False] 2.0915ms 1.7657ms 566.3399 Ops/s 565.6777 Ops/s $\color{#35bf28}+0.12\%$
test_values[td_lambda_return_estimate-True-False] 86.3468ms 84.9924ms 11.7658 Ops/s 11.6897 Ops/s $\color{#35bf28}+0.65\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0582ms 1.7626ms 567.3533 Ops/s 566.4509 Ops/s $\color{#35bf28}+0.16\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.3859ms 23.2231ms 43.0606 Ops/s 42.6086 Ops/s $\color{#35bf28}+1.06\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8780ms 0.7020ms 1.4245 KOps/s 1.4250 KOps/s $\color{#d91a1a}-0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7287ms 0.6524ms 1.5329 KOps/s 1.5355 KOps/s $\color{#d91a1a}-0.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4787ms 1.4521ms 688.6374 Ops/s 687.0951 Ops/s $\color{#35bf28}+0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9399ms 0.6723ms 1.4874 KOps/s 1.4853 KOps/s $\color{#35bf28}+0.15\%$
test_dqn_speed 2.0378ms 1.3854ms 721.7934 Ops/s 690.8706 Ops/s $\color{#35bf28}+4.48\%$
test_ddpg_speed 3.2033ms 2.6669ms 374.9696 Ops/s 366.9029 Ops/s $\color{#35bf28}+2.20\%$
test_sac_speed 8.4061ms 7.8929ms 126.6957 Ops/s 123.1594 Ops/s $\color{#35bf28}+2.87\%$
test_redq_speed 11.0987ms 10.1566ms 98.4578 Ops/s 98.5716 Ops/s $\color{#d91a1a}-0.12\%$
test_redq_deprec_speed 11.5022ms 10.9670ms 91.1827 Ops/s 90.5001 Ops/s $\color{#35bf28}+0.75\%$
test_td3_speed 8.1899ms 7.8519ms 127.3573 Ops/s 124.3758 Ops/s $\color{#35bf28}+2.40\%$
test_cql_speed 0.1035s 26.8718ms 37.2137 Ops/s 39.6008 Ops/s $\textbf{\color{#d91a1a}-6.03\%}$
test_a2c_speed 5.5605ms 5.3340ms 187.4766 Ops/s 182.8003 Ops/s $\color{#35bf28}+2.56\%$
test_ppo_speed 5.9763ms 5.7630ms 173.5204 Ops/s 171.7704 Ops/s $\color{#35bf28}+1.02\%$
test_reinforce_speed 5.2608ms 4.3792ms 228.3497 Ops/s 225.4061 Ops/s $\color{#35bf28}+1.31\%$
test_iql_speed 20.2314ms 19.0823ms 52.4045 Ops/s 51.7169 Ops/s $\color{#35bf28}+1.33\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9936ms 2.8216ms 354.4122 Ops/s 353.3018 Ops/s $\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4802ms 0.5338ms 1.8734 KOps/s 1.8517 KOps/s $\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7206ms 0.5146ms 1.9433 KOps/s 1.9321 KOps/s $\color{#35bf28}+0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9469ms 2.8241ms 354.0988 Ops/s 350.6929 Ops/s $\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4524ms 0.5238ms 1.9092 KOps/s 1.8633 KOps/s $\color{#35bf28}+2.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7595ms 0.5004ms 1.9985 KOps/s 1.9395 KOps/s $\color{#35bf28}+3.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5919ms 1.4946ms 669.0653 Ops/s 641.3216 Ops/s $\color{#35bf28}+4.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5927ms 1.4247ms 701.8902 Ops/s 679.2986 Ops/s $\color{#35bf28}+3.33\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0092ms 2.9392ms 340.2252 Ops/s 337.6863 Ops/s $\color{#35bf28}+0.75\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2529ms 0.6509ms 1.5363 KOps/s 1.3352 KOps/s $\textbf{\color{#35bf28}+15.06\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8752ms 0.6272ms 1.5944 KOps/s 1.5664 KOps/s $\color{#35bf28}+1.79\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9034ms 2.8154ms 355.1836 Ops/s 351.3118 Ops/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6725ms 0.5333ms 1.8751 KOps/s 1.8450 KOps/s $\color{#35bf28}+1.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4014ms 0.5164ms 1.9366 KOps/s 1.9312 KOps/s $\color{#35bf28}+0.28\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0281ms 2.8470ms 351.2530 Ops/s 349.6180 Ops/s $\color{#35bf28}+0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6032ms 0.5277ms 1.8950 KOps/s 1.8641 KOps/s $\color{#35bf28}+1.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7440ms 0.5081ms 1.9680 KOps/s 1.9430 KOps/s $\color{#35bf28}+1.29\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0536ms 2.9499ms 338.9922 Ops/s 332.7934 Ops/s $\color{#35bf28}+1.86\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4460ms 0.6579ms 1.5200 KOps/s 1.5054 KOps/s $\color{#35bf28}+0.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8619ms 0.6317ms 1.5830 KOps/s 1.5367 KOps/s $\color{#35bf28}+3.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1055s 8.5838ms 116.4986 Ops/s 111.9548 Ops/s $\color{#35bf28}+4.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.4645ms 14.1672ms 70.5858 Ops/s 69.0895 Ops/s $\color{#35bf28}+2.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.9793ms 1.2578ms 795.0279 Ops/s 961.6707 Ops/s $\textbf{\color{#d91a1a}-17.33\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 98.7593ms 6.5898ms 151.7507 Ops/s 150.7730 Ops/s $\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.2594ms 14.1159ms 70.8422 Ops/s 67.9181 Ops/s $\color{#35bf28}+4.31\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.9210ms 1.2558ms 796.3145 Ops/s 918.6811 Ops/s $\textbf{\color{#d91a1a}-13.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1013s 8.9061ms 112.2821 Ops/s 110.9134 Ops/s $\color{#35bf28}+1.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.8878ms 14.5009ms 68.9615 Ops/s 67.4762 Ops/s $\color{#35bf28}+2.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.6282ms 1.4504ms 689.4564 Ops/s 721.4221 Ops/s $\color{#d91a1a}-4.43\%$

@matteobettini
Copy link
Contributor

Big fan!

sota might not mean much to first comers

I prefer algorithm-implementations to sota-implementations, but this is just my opinion

@vmoens vmoens merged commit c29c345 into main Mar 18, 2024
64 of 67 checks passed
@vmoens vmoens deleted the move-examples branch March 18, 2024 18:11
vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[Examples] Move examples to sota-implementations (#2016)
3f0cab1 
(cherry picked from commit c29c345)
vmoens added a commit that referenced this pull request Apr 7, 2024
@vmoens
[Examples] Move examples to sota-implementations (#2016)
76f91e3 
(cherry picked from commit c29c345)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Examples
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot