Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Follow-up on tensordict PR 473 #1361

Merged
merged 1 commit into from
Jul 6, 2023
Merged

[Refactor] Follow-up on tensordict PR 473 #1361

merged 1 commit into from
Jul 6, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 6, 2023 •
edited
Loading

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2023
@vmoens vmoens changed the title [Refactor] Follow-up on tensordict \#473 [Refactor] Follow-up on tensordict PR 473 Jul 6, 2023
@vmoens vmoens added the Refactoring Refactoring of an existing feature label Jul 6, 2023
@github-actions
Copy link

github-actions bot commented Jul 6, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}53$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1688s 0.1673s 5.9782 Ops/s 5.9527 Ops/s $\color{#35bf28}+0.43\%$
test_sync 86.8182ms 86.2742ms 11.5909 Ops/s 10.9854 Ops/s $\textbf{\color{#35bf28}+5.51\%}$
test_async 0.1700s 84.4867ms 11.8362 Ops/s 11.5560 Ops/s $\color{#35bf28}+2.42\%$
test_simple 0.7419s 0.6551s 1.5264 Ops/s 1.4923 Ops/s $\color{#35bf28}+2.29\%$
test_transformed 1.7649s 1.7074s 0.5857 Ops/s 0.5909 Ops/s $\color{#d91a1a}-0.88\%$
test_serial 2.0664s 1.9844s 0.5039 Ops/s 0.4991 Ops/s $\color{#35bf28}+0.97\%$
test_parallel 1.7631s 1.6022s 0.6241 Ops/s 0.6279 Ops/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-True-True-True-True] 0.1958ms 42.8206μs 23.3532 KOps/s 18.9058 KOps/s $\textbf{\color{#35bf28}+23.52\%}$
test_step_mdp_speed[True-True-True-True-False] 53.7010μs 23.6461μs 42.2904 KOps/s 35.6192 KOps/s $\textbf{\color{#35bf28}+18.73\%}$
test_step_mdp_speed[True-True-True-False-True] 93.6010μs 31.5269μs 31.7189 KOps/s 26.3253 KOps/s $\textbf{\color{#35bf28}+20.49\%}$
test_step_mdp_speed[True-True-True-False-False] 43.4010μs 17.3105μs 57.7684 KOps/s 49.3474 KOps/s $\textbf{\color{#35bf28}+17.06\%}$
test_step_mdp_speed[True-True-False-True-True] 75.2010μs 44.1000μs 22.6757 KOps/s 18.3723 KOps/s $\textbf{\color{#35bf28}+23.42\%}$
test_step_mdp_speed[True-True-False-True-False] 0.1027ms 25.1743μs 39.7230 KOps/s 34.1065 KOps/s $\textbf{\color{#35bf28}+16.47\%}$
test_step_mdp_speed[True-True-False-False-True] 70.8010μs 32.7584μs 30.5265 KOps/s 25.2138 KOps/s $\textbf{\color{#35bf28}+21.07\%}$
test_step_mdp_speed[True-True-False-False-False] 50.9000μs 18.8556μs 53.0348 KOps/s 47.5948 KOps/s $\textbf{\color{#35bf28}+11.43\%}$
test_step_mdp_speed[True-False-True-True-True] 75.0010μs 45.7729μs 21.8470 KOps/s 17.9717 KOps/s $\textbf{\color{#35bf28}+21.56\%}$
test_step_mdp_speed[True-False-True-True-False] 56.8010μs 26.6215μs 37.5636 KOps/s 32.1319 KOps/s $\textbf{\color{#35bf28}+16.90\%}$
test_step_mdp_speed[True-False-True-False-True] 58.1010μs 32.7092μs 30.5725 KOps/s 25.1323 KOps/s $\textbf{\color{#35bf28}+21.65\%}$
test_step_mdp_speed[True-False-True-False-False] 48.7010μs 18.6325μs 53.6696 KOps/s 47.9884 KOps/s $\textbf{\color{#35bf28}+11.84\%}$
test_step_mdp_speed[True-False-False-True-True] 67.5010μs 47.0085μs 21.2727 KOps/s 17.4593 KOps/s $\textbf{\color{#35bf28}+21.84\%}$
test_step_mdp_speed[True-False-False-True-False] 59.4000μs 28.0358μs 35.6687 KOps/s 31.0182 KOps/s $\textbf{\color{#35bf28}+14.99\%}$
test_step_mdp_speed[True-False-False-False-True] 60.9010μs 34.1425μs 29.2890 KOps/s 24.0986 KOps/s $\textbf{\color{#35bf28}+21.54\%}$
test_step_mdp_speed[True-False-False-False-False] 98.1010μs 19.9465μs 50.1342 KOps/s 44.5838 KOps/s $\textbf{\color{#35bf28}+12.45\%}$
test_step_mdp_speed[False-True-True-True-True] 99.0010μs 45.9939μs 21.7420 KOps/s 17.8904 KOps/s $\textbf{\color{#35bf28}+21.53\%}$
test_step_mdp_speed[False-True-True-True-False] 76.1010μs 26.6373μs 37.5413 KOps/s 32.1806 KOps/s $\textbf{\color{#35bf28}+16.66\%}$
test_step_mdp_speed[False-True-True-False-True] 65.5010μs 37.3221μs 26.7938 KOps/s 22.2714 KOps/s $\textbf{\color{#35bf28}+20.31\%}$
test_step_mdp_speed[False-True-True-False-False] 46.6000μs 20.5973μs 48.5500 KOps/s 43.3987 KOps/s $\textbf{\color{#35bf28}+11.87\%}$
test_step_mdp_speed[False-True-False-True-True] 97.6010μs 47.5310μs 21.0389 KOps/s 17.4193 KOps/s $\textbf{\color{#35bf28}+20.78\%}$
test_step_mdp_speed[False-True-False-True-False] 69.7000μs 28.1469μs 35.5279 KOps/s 30.7401 KOps/s $\textbf{\color{#35bf28}+15.58\%}$
test_step_mdp_speed[False-True-False-False-True] 68.3010μs 38.9129μs 25.6984 KOps/s 21.5025 KOps/s $\textbf{\color{#35bf28}+19.51\%}$
test_step_mdp_speed[False-True-False-False-False] 0.1197ms 21.9948μs 45.4653 KOps/s 40.6452 KOps/s $\textbf{\color{#35bf28}+11.86\%}$
test_step_mdp_speed[False-False-True-True-True] 0.2134ms 48.3882μs 20.6662 KOps/s 17.0748 KOps/s $\textbf{\color{#35bf28}+21.03\%}$
test_step_mdp_speed[False-False-True-True-False] 0.1059ms 29.7573μs 33.6052 KOps/s 29.8776 KOps/s $\textbf{\color{#35bf28}+12.48\%}$
test_step_mdp_speed[False-False-True-False-True] 78.5010μs 39.3747μs 25.3970 KOps/s 21.6359 KOps/s $\textbf{\color{#35bf28}+17.38\%}$
test_step_mdp_speed[False-False-True-False-False] 55.1010μs 21.7228μs 46.0345 KOps/s 40.8026 KOps/s $\textbf{\color{#35bf28}+12.82\%}$
test_step_mdp_speed[False-False-False-True-True] 82.3010μs 49.6338μs 20.1475 KOps/s 16.7141 KOps/s $\textbf{\color{#35bf28}+20.54\%}$
test_step_mdp_speed[False-False-False-True-False] 0.1048ms 31.1134μs 32.1405 KOps/s 28.1625 KOps/s $\textbf{\color{#35bf28}+14.13\%}$
test_step_mdp_speed[False-False-False-False-True] 71.6010μs 39.8199μs 25.1131 KOps/s 21.2391 KOps/s $\textbf{\color{#35bf28}+18.24\%}$
test_step_mdp_speed[False-False-False-False-False] 47.9010μs 23.2266μs 43.0541 KOps/s 38.4561 KOps/s $\textbf{\color{#35bf28}+11.96\%}$
test_values[generalized_advantage_estimate-True-True] 14.1871ms 13.2683ms 75.3676 Ops/s 72.1497 Ops/s $\color{#35bf28}+4.46\%$
test_values[vec_generalized_advantage_estimate-True-True] 54.6567ms 43.4527ms 23.0135 Ops/s 22.2055 Ops/s $\color{#35bf28}+3.64\%$
test_values[td0_return_estimate-False-False] 0.3957ms 0.2234ms 4.4762 KOps/s 3.0601 KOps/s $\textbf{\color{#35bf28}+46.28\%}$
test_values[td1_return_estimate-False-False] 13.3614ms 13.1625ms 75.9736 Ops/s 73.1112 Ops/s $\color{#35bf28}+3.92\%$
test_values[vec_td1_return_estimate-False-False] 49.7108ms 42.7380ms 23.3984 Ops/s 22.6577 Ops/s $\color{#35bf28}+3.27\%$
test_values[td_lambda_return_estimate-True-False] 32.1686ms 31.6762ms 31.5695 Ops/s 30.1007 Ops/s $\color{#35bf28}+4.88\%$
test_values[vec_td_lambda_return_estimate-True-False] 47.7956ms 43.3008ms 23.0943 Ops/s 22.6103 Ops/s $\color{#35bf28}+2.14\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.5060ms 11.4232ms 87.5411 Ops/s 85.1974 Ops/s $\color{#35bf28}+2.75\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 4.0048ms 3.4132ms 292.9828 Ops/s 289.4881 Ops/s $\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1.4128ms 0.4493ms 2.2258 KOps/s 2.1456 KOps/s $\color{#35bf28}+3.74\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 55.7993ms 54.7602ms 18.2614 Ops/s 16.0709 Ops/s $\textbf{\color{#35bf28}+13.63\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.4535ms 2.8473ms 351.2091 Ops/s 336.2529 Ops/s $\color{#35bf28}+4.45\%$
test_dqn_speed 8.4085ms 1.7013ms 587.7795 Ops/s 557.9574 Ops/s $\textbf{\color{#35bf28}+5.34\%}$
test_ddpg_speed 4.1812ms 2.8278ms 353.6372 Ops/s 359.7266 Ops/s $\color{#d91a1a}-1.69\%$
test_sac_speed 0.1107s 8.9929ms 111.1983 Ops/s 121.9686 Ops/s $\textbf{\color{#d91a1a}-8.83\%}$
test_redq_speed 15.7117ms 14.2987ms 69.9366 Ops/s 63.2634 Ops/s $\textbf{\color{#35bf28}+10.55\%}$
test_redq_deprec_speed 11.9257ms 10.9532ms 91.2976 Ops/s 82.1973 Ops/s $\textbf{\color{#35bf28}+11.07\%}$
test_td3_speed 12.1540ms 10.8633ms 92.0530 Ops/s 74.1504 Ops/s $\textbf{\color{#35bf28}+24.14\%}$
test_cql_speed 28.8154ms 27.4687ms 36.4050 Ops/s 34.0119 Ops/s $\textbf{\color{#35bf28}+7.04\%}$
test_a2c_speed 8.8767ms 5.9389ms 168.3800 Ops/s 172.1747 Ops/s $\color{#d91a1a}-2.20\%$
test_ppo_speed 6.8909ms 5.5605ms 179.8395 Ops/s 164.6451 Ops/s $\textbf{\color{#35bf28}+9.23\%}$
test_reinforce_speed 5.2364ms 4.0810ms 245.0396 Ops/s 218.3495 Ops/s $\textbf{\color{#35bf28}+12.22\%}$
test_iql_speed 25.7199ms 21.5173ms 46.4743 Ops/s 40.9030 Ops/s $\textbf{\color{#35bf28}+13.62\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2315ms 3.9334ms 254.2356 Ops/s 210.9872 Ops/s $\textbf{\color{#35bf28}+20.50\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 7.4141ms 4.1465ms 241.1657 Ops/s 223.8086 Ops/s $\textbf{\color{#35bf28}+7.76\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.9306ms 4.0799ms 245.1058 Ops/s 190.3161 Ops/s $\textbf{\color{#35bf28}+28.79\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.9954ms 3.9315ms 254.3576 Ops/s 239.0922 Ops/s $\textbf{\color{#35bf28}+6.38\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.9121ms 4.1655ms 240.0668 Ops/s 230.5739 Ops/s $\color{#35bf28}+4.12\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.5381ms 4.1602ms 240.3714 Ops/s 228.8023 Ops/s $\textbf{\color{#35bf28}+5.06\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1508s 4.5768ms 218.4922 Ops/s 210.8087 Ops/s $\color{#35bf28}+3.64\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 6.2394ms 4.1089ms 243.3767 Ops/s 230.3119 Ops/s $\textbf{\color{#35bf28}+5.67\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1934s 4.9283ms 202.9091 Ops/s 229.8330 Ops/s $\textbf{\color{#d91a1a}-11.71\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.5325ms 4.0217ms 248.6525 Ops/s 180.0791 Ops/s $\textbf{\color{#35bf28}+38.08\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.1606ms 4.1993ms 238.1362 Ops/s 234.4723 Ops/s $\color{#35bf28}+1.56\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.6226ms 4.1996ms 238.1183 Ops/s 192.1909 Ops/s $\textbf{\color{#35bf28}+23.90\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.1487s 4.6572ms 214.7200 Ops/s 240.7016 Ops/s $\textbf{\color{#d91a1a}-10.79\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 8.1057ms 4.1641ms 240.1482 Ops/s 230.0531 Ops/s $\color{#35bf28}+4.39\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.2027s 5.0378ms 198.4985 Ops/s 225.7366 Ops/s $\textbf{\color{#d91a1a}-12.07\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5068ms 3.9968ms 250.2031 Ops/s 208.4783 Ops/s $\textbf{\color{#35bf28}+20.01\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 7.0320ms 4.1713ms 239.7338 Ops/s 233.2512 Ops/s $\color{#35bf28}+2.78\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.3523ms 4.2365ms 236.0424 Ops/s 189.2051 Ops/s $\textbf{\color{#35bf28}+24.75\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3337s 38.6280ms 25.8880 Ops/s 24.7543 Ops/s $\color{#35bf28}+4.58\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1942s 39.4342ms 25.3587 Ops/s 27.6109 Ops/s $\textbf{\color{#d91a1a}-8.16\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1929s 36.1543ms 27.6592 Ops/s 28.0306 Ops/s $\color{#d91a1a}-1.32\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1985s 36.3875ms 27.4820 Ops/s 27.8853 Ops/s $\color{#d91a1a}-1.45\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1928s 36.0926ms 27.7065 Ops/s 27.8430 Ops/s $\color{#d91a1a}-0.49\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1870s 38.9170ms 25.6957 Ops/s 25.3671 Ops/s $\color{#35bf28}+1.30\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1967s 36.0600ms 27.7315 Ops/s 27.2427 Ops/s $\color{#35bf28}+1.79\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1919s 35.8311ms 27.9087 Ops/s 27.9300 Ops/s $\color{#d91a1a}-0.08\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1893s 35.9644ms 27.8053 Ops/s 27.9721 Ops/s $\color{#d91a1a}-0.60\%$

@github-actions
Copy link

github-actions bot commented Jul 6, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}35$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1347s 0.1344s 7.4427 Ops/s 7.2732 Ops/s $\color{#35bf28}+2.33\%$
test_sync 75.5263ms 72.0608ms 13.8772 Ops/s 13.8153 Ops/s $\color{#35bf28}+0.45\%$
test_async 93.6082ms 70.5511ms 14.1741 Ops/s 14.0443 Ops/s $\color{#35bf28}+0.92\%$
test_simple 0.6117s 0.5465s 1.8300 Ops/s 1.7752 Ops/s $\color{#35bf28}+3.09\%$
test_transformed 1.3653s 1.3246s 0.7549 Ops/s 0.7489 Ops/s $\color{#35bf28}+0.81\%$
test_serial 1.7441s 1.6963s 0.5895 Ops/s 0.5825 Ops/s $\color{#35bf28}+1.21\%$
test_parallel 1.6713s 1.5467s 0.6465 Ops/s 0.6675 Ops/s $\color{#d91a1a}-3.14\%$
test_step_mdp_speed[True-True-True-True-True] 0.1564ms 34.5036μs 28.9825 KOps/s 23.2472 KOps/s $\textbf{\color{#35bf28}+24.67\%}$
test_step_mdp_speed[True-True-True-True-False] 48.2000μs 19.8047μs 50.4931 KOps/s 43.4522 KOps/s $\textbf{\color{#35bf28}+16.20\%}$
test_step_mdp_speed[True-True-True-False-True] 75.7000μs 24.9634μs 40.0586 KOps/s 33.1967 KOps/s $\textbf{\color{#35bf28}+20.67\%}$
test_step_mdp_speed[True-True-True-False-False] 53.8000μs 14.2656μs 70.0986 KOps/s 63.8524 KOps/s $\textbf{\color{#35bf28}+9.78\%}$
test_step_mdp_speed[True-True-False-True-True] 0.1336ms 36.9188μs 27.0865 KOps/s 22.5507 KOps/s $\textbf{\color{#35bf28}+20.11\%}$
test_step_mdp_speed[True-True-False-True-False] 39.5000μs 21.2202μs 47.1248 KOps/s 40.8419 KOps/s $\textbf{\color{#35bf28}+15.38\%}$
test_step_mdp_speed[True-True-False-False-True] 53.3000μs 26.5513μs 37.6629 KOps/s 31.4311 KOps/s $\textbf{\color{#35bf28}+19.83\%}$
test_step_mdp_speed[True-True-False-False-False] 42.5000μs 15.5262μs 64.4073 KOps/s 57.5258 KOps/s $\textbf{\color{#35bf28}+11.96\%}$
test_step_mdp_speed[True-False-True-True-True] 63.5000μs 37.4045μs 26.7348 KOps/s 21.8504 KOps/s $\textbf{\color{#35bf28}+22.35\%}$
test_step_mdp_speed[True-False-True-True-False] 47.9000μs 22.3323μs 44.7782 KOps/s 38.6069 KOps/s $\textbf{\color{#35bf28}+15.98\%}$
test_step_mdp_speed[True-False-True-False-True] 0.1141ms 26.6938μs 37.4619 KOps/s 31.6335 KOps/s $\textbf{\color{#35bf28}+18.42\%}$
test_step_mdp_speed[True-False-True-False-False] 60.1000μs 15.2485μs 65.5801 KOps/s 58.2542 KOps/s $\textbf{\color{#35bf28}+12.58\%}$
test_step_mdp_speed[True-False-False-True-True] 64.7000μs 38.7004μs 25.8395 KOps/s 21.0837 KOps/s $\textbf{\color{#35bf28}+22.56\%}$
test_step_mdp_speed[True-False-False-True-False] 64.7000μs 23.5421μs 42.4771 KOps/s 36.8471 KOps/s $\textbf{\color{#35bf28}+15.28\%}$
test_step_mdp_speed[True-False-False-False-True] 74.0000μs 27.9910μs 35.7257 KOps/s 30.4059 KOps/s $\textbf{\color{#35bf28}+17.50\%}$
test_step_mdp_speed[True-False-False-False-False] 41.4000μs 16.4919μs 60.6359 KOps/s 54.1738 KOps/s $\textbf{\color{#35bf28}+11.93\%}$
test_step_mdp_speed[False-True-True-True-True] 63.2000μs 37.8023μs 26.4534 KOps/s 21.8339 KOps/s $\textbf{\color{#35bf28}+21.16\%}$
test_step_mdp_speed[False-True-True-True-False] 54.9000μs 22.6047μs 44.2386 KOps/s 38.8903 KOps/s $\textbf{\color{#35bf28}+13.75\%}$
test_step_mdp_speed[False-True-True-False-True] 72.8000μs 30.5897μs 32.6908 KOps/s 28.1580 KOps/s $\textbf{\color{#35bf28}+16.10\%}$
test_step_mdp_speed[False-True-True-False-False] 97.0000μs 17.1169μs 58.4217 KOps/s 53.4860 KOps/s $\textbf{\color{#35bf28}+9.23\%}$
test_step_mdp_speed[False-True-False-True-True] 70.3000μs 38.9241μs 25.6910 KOps/s 21.0874 KOps/s $\textbf{\color{#35bf28}+21.83\%}$
test_step_mdp_speed[False-True-False-True-False] 52.3000μs 23.8416μs 41.9435 KOps/s 36.9332 KOps/s $\textbf{\color{#35bf28}+13.57\%}$
test_step_mdp_speed[False-True-False-False-True] 84.3000μs 31.4181μs 31.8288 KOps/s 27.1254 KOps/s $\textbf{\color{#35bf28}+17.34\%}$
test_step_mdp_speed[False-True-False-False-False] 10.5107ms 18.7863μs 53.2302 KOps/s 49.6489 KOps/s $\textbf{\color{#35bf28}+7.21\%}$
test_step_mdp_speed[False-False-True-True-True] 64.2000μs 40.1261μs 24.9214 KOps/s 20.7463 KOps/s $\textbf{\color{#35bf28}+20.12\%}$
test_step_mdp_speed[False-False-True-True-False] 0.1074ms 24.8208μs 40.2889 KOps/s 35.2293 KOps/s $\textbf{\color{#35bf28}+14.36\%}$
test_step_mdp_speed[False-False-True-False-True] 62.3000μs 31.8386μs 31.4085 KOps/s 26.8081 KOps/s $\textbf{\color{#35bf28}+17.16\%}$
test_step_mdp_speed[False-False-True-False-False] 62.1000μs 18.2224μs 54.8774 KOps/s 50.4207 KOps/s $\textbf{\color{#35bf28}+8.84\%}$
test_step_mdp_speed[False-False-False-True-True] 68.1000μs 41.2826μs 24.2233 KOps/s 20.1767 KOps/s $\textbf{\color{#35bf28}+20.06\%}$
test_step_mdp_speed[False-False-False-True-False] 0.5712ms 26.2695μs 38.0670 KOps/s 33.2985 KOps/s $\textbf{\color{#35bf28}+14.32\%}$
test_step_mdp_speed[False-False-False-False-True] 0.3891ms 32.4967μs 30.7723 KOps/s 26.4929 KOps/s $\textbf{\color{#35bf28}+16.15\%}$
test_step_mdp_speed[False-False-False-False-False] 0.1003ms 19.6166μs 50.9773 KOps/s 46.8839 KOps/s $\textbf{\color{#35bf28}+8.73\%}$
test_values[generalized_advantage_estimate-True-True] 14.2081ms 13.7127ms 72.9251 Ops/s 73.1378 Ops/s $\color{#d91a1a}-0.29\%$
test_values[vec_generalized_advantage_estimate-True-True] 55.9479ms 51.1759ms 19.5404 Ops/s 19.4208 Ops/s $\color{#35bf28}+0.62\%$
test_values[td0_return_estimate-False-False] 0.3351ms 0.2267ms 4.4118 KOps/s 4.6460 KOps/s $\textbf{\color{#d91a1a}-5.04\%}$
test_values[td1_return_estimate-False-False] 13.5919ms 13.4442ms 74.3816 Ops/s 75.1265 Ops/s $\color{#d91a1a}-0.99\%$
test_values[vec_td1_return_estimate-False-False] 53.9083ms 51.3749ms 19.4648 Ops/s 19.6556 Ops/s $\color{#d91a1a}-0.97\%$
test_values[td_lambda_return_estimate-True-False] 32.4980ms 31.8635ms 31.3839 Ops/s 31.2714 Ops/s $\color{#35bf28}+0.36\%$
test_values[vec_td_lambda_return_estimate-True-False] 52.0996ms 51.0358ms 19.5941 Ops/s 19.6179 Ops/s $\color{#d91a1a}-0.12\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.8803ms 11.7823ms 84.8731 Ops/s 84.2680 Ops/s $\color{#35bf28}+0.72\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 4.5103ms 2.5088ms 398.5931 Ops/s 406.6630 Ops/s $\color{#d91a1a}-1.98\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1.1479ms 0.4081ms 2.4504 KOps/s 2.4509 KOps/s $\color{#d91a1a}-0.02\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 56.3613ms 54.2759ms 18.4244 Ops/s 19.8802 Ops/s $\textbf{\color{#d91a1a}-7.32\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.8915ms 3.8795ms 257.7620 Ops/s 257.4647 Ops/s $\color{#35bf28}+0.12\%$
test_dqn_speed 1.9663ms 1.4622ms 683.8787 Ops/s 669.6634 Ops/s $\color{#35bf28}+2.12\%$
test_ddpg_speed 2.8766ms 2.3169ms 431.6078 Ops/s 423.1852 Ops/s $\color{#35bf28}+1.99\%$
test_sac_speed 8.5632ms 7.8183ms 127.9043 Ops/s 129.5184 Ops/s $\color{#d91a1a}-1.25\%$
test_redq_speed 85.1010ms 14.6190ms 68.4042 Ops/s 73.0181 Ops/s $\textbf{\color{#d91a1a}-6.32\%}$
test_redq_deprec_speed 12.9700ms 11.6442ms 85.8794 Ops/s 85.3426 Ops/s $\color{#35bf28}+0.63\%$
test_td3_speed 12.0880ms 10.4685ms 95.5248 Ops/s 95.9350 Ops/s $\color{#d91a1a}-0.43\%$
test_cql_speed 39.9161ms 36.4129ms 27.4628 Ops/s 38.2853 Ops/s $\textbf{\color{#d91a1a}-28.27\%}$
test_a2c_speed 6.9945ms 5.4507ms 183.4629 Ops/s 175.3743 Ops/s $\color{#35bf28}+4.61\%$
test_ppo_speed 10.4475ms 5.8397ms 171.2405 Ops/s 163.9416 Ops/s $\color{#35bf28}+4.45\%$
test_reinforce_speed 6.1170ms 4.1182ms 242.8263 Ops/s 235.2766 Ops/s $\color{#35bf28}+3.21\%$
test_iql_speed 26.7753ms 21.6667ms 46.1537 Ops/s 48.1132 Ops/s $\color{#d91a1a}-4.07\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.4240ms 3.5095ms 284.9421 Ops/s 276.4430 Ops/s $\color{#35bf28}+3.07\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.6571ms 3.6707ms 272.4260 Ops/s 271.6957 Ops/s $\color{#35bf28}+0.27\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 6.7729ms 3.6587ms 273.3222 Ops/s 270.2724 Ops/s $\color{#35bf28}+1.13\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.3285ms 3.4716ms 288.0505 Ops/s 275.6701 Ops/s $\color{#35bf28}+4.49\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.9867ms 3.6413ms 274.6298 Ops/s 270.3171 Ops/s $\color{#35bf28}+1.60\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.5635ms 3.6170ms 276.4746 Ops/s 239.4206 Ops/s $\textbf{\color{#35bf28}+15.48\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.7301ms 3.5579ms 281.0680 Ops/s 276.2378 Ops/s $\color{#35bf28}+1.75\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 6.9857ms 3.6436ms 274.4538 Ops/s 269.9087 Ops/s $\color{#35bf28}+1.68\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.2942ms 3.6442ms 274.4052 Ops/s 267.6865 Ops/s $\color{#35bf28}+2.51\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1040s 3.9179ms 255.2399 Ops/s 243.5715 Ops/s $\color{#35bf28}+4.79\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 6.5294ms 3.6145ms 276.6646 Ops/s 268.6029 Ops/s $\color{#35bf28}+3.00\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.8286ms 3.6359ms 275.0377 Ops/s 238.5300 Ops/s $\textbf{\color{#35bf28}+15.31\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.5813ms 3.4896ms 286.5683 Ops/s 276.2164 Ops/s $\color{#35bf28}+3.75\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.4805ms 3.6267ms 275.7358 Ops/s 265.5042 Ops/s $\color{#35bf28}+3.85\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.6519ms 3.6134ms 276.7504 Ops/s 262.6265 Ops/s $\textbf{\color{#35bf28}+5.38\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1023s 3.8334ms 260.8630 Ops/s 248.7772 Ops/s $\color{#35bf28}+4.86\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 7.1482ms 3.6236ms 275.9679 Ops/s 264.6614 Ops/s $\color{#35bf28}+4.27\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.4635ms 3.6370ms 274.9541 Ops/s 267.3758 Ops/s $\color{#35bf28}+2.83\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2274s 31.5941ms 31.6515 Ops/s 32.8434 Ops/s $\color{#d91a1a}-3.63\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1339s 27.5542ms 36.2921 Ops/s 36.3167 Ops/s $\color{#d91a1a}-0.07\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1299s 27.1972ms 36.7685 Ops/s 36.0350 Ops/s $\color{#35bf28}+2.04\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1323s 27.2465ms 36.7019 Ops/s 36.2608 Ops/s $\color{#35bf28}+1.22\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1335s 27.4178ms 36.4727 Ops/s 36.1421 Ops/s $\color{#35bf28}+0.91\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1303s 29.4169ms 33.9940 Ops/s 33.6480 Ops/s $\color{#35bf28}+1.03\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1315s 27.2135ms 36.7465 Ops/s 36.2134 Ops/s $\color{#35bf28}+1.47\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1301s 27.2102ms 36.7510 Ops/s 36.3252 Ops/s $\color{#35bf28}+1.17\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1322s 27.2355ms 36.7167 Ops/s 36.1572 Ops/s $\color{#35bf28}+1.55\%$

matteobettini
matteobettini approved these changes Jul 6, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM les do this!

@vmoens vmoens merged commit b5cddf0 into main Jul 6, 2023
@vmoens vmoens deleted the followu-473 branch July 6, 2023 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot