Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix args/kwargs passing in advantages #2001

Merged
merged 1 commit into from
Mar 7, 2024
Merged

[BugFix] Fix args/kwargs passing in advantages #2001

merged 1 commit into from
Mar 7, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 7, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2001

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 19 Unrelated Failures

As of commit 535c1e0 with merge base 07eb02d (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 7, 2024
@vmoens vmoens added the bug Something isn't working label Mar 7, 2024
@vmoens vmoens marked this pull request as ready for review March 7, 2024 18:03
@vmoens vmoens merged commit 130a213 into main Mar 7, 2024
28 of 46 checks passed
@vmoens vmoens deleted the fix-dim-args branch March 7, 2024 18:03
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 61.4534ms 60.2016ms 16.6109 Ops/s 16.4849 Ops/s $\color{#35bf28}+0.76\%$
test_sync 36.7056ms 32.6503ms 30.6276 Ops/s 29.9106 Ops/s $\color{#35bf28}+2.40\%$
test_async 61.3446ms 29.9534ms 33.3852 Ops/s 34.2656 Ops/s $\color{#d91a1a}-2.57\%$
test_simple 0.4782s 0.4219s 2.3703 Ops/s 2.3659 Ops/s $\color{#35bf28}+0.19\%$
test_transformed 0.6210s 0.5707s 1.7523 Ops/s 1.7153 Ops/s $\color{#35bf28}+2.16\%$
test_serial 1.4417s 1.3923s 0.7182 Ops/s 0.7003 Ops/s $\color{#35bf28}+2.57\%$
test_parallel 1.3888s 1.3410s 0.7457 Ops/s 0.7272 Ops/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-True-True-True-True] 0.1349ms 22.2050μs 45.0348 KOps/s 46.2408 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-True-True-True-False] 62.0170μs 13.4880μs 74.1400 KOps/s 74.6064 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-True-False-True] 52.8690μs 12.9893μs 76.9862 KOps/s 78.8226 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[True-True-True-False-False] 61.7460μs 7.8345μs 127.6404 KOps/s 129.4810 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-True-False-True-True] 74.7210μs 23.3842μs 42.7638 KOps/s 42.4498 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-False-True-False] 41.7790μs 14.9978μs 66.6763 KOps/s 68.5997 KOps/s $\color{#d91a1a}-2.80\%$
test_step_mdp_speed[True-True-False-False-True] 32.8110μs 14.2245μs 70.3012 KOps/s 70.5756 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[True-True-False-False-False] 30.0070μs 9.1319μs 109.5067 KOps/s 111.3817 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[True-False-True-True-True] 69.4400μs 24.8619μs 40.2222 KOps/s 40.2089 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[True-False-True-True-False] 42.9710μs 16.0865μs 62.1639 KOps/s 62.0440 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-True-False-True] 61.1550μs 14.3023μs 69.9190 KOps/s 69.7506 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-False-True-False-False] 30.1070μs 9.0752μs 110.1908 KOps/s 111.1079 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-False-False-True-True] 67.5970μs 25.9617μs 38.5183 KOps/s 38.1512 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-False-False-True-False] 63.4000μs 17.1582μs 58.2812 KOps/s 58.0873 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-False-False-False-True] 42.7810μs 15.2907μs 65.3994 KOps/s 65.6020 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-False-False-False] 49.7130μs 10.3009μs 97.0788 KOps/s 97.0542 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-True-True-True] 57.3870μs 25.1129μs 39.8202 KOps/s 39.3092 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-True-True-True-False] 71.1940μs 16.2059μs 61.7059 KOps/s 62.5337 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[False-True-True-False-True] 53.6510μs 16.6424μs 60.0876 KOps/s 59.4755 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-True-True-False-False] 32.5110μs 10.4873μs 95.3533 KOps/s 95.9679 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-False-True-True] 33.7740μs 26.0822μs 38.3403 KOps/s 37.6042 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-True-False-True-False] 63.7600μs 17.2230μs 58.0619 KOps/s 57.8754 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-True-False-False-True] 42.4300μs 17.5096μs 57.1114 KOps/s 56.4345 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-False-False-False] 59.1510μs 11.5440μs 86.6253 KOps/s 87.0602 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-False-True-True-True] 67.9280μs 27.0865μs 36.9187 KOps/s 36.8559 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-True-True-False] 52.1090μs 18.6274μs 53.6843 KOps/s 54.8705 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-False-True-False-True] 65.2130μs 17.7743μs 56.2611 KOps/s 56.7003 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-True-False-False] 39.0830μs 11.4404μs 87.4095 KOps/s 86.2622 KOps/s $\color{#35bf28}+1.33\%$
test_step_mdp_speed[False-False-False-True-True] 76.4340μs 28.0068μs 35.7056 KOps/s 35.1136 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[False-False-False-True-False] 65.2530μs 19.7695μs 50.5829 KOps/s 50.6723 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-False-False-False-True] 50.1640μs 18.8351μs 53.0923 KOps/s 52.6958 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-False-False-False] 32.5320μs 12.7048μs 78.7103 KOps/s 79.8607 KOps/s $\color{#d91a1a}-1.44\%$
test_values[generalized_advantage_estimate-True-True] 9.3765ms 9.0716ms 110.2338 Ops/s 108.4449 Ops/s $\color{#35bf28}+1.65\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.6764ms 33.5133ms 29.8389 Ops/s 28.1586 Ops/s $\textbf{\color{#35bf28}+5.97\%}$
test_values[td0_return_estimate-False-False] 0.2361ms 0.1681ms 5.9486 KOps/s 5.9513 KOps/s $\color{#d91a1a}-0.04\%$
test_values[td1_return_estimate-False-False] 27.2100ms 22.8847ms 43.6973 Ops/s 43.0982 Ops/s $\color{#35bf28}+1.39\%$
test_values[vec_td1_return_estimate-False-False] 35.2037ms 33.5838ms 29.7763 Ops/s 28.1438 Ops/s $\textbf{\color{#35bf28}+5.80\%}$
test_values[td_lambda_return_estimate-True-False] 34.3372ms 33.1248ms 30.1889 Ops/s 29.7855 Ops/s $\color{#35bf28}+1.35\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.4448ms 33.5940ms 29.7672 Ops/s 28.1200 Ops/s $\textbf{\color{#35bf28}+5.86\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.7646ms 7.9853ms 125.2297 Ops/s 123.3544 Ops/s $\color{#35bf28}+1.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9121ms 1.8163ms 550.5835 Ops/s 497.5632 Ops/s $\textbf{\color{#35bf28}+10.66\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4165ms 0.3478ms 2.8748 KOps/s 2.8375 KOps/s $\color{#35bf28}+1.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.5776ms 41.7455ms 23.9547 Ops/s 21.3643 Ops/s $\textbf{\color{#35bf28}+12.12\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.6525ms 3.0158ms 331.5822 Ops/s 329.8534 Ops/s $\color{#35bf28}+0.52\%$
test_dqn_speed 7.3202ms 1.3758ms 726.8530 Ops/s 705.5100 Ops/s $\color{#35bf28}+3.03\%$
test_ddpg_speed 2.8129ms 2.6910ms 371.6057 Ops/s 358.5218 Ops/s $\color{#35bf28}+3.65\%$
test_sac_speed 9.6676ms 8.2982ms 120.5080 Ops/s 119.1754 Ops/s $\color{#35bf28}+1.12\%$
test_redq_speed 14.2626ms 13.1945ms 75.7889 Ops/s 76.9520 Ops/s $\color{#d91a1a}-1.51\%$
test_redq_deprec_speed 83.9326ms 14.1906ms 70.4691 Ops/s 76.1832 Ops/s $\textbf{\color{#d91a1a}-7.50\%}$
test_td3_speed 8.4558ms 8.1894ms 122.1085 Ops/s 120.1137 Ops/s $\color{#35bf28}+1.66\%$
test_cql_speed 37.8515ms 36.3905ms 27.4797 Ops/s 27.6047 Ops/s $\color{#d91a1a}-0.45\%$
test_a2c_speed 8.6217ms 7.3073ms 136.8486 Ops/s 135.2087 Ops/s $\color{#35bf28}+1.21\%$
test_ppo_speed 10.0879ms 7.5705ms 132.0917 Ops/s 130.6117 Ops/s $\color{#35bf28}+1.13\%$
test_reinforce_speed 7.7012ms 6.5159ms 153.4708 Ops/s 152.3577 Ops/s $\color{#35bf28}+0.73\%$
test_iql_speed 34.1618ms 32.5245ms 30.7461 Ops/s 30.6194 Ops/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3976ms 2.1671ms 461.4421 Ops/s 456.0361 Ops/s $\color{#35bf28}+1.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8336ms 0.4955ms 2.0184 KOps/s 1.9776 KOps/s $\color{#35bf28}+2.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7286ms 0.4694ms 2.1302 KOps/s 2.0907 KOps/s $\color{#35bf28}+1.89\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4320ms 2.1364ms 468.0681 Ops/s 465.7651 Ops/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0399ms 0.4856ms 2.0591 KOps/s 2.0251 KOps/s $\color{#35bf28}+1.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7581ms 0.4663ms 2.1446 KOps/s 2.1305 KOps/s $\color{#35bf28}+0.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8627ms 1.3028ms 767.5684 Ops/s 773.7135 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4492ms 1.1925ms 838.5876 Ops/s 721.3007 Ops/s $\textbf{\color{#35bf28}+16.26\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5317ms 2.2417ms 446.0951 Ops/s 437.4993 Ops/s $\color{#35bf28}+1.96\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7419ms 0.6047ms 1.6538 KOps/s 1.6135 KOps/s $\color{#35bf28}+2.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.9074ms 0.5804ms 1.7230 KOps/s 1.6918 KOps/s $\color{#35bf28}+1.84\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.5046ms 2.0979ms 476.6577 Ops/s 469.0939 Ops/s $\color{#35bf28}+1.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6177ms 0.4932ms 2.0274 KOps/s 1.6185 KOps/s $\textbf{\color{#35bf28}+25.26\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.7112ms 0.4774ms 2.0946 KOps/s 2.0877 KOps/s $\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5994ms 2.1356ms 468.2460 Ops/s 456.6821 Ops/s $\color{#35bf28}+2.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5840ms 0.4848ms 2.0629 KOps/s 2.0033 KOps/s $\color{#35bf28}+2.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6247ms 0.4611ms 2.1688 KOps/s 2.1298 KOps/s $\color{#35bf28}+1.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.4325ms 2.1784ms 459.0522 Ops/s 439.5932 Ops/s $\color{#35bf28}+4.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0430ms 0.6085ms 1.6433 KOps/s 1.6245 KOps/s $\color{#35bf28}+1.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7341ms 0.5799ms 1.7244 KOps/s 1.6744 KOps/s $\color{#35bf28}+2.99\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 98.2356ms 7.2425ms 138.0732 Ops/s 176.7666 Ops/s $\textbf{\color{#d91a1a}-21.89\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.5751ms 12.1850ms 82.0679 Ops/s 82.2618 Ops/s $\color{#d91a1a}-0.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8696ms 1.1064ms 903.8552 Ops/s 975.1992 Ops/s $\textbf{\color{#d91a1a}-7.32\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 94.0643ms 7.2052ms 138.7881 Ops/s 141.3597 Ops/s $\color{#d91a1a}-1.82\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.4112ms 12.1129ms 82.5567 Ops/s 82.5319 Ops/s $\color{#35bf28}+0.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5385ms 1.0479ms 954.2442 Ops/s 978.0965 Ops/s $\color{#d91a1a}-2.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 92.5984ms 5.7201ms 174.8215 Ops/s 177.3063 Ops/s $\color{#d91a1a}-1.40\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.4933ms 12.1567ms 82.2595 Ops/s 70.1754 Ops/s $\textbf{\color{#35bf28}+17.22\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4013ms 1.4190ms 704.7180 Ops/s 707.0338 Ops/s $\color{#d91a1a}-0.33\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 7, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1163s 0.1154s 8.6682 Ops/s 8.9257 Ops/s $\color{#d91a1a}-2.88\%$
test_sync 95.2562ms 94.8845ms 10.5391 Ops/s 10.4944 Ops/s $\color{#35bf28}+0.43\%$
test_async 0.1793s 90.8637ms 11.0055 Ops/s 11.0166 Ops/s $\color{#d91a1a}-0.10\%$
test_single_pixels 0.1240s 0.1237s 8.0868 Ops/s 8.1540 Ops/s $\color{#d91a1a}-0.82\%$
test_sync_pixels 81.2600ms 76.9082ms 13.0025 Ops/s 12.4564 Ops/s $\color{#35bf28}+4.38\%$
test_async_pixels 0.1544s 76.6333ms 13.0492 Ops/s 14.9976 Ops/s $\textbf{\color{#d91a1a}-12.99\%}$
test_simple 0.8867s 0.8266s 1.2097 Ops/s 1.2237 Ops/s $\color{#d91a1a}-1.14\%$
test_transformed 1.1186s 1.0581s 0.9451 Ops/s 0.9618 Ops/s $\color{#d91a1a}-1.74\%$
test_serial 2.4899s 2.4291s 0.4117 Ops/s 0.4215 Ops/s $\color{#d91a1a}-2.32\%$
test_parallel 2.2000s 2.1128s 0.4733 Ops/s 0.4773 Ops/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-True-True-True] 0.1003ms 32.3668μs 30.8959 KOps/s 30.8842 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-True-True-True-False] 0.1474ms 19.5481μs 51.1559 KOps/s 50.6443 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[True-True-True-False-True] 37.3800μs 18.5214μs 53.9915 KOps/s 53.8147 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-True-False-False] 36.2200μs 11.1545μs 89.6502 KOps/s 89.1524 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[True-True-False-True-True] 56.7710μs 34.7137μs 28.8070 KOps/s 28.5704 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-True-False-True-False] 39.2500μs 21.3896μs 46.7517 KOps/s 46.3410 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-True-False-False-True] 38.8400μs 20.6154μs 48.5075 KOps/s 48.2744 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-True-False-False-False] 37.7710μs 13.0844μs 76.4270 KOps/s 75.7394 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-True-True-True] 61.2510μs 36.8247μs 27.1557 KOps/s 26.9047 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-False-True-True-False] 48.0910μs 23.4906μs 42.5703 KOps/s 42.2209 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-True-False-True] 37.7710μs 20.4113μs 48.9925 KOps/s 48.3944 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-True-False-False] 80.2320μs 13.1328μs 76.1454 KOps/s 76.3284 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-False-False-True-True] 87.0020μs 38.2697μs 26.1303 KOps/s 25.7399 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-False-False-True-False] 45.1210μs 25.0925μs 39.8526 KOps/s 39.3890 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-False-True] 47.0610μs 22.0452μs 45.3614 KOps/s 44.9264 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-False-False-False] 29.6900μs 14.9003μs 67.1127 KOps/s 67.2831 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-True-True-True-True] 0.1035ms 36.6173μs 27.3095 KOps/s 26.9556 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[False-True-True-True-False] 42.2810μs 23.3882μs 42.7566 KOps/s 42.0251 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-True-True-False-True] 0.1875ms 24.2840μs 41.1794 KOps/s 41.2368 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-True-True-False-False] 0.2027ms 15.0233μs 66.5633 KOps/s 67.1383 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[False-True-False-True-True] 74.5620μs 38.9865μs 25.6499 KOps/s 25.3982 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-True-False-True-False] 0.1994ms 25.3244μs 39.4877 KOps/s 38.8386 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-True-False-False-True] 0.2217ms 26.2767μs 38.0566 KOps/s 38.0532 KOps/s $+0.01\%$
test_step_mdp_speed[False-True-False-False-False] 0.1914ms 16.8019μs 59.5171 KOps/s 59.7719 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-False-True-True-True] 0.2532ms 40.7958μs 24.5123 KOps/s 24.6516 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[False-False-True-True-False] 0.1862ms 27.2656μs 36.6763 KOps/s 36.3312 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-False-True-False-True] 49.4610μs 26.2127μs 38.1495 KOps/s 37.6354 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[False-False-True-False-False] 37.1800μs 16.8170μs 59.4635 KOps/s 59.5956 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-False-True-True] 0.2107ms 41.7581μs 23.9475 KOps/s 23.8048 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-False-True-False] 63.5710μs 29.0593μs 34.4124 KOps/s 34.2591 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-False-False-False-True] 51.7510μs 27.6666μs 36.1447 KOps/s 36.2441 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-False-False-False] 41.8710μs 18.5002μs 54.0536 KOps/s 54.0711 KOps/s $\color{#d91a1a}-0.03\%$
test_values[generalized_advantage_estimate-True-True] 25.9428ms 25.2247ms 39.6437 Ops/s 41.8646 Ops/s $\textbf{\color{#d91a1a}-5.31\%}$
test_values[vec_generalized_advantage_estimate-True-True] 84.4744ms 3.2488ms 307.8014 Ops/s 308.1133 Ops/s $\color{#d91a1a}-0.10\%$
test_values[td0_return_estimate-False-False] 98.6220μs 65.3576μs 15.3004 KOps/s 15.9112 KOps/s $\color{#d91a1a}-3.84\%$
test_values[td1_return_estimate-False-False] 55.0718ms 54.1614ms 18.4633 Ops/s 19.4960 Ops/s $\textbf{\color{#d91a1a}-5.30\%}$
test_values[vec_td1_return_estimate-False-False] 2.1655ms 1.7683ms 565.5275 Ops/s 569.3957 Ops/s $\color{#d91a1a}-0.68\%$
test_values[td_lambda_return_estimate-True-False] 87.3882ms 86.4715ms 11.5645 Ops/s 12.0465 Ops/s $\color{#d91a1a}-4.00\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1223ms 1.7640ms 566.8959 Ops/s 570.3697 Ops/s $\color{#d91a1a}-0.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.1601ms 23.7002ms 42.1937 Ops/s 43.1795 Ops/s $\color{#d91a1a}-2.28\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8755ms 0.6850ms 1.4599 KOps/s 1.4392 KOps/s $\color{#35bf28}+1.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8389ms 0.6637ms 1.5068 KOps/s 1.5529 KOps/s $\color{#d91a1a}-2.97\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6220ms 1.4657ms 682.2868 Ops/s 694.0920 Ops/s $\color{#d91a1a}-1.70\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9484ms 0.6815ms 1.4674 KOps/s 1.5117 KOps/s $\color{#d91a1a}-2.93\%$
test_dqn_speed 9.0825ms 1.4688ms 680.8087 Ops/s 652.8008 Ops/s $\color{#35bf28}+4.29\%$
test_ddpg_speed 3.1196ms 2.8127ms 355.5353 Ops/s 371.9262 Ops/s $\color{#d91a1a}-4.41\%$
test_sac_speed 8.7901ms 8.3031ms 120.4371 Ops/s 126.2450 Ops/s $\color{#d91a1a}-4.60\%$
test_redq_speed 90.3946ms 11.5054ms 86.9155 Ops/s 96.2335 Ops/s $\textbf{\color{#d91a1a}-9.68\%}$
test_redq_deprec_speed 12.5204ms 11.5956ms 86.2396 Ops/s 89.8217 Ops/s $\color{#d91a1a}-3.99\%$
test_td3_speed 8.3228ms 8.1997ms 121.9555 Ops/s 127.3339 Ops/s $\color{#d91a1a}-4.22\%$
test_cql_speed 27.6900ms 26.2457ms 38.1015 Ops/s 39.8841 Ops/s $\color{#d91a1a}-4.47\%$
test_a2c_speed 5.9995ms 5.7196ms 174.8360 Ops/s 181.4161 Ops/s $\color{#d91a1a}-3.63\%$
test_ppo_speed 6.3937ms 5.9823ms 167.1602 Ops/s 171.8962 Ops/s $\color{#d91a1a}-2.76\%$
test_reinforce_speed 5.5692ms 4.6239ms 216.2694 Ops/s 220.2915 Ops/s $\color{#d91a1a}-1.83\%$
test_iql_speed 20.2398ms 19.5540ms 51.1405 Ops/s 50.1342 Ops/s $\color{#35bf28}+2.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1189ms 2.9085ms 343.8247 Ops/s 343.1518 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2558ms 0.5500ms 1.8180 KOps/s 1.8245 KOps/s $\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7196ms 0.5286ms 1.8919 KOps/s 1.8900 KOps/s $\color{#35bf28}+0.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1904ms 2.9186ms 342.6322 Ops/s 344.6518 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5294ms 0.5417ms 1.8462 KOps/s 1.8549 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6891ms 0.5162ms 1.9374 KOps/s 1.9327 KOps/s $\color{#35bf28}+0.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 4.4122ms 1.5312ms 653.0975 Ops/s 674.6616 Ops/s $\color{#d91a1a}-3.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1173ms 1.5353ms 651.3376 Ops/s 705.5139 Ops/s $\textbf{\color{#d91a1a}-7.68\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2205ms 3.0385ms 329.1063 Ops/s 332.4278 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9334ms 0.6721ms 1.4879 KOps/s 1.4897 KOps/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6457ms 0.6497ms 1.5392 KOps/s 1.5291 KOps/s $\color{#35bf28}+0.66\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1179ms 2.9131ms 343.2795 Ops/s 347.1462 Ops/s $\color{#d91a1a}-1.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7410ms 0.5469ms 1.8284 KOps/s 1.8082 KOps/s $\color{#35bf28}+1.12\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5535ms 0.5309ms 1.8836 KOps/s 1.8751 KOps/s $\color{#35bf28}+0.45\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1940ms 2.9331ms 340.9381 Ops/s 343.8863 Ops/s $\color{#d91a1a}-0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5807ms 0.5406ms 1.8497 KOps/s 1.8280 KOps/s $\color{#35bf28}+1.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7166ms 0.5205ms 1.9213 KOps/s 1.9188 KOps/s $\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2089ms 3.0559ms 327.2379 Ops/s 329.2190 Ops/s $\color{#d91a1a}-0.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8645ms 0.6750ms 1.4816 KOps/s 1.4911 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.9415ms 0.6590ms 1.5175 KOps/s 1.2534 KOps/s $\textbf{\color{#35bf28}+21.07\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1134s 9.0231ms 110.8265 Ops/s 146.9276 Ops/s $\textbf{\color{#d91a1a}-24.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.0993ms 14.5192ms 68.8741 Ops/s 69.3886 Ops/s $\color{#d91a1a}-0.74\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2075ms 1.1069ms 903.4296 Ops/s 920.5852 Ops/s $\color{#d91a1a}-1.86\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1060s 8.7428ms 114.3797 Ops/s 112.6426 Ops/s $\color{#35bf28}+1.54\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.0355ms 14.5069ms 68.9326 Ops/s 69.2000 Ops/s $\color{#d91a1a}-0.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1048ms 1.0920ms 915.7860 Ops/s 923.9685 Ops/s $\color{#d91a1a}-0.89\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1050s 7.1892ms 139.0968 Ops/s 138.4952 Ops/s $\color{#35bf28}+0.43\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.2678ms 14.7418ms 67.8342 Ops/s 59.7757 Ops/s $\textbf{\color{#35bf28}+13.48\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5752ms 1.4496ms 689.8239 Ops/s 714.3311 Ops/s $\color{#d91a1a}-3.43\%$

vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[BugFix] Fix args/kwargs passing in advantages (#2001)
dd15127 
(cherry picked from commit 130a213)
vmoens added a commit that referenced this pull request Apr 7, 2024
@vmoens
[BugFix] Fix args/kwargs passing in advantages (#2001)
064658d 
(cherry picked from commit 130a213)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot