Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Expose MARL modules #2321

Merged
merged 2 commits into from
Jul 25, 2024
Merged

[BugFix] Expose MARL modules #2321

merged 2 commits into from
Jul 25, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 25, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 25, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2321

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Pending, 2 Unrelated Failures

As of commit 1420edc with merge base c6ef080 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 25, 2024
@vmoens vmoens linked an issue Jul 25, 2024 that may be closed by this pull request
[BUG] SACLoss module does not allow stochastic modules (i.e. Dropout, etc.) due to vmap #2313
Closed
3 tasks
@vmoens vmoens added the bug Something isn't working label Jul 25, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.9278ms 57.8061ms 17.2992 Ops/s 16.9481 Ops/s $\color{#35bf28}+2.07\%$
test_sync 49.5898ms 34.2150ms 29.2270 Ops/s 31.4207 Ops/s $\textbf{\color{#d91a1a}-6.98\%}$
test_async 54.3993ms 30.0475ms 33.2806 Ops/s 33.3922 Ops/s $\color{#d91a1a}-0.33\%$
test_simple 0.5024s 0.4149s 2.4104 Ops/s 2.4264 Ops/s $\color{#d91a1a}-0.66\%$
test_transformed 0.6220s 0.5681s 1.7603 Ops/s 1.7963 Ops/s $\color{#d91a1a}-2.00\%$
test_serial 1.3146s 1.2535s 0.7977 Ops/s 0.7870 Ops/s $\color{#35bf28}+1.37\%$
test_parallel 1.1741s 1.1182s 0.8943 Ops/s 0.9004 Ops/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-True-True-True] 0.1944ms 25.7577μs 38.8233 KOps/s 39.2489 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-True-True-False] 62.7670μs 14.7677μs 67.7154 KOps/s 68.0905 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-True-True-False-True] 74.8600μs 14.7192μs 67.9384 KOps/s 68.4790 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-True-False-False] 35.5660μs 8.5173μs 117.4087 KOps/s 117.4823 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-True-False-True-True] 95.8990μs 27.1590μs 36.8202 KOps/s 36.9508 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-False-True-False] 57.2380μs 16.5384μs 60.4655 KOps/s 61.3710 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[True-True-False-False-True] 71.1430μs 16.3732μs 61.0754 KOps/s 60.2508 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-False-False-False] 37.4900μs 10.1541μs 98.4820 KOps/s 98.4174 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-True-True-True] 81.6430μs 28.9143μs 34.5849 KOps/s 34.4268 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-False-True-True-False] 71.0930μs 18.2671μs 54.7433 KOps/s 55.8977 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-False-True-False-True] 60.7140μs 16.4130μs 60.9275 KOps/s 61.1263 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-False-True-False-False] 55.4950μs 10.1368μs 98.6503 KOps/s 98.9840 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-False-False-True-True] 57.6080μs 30.4272μs 32.8653 KOps/s 33.1382 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-False-False-True-False] 69.8610μs 19.6486μs 50.8943 KOps/s 51.2944 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-False-False-False-True] 50.3340μs 17.8224μs 56.1090 KOps/s 56.5580 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-False-False-False-False] 70.6020μs 11.7233μs 85.3004 KOps/s 85.8412 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-True-True-True-True] 84.3880μs 28.7841μs 34.7414 KOps/s 34.6425 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-True-True-False] 45.1640μs 18.1976μs 54.9524 KOps/s 55.3029 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-True-True-False-True] 72.5660μs 18.9477μs 52.7769 KOps/s 53.1101 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-True-True-False-False] 27.0410μs 11.4365μs 87.4395 KOps/s 87.5546 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-False-True-True] 83.9470μs 30.4318μs 32.8604 KOps/s 33.2405 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[False-True-False-True-False] 72.6660μs 19.5906μs 51.0448 KOps/s 51.4463 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-True-False-False-True] 48.5810μs 20.3084μs 49.2408 KOps/s 49.4929 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-True-False-False-False] 59.3010μs 12.8260μs 77.9668 KOps/s 78.0795 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-False-True-True-True] 3.3941ms 32.7587μs 30.5262 KOps/s 31.2280 KOps/s $\color{#d91a1a}-2.25\%$
test_step_mdp_speed[False-False-True-True-False] 48.6510μs 21.2448μs 47.0703 KOps/s 47.8359 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[False-False-True-False-True] 53.4600μs 20.2376μs 49.4129 KOps/s 48.4621 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-True-False-False] 39.4340μs 12.8634μs 77.7397 KOps/s 78.1650 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-False-True-True] 87.0130μs 33.4323μs 29.9112 KOps/s 30.1205 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-False-True-False] 46.8480μs 22.5448μs 44.3561 KOps/s 44.3145 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-False-False-False-True] 74.8400μs 21.5366μs 46.4326 KOps/s 46.5562 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-False-False-False] 63.2180μs 14.3441μs 69.7152 KOps/s 69.9999 KOps/s $\color{#d91a1a}-0.41\%$
test_values[generalized_advantage_estimate-True-True] 10.8955ms 9.7021ms 103.0706 Ops/s 105.2550 Ops/s $\color{#d91a1a}-2.08\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.7187ms 35.4316ms 28.2234 Ops/s 29.8790 Ops/s $\textbf{\color{#d91a1a}-5.54\%}$
test_values[td0_return_estimate-False-False] 0.2466ms 0.1744ms 5.7334 KOps/s 5.9124 KOps/s $\color{#d91a1a}-3.03\%$
test_values[td1_return_estimate-False-False] 27.6493ms 23.8539ms 41.9219 Ops/s 41.7875 Ops/s $\color{#35bf28}+0.32\%$
test_values[vec_td1_return_estimate-False-False] 37.5231ms 35.5243ms 28.1498 Ops/s 29.8727 Ops/s $\textbf{\color{#d91a1a}-5.77\%}$
test_values[td_lambda_return_estimate-True-False] 38.0400ms 34.2744ms 29.1763 Ops/s 28.8884 Ops/s $\color{#35bf28}+1.00\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.7077ms 35.5774ms 28.1077 Ops/s 29.9729 Ops/s $\textbf{\color{#d91a1a}-6.22\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5868ms 8.4114ms 118.8864 Ops/s 118.8178 Ops/s $\color{#35bf28}+0.06\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3408ms 1.9943ms 501.4299 Ops/s 503.0225 Ops/s $\color{#d91a1a}-0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4247ms 0.3483ms 2.8714 KOps/s 2.7878 KOps/s $\color{#35bf28}+3.00\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.2681ms 45.5851ms 21.9370 Ops/s 21.6377 Ops/s $\color{#35bf28}+1.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.9557ms 3.0422ms 328.7121 Ops/s 330.0656 Ops/s $\color{#d91a1a}-0.41\%$
test_dqn_speed 1.7220ms 1.3600ms 735.2795 Ops/s 715.6047 Ops/s $\color{#35bf28}+2.75\%$
test_ddpg_speed 3.2992ms 2.8887ms 346.1795 Ops/s 339.8206 Ops/s $\color{#35bf28}+1.87\%$
test_sac_speed 10.1468ms 8.3852ms 119.2580 Ops/s 118.6032 Ops/s $\color{#35bf28}+0.55\%$
test_redq_speed 15.6527ms 13.7746ms 72.5974 Ops/s 74.7139 Ops/s $\color{#d91a1a}-2.83\%$
test_redq_deprec_speed 18.0453ms 13.5826ms 73.6237 Ops/s 73.9535 Ops/s $\color{#d91a1a}-0.45\%$
test_td3_speed 9.0115ms 8.3271ms 120.0894 Ops/s 119.7499 Ops/s $\color{#35bf28}+0.28\%$
test_cql_speed 37.9257ms 36.5756ms 27.3406 Ops/s 27.4145 Ops/s $\color{#d91a1a}-0.27\%$
test_a2c_speed 9.0838ms 7.5596ms 132.2819 Ops/s 134.1747 Ops/s $\color{#d91a1a}-1.41\%$
test_ppo_speed 8.9879ms 7.8664ms 127.1226 Ops/s 128.8012 Ops/s $\color{#d91a1a}-1.30\%$
test_reinforce_speed 8.1108ms 6.6227ms 150.9966 Ops/s 151.3628 Ops/s $\color{#d91a1a}-0.24\%$
test_iql_speed 34.4745ms 32.4720ms 30.7957 Ops/s 30.6680 Ops/s $\color{#35bf28}+0.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6039ms 4.9334ms 202.6985 Ops/s 202.5885 Ops/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8906ms 0.4803ms 2.0822 KOps/s 2.0849 KOps/s $\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6566ms 0.4554ms 2.1960 KOps/s 2.1959 KOps/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.5259ms 4.7852ms 208.9778 Ops/s 205.2560 Ops/s $\color{#35bf28}+1.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5734ms 0.4711ms 2.1227 KOps/s 2.1134 KOps/s $\color{#35bf28}+0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6362ms 0.4508ms 2.2182 KOps/s 2.1831 KOps/s $\color{#35bf28}+1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0180ms 1.6844ms 593.6737 Ops/s 585.1833 Ops/s $\color{#35bf28}+1.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2047ms 1.5944ms 627.1962 Ops/s 623.3717 Ops/s $\color{#35bf28}+0.61\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4021ms 5.1053ms 195.8757 Ops/s 199.2995 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7676ms 0.6186ms 1.6166 KOps/s 1.6271 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7370ms 0.5888ms 1.6984 KOps/s 1.6988 KOps/s $\color{#d91a1a}-0.02\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2371ms 4.8863ms 204.6551 Ops/s 203.7907 Ops/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6958ms 0.4856ms 2.0593 KOps/s 2.0628 KOps/s $\color{#d91a1a}-0.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.2257ms 0.4656ms 2.1476 KOps/s 2.1863 KOps/s $\color{#d91a1a}-1.77\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2684ms 4.8540ms 206.0140 Ops/s 206.7135 Ops/s $\color{#d91a1a}-0.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8664ms 0.4793ms 2.0864 KOps/s 2.0994 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7666ms 0.4553ms 2.1963 KOps/s 2.1720 KOps/s $\color{#35bf28}+1.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9198ms 4.9904ms 200.3850 Ops/s 200.8018 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1129s 0.7728ms 1.2940 KOps/s 1.5980 KOps/s $\textbf{\color{#d91a1a}-19.02\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7652ms 0.5886ms 1.6989 KOps/s 1.6787 KOps/s $\color{#35bf28}+1.20\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1141s 5.9747ms 167.3713 Ops/s 119.6717 Ops/s $\textbf{\color{#35bf28}+39.86\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.7836ms 12.7898ms 78.1871 Ops/s 77.0414 Ops/s $\color{#35bf28}+1.49\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.2388ms 1.2025ms 831.5768 Ops/s 914.7567 Ops/s $\textbf{\color{#d91a1a}-9.09\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1070s 5.8031ms 172.3207 Ops/s 168.6356 Ops/s $\color{#35bf28}+2.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.9603ms 12.7563ms 78.3928 Ops/s 75.3186 Ops/s $\color{#35bf28}+4.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.0206ms 1.1901ms 840.2551 Ops/s 885.5925 Ops/s $\textbf{\color{#d91a1a}-5.12\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1136s 8.2189ms 121.6703 Ops/s 164.5820 Ops/s $\textbf{\color{#d91a1a}-26.07\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.1747ms 12.9211ms 77.3929 Ops/s 75.6691 Ops/s $\color{#35bf28}+2.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.9979ms 1.3383ms 747.2251 Ops/s 803.5026 Ops/s $\textbf{\color{#d91a1a}-7.00\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1085s 0.1083s 9.2337 Ops/s 9.3238 Ops/s $\color{#d91a1a}-0.97\%$
test_sync 96.1498ms 95.1794ms 10.5065 Ops/s 10.4798 Ops/s $\color{#35bf28}+0.25\%$
test_async 0.1794s 90.4379ms 11.0573 Ops/s 11.0329 Ops/s $\color{#35bf28}+0.22\%$
test_single_pixels 0.1182s 0.1179s 8.4789 Ops/s 8.4873 Ops/s $\color{#d91a1a}-0.10\%$
test_sync_pixels 76.8460ms 74.5811ms 13.4082 Ops/s 13.4233 Ops/s $\color{#d91a1a}-0.11\%$
test_async_pixels 0.1395s 69.1012ms 14.4715 Ops/s 14.1013 Ops/s $\color{#35bf28}+2.63\%$
test_simple 0.8743s 0.7986s 1.2521 Ops/s 1.2366 Ops/s $\color{#35bf28}+1.26\%$
test_transformed 1.1053s 1.0336s 0.9674 Ops/s 1.0001 Ops/s $\color{#d91a1a}-3.26\%$
test_serial 2.3211s 2.2510s 0.4443 Ops/s 0.4425 Ops/s $\color{#35bf28}+0.40\%$
test_parallel 2.0418s 1.9785s 0.5054 Ops/s 0.5012 Ops/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-True-True-True-True] 0.1405ms 37.0958μs 26.9572 KOps/s 27.5130 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-True-True-True-False] 0.1820ms 21.2908μs 46.9686 KOps/s 48.8479 KOps/s $\color{#d91a1a}-3.85\%$
test_step_mdp_speed[True-True-True-False-True] 0.2102ms 21.3320μs 46.8780 KOps/s 48.7620 KOps/s $\color{#d91a1a}-3.86\%$
test_step_mdp_speed[True-True-True-False-False] 27.6200μs 12.0442μs 83.0276 KOps/s 85.5272 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[True-True-False-True-True] 57.5610μs 39.6455μs 25.2235 KOps/s 25.9305 KOps/s $\color{#d91a1a}-2.73\%$
test_step_mdp_speed[True-True-False-True-False] 0.2193ms 23.3659μs 42.7974 KOps/s 44.4144 KOps/s $\color{#d91a1a}-3.64\%$
test_step_mdp_speed[True-True-False-False-True] 41.1110μs 23.1852μs 43.1310 KOps/s 44.0709 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[True-True-False-False-False] 0.1251ms 14.1278μs 70.7823 KOps/s 71.7281 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-False-True-True-True] 66.3910μs 41.8011μs 23.9228 KOps/s 24.9453 KOps/s $\color{#d91a1a}-4.10\%$
test_step_mdp_speed[True-False-True-True-False] 45.5810μs 25.5993μs 39.0635 KOps/s 40.0440 KOps/s $\color{#d91a1a}-2.45\%$
test_step_mdp_speed[True-False-True-False-True] 40.1010μs 22.7180μs 44.0180 KOps/s 44.3313 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-False-True-False-False] 31.5510μs 14.1000μs 70.9220 KOps/s 72.7164 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-False-False-True-True] 62.2220μs 42.9041μs 23.3078 KOps/s 23.2761 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-False-True-False] 46.0510μs 27.8352μs 35.9257 KOps/s 36.6415 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[True-False-False-False-True] 45.2410μs 25.0504μs 39.9194 KOps/s 40.1405 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-False-False-False-False] 37.8510μs 16.1591μs 61.8847 KOps/s 62.6790 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[False-True-True-True-True] 70.3220μs 41.5141μs 24.0882 KOps/s 24.7934 KOps/s $\color{#d91a1a}-2.84\%$
test_step_mdp_speed[False-True-True-True-False] 0.1301ms 25.4152μs 39.3465 KOps/s 40.2057 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-True-True-False-True] 64.7720μs 27.6405μs 36.1788 KOps/s 37.4175 KOps/s $\color{#d91a1a}-3.31\%$
test_step_mdp_speed[False-True-True-False-False] 0.1767ms 16.2737μs 61.4490 KOps/s 62.9768 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[False-True-False-True-True] 0.2342ms 43.6739μs 22.8970 KOps/s 23.2989 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-True-False-True-False] 90.7320μs 27.7617μs 36.0209 KOps/s 36.8832 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[False-True-False-False-True] 49.6910μs 29.3976μs 34.0163 KOps/s 34.6518 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[False-True-False-False-False] 0.2125ms 18.5283μs 53.9714 KOps/s 55.2290 KOps/s $\color{#d91a1a}-2.28\%$
test_step_mdp_speed[False-False-True-True-True] 3.9132ms 46.5919μs 21.4630 KOps/s 22.1537 KOps/s $\color{#d91a1a}-3.12\%$
test_step_mdp_speed[False-False-True-True-False] 59.9820μs 30.2052μs 33.1069 KOps/s 34.0757 KOps/s $\color{#d91a1a}-2.84\%$
test_step_mdp_speed[False-False-True-False-True] 0.1742ms 29.3902μs 34.0250 KOps/s 34.3493 KOps/s $\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-False-True-False-False] 0.1115ms 18.3915μs 54.3729 KOps/s 54.0061 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-False-False-True-True] 79.0320μs 47.3963μs 21.0987 KOps/s 21.7921 KOps/s $\color{#d91a1a}-3.18\%$
test_step_mdp_speed[False-False-False-True-False] 53.4110μs 32.0818μs 31.1704 KOps/s 32.1542 KOps/s $\color{#d91a1a}-3.06\%$
test_step_mdp_speed[False-False-False-False-True] 60.0810μs 31.5176μs 31.7283 KOps/s 32.5059 KOps/s $\color{#d91a1a}-2.39\%$
test_step_mdp_speed[False-False-False-False-False] 37.4810μs 20.2299μs 49.4317 KOps/s 50.0368 KOps/s $\color{#d91a1a}-1.21\%$
test_values[generalized_advantage_estimate-True-True] 25.5451ms 24.4371ms 40.9213 Ops/s 41.4756 Ops/s $\color{#d91a1a}-1.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.6744ms 2.8377ms 352.4016 Ops/s 353.6507 Ops/s $\color{#d91a1a}-0.35\%$
test_values[td0_return_estimate-False-False] 93.2720μs 67.5823μs 14.7968 KOps/s 15.2973 KOps/s $\color{#d91a1a}-3.27\%$
test_values[td1_return_estimate-False-False] 54.9462ms 54.5225ms 18.3410 Ops/s 18.3130 Ops/s $\color{#35bf28}+0.15\%$
test_values[vec_td1_return_estimate-False-False] 1.5243ms 1.0905ms 917.0223 Ops/s 922.8242 Ops/s $\color{#d91a1a}-0.63\%$
test_values[td_lambda_return_estimate-True-False] 87.5307ms 86.9039ms 11.5070 Ops/s 11.4760 Ops/s $\color{#35bf28}+0.27\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2475ms 1.0794ms 926.4754 Ops/s 924.1566 Ops/s $\color{#35bf28}+0.25\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.9305ms 24.5761ms 40.6900 Ops/s 41.1428 Ops/s $\color{#d91a1a}-1.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9583ms 0.7231ms 1.3830 KOps/s 1.3426 KOps/s $\color{#35bf28}+3.01\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8154ms 0.6688ms 1.4951 KOps/s 1.4906 KOps/s $\color{#35bf28}+0.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6123ms 1.4698ms 680.3703 Ops/s 679.6469 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8391ms 0.6859ms 1.4580 KOps/s 1.4642 KOps/s $\color{#d91a1a}-0.42\%$
test_dqn_speed 7.3264ms 1.4767ms 677.1993 Ops/s 698.2179 Ops/s $\color{#d91a1a}-3.01\%$
test_ddpg_speed 3.3476ms 2.9997ms 333.3636 Ops/s 340.0717 Ops/s $\color{#d91a1a}-1.97\%$
test_sac_speed 0.1015s 9.2721ms 107.8505 Ops/s 120.0286 Ops/s $\textbf{\color{#d91a1a}-10.15\%}$
test_redq_speed 11.6812ms 10.9639ms 91.2081 Ops/s 91.9535 Ops/s $\color{#d91a1a}-0.81\%$
test_redq_deprec_speed 12.2096ms 11.6216ms 86.0469 Ops/s 88.1535 Ops/s $\color{#d91a1a}-2.39\%$
test_td3_speed 9.3279ms 8.4638ms 118.1502 Ops/s 120.5186 Ops/s $\color{#d91a1a}-1.97\%$
test_cql_speed 27.2289ms 26.4924ms 37.7466 Ops/s 34.9408 Ops/s $\textbf{\color{#35bf28}+8.03\%}$
test_a2c_speed 6.1245ms 5.8224ms 171.7515 Ops/s 180.0292 Ops/s $\color{#d91a1a}-4.60\%$
test_ppo_speed 6.3865ms 6.1717ms 162.0293 Ops/s 169.8192 Ops/s $\color{#d91a1a}-4.59\%$
test_reinforce_speed 4.9910ms 4.6583ms 214.6715 Ops/s 221.1294 Ops/s $\color{#d91a1a}-2.92\%$
test_iql_speed 20.8481ms 20.1504ms 49.6267 Ops/s 50.9632 Ops/s $\color{#d91a1a}-2.62\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7903ms 6.6030ms 151.4472 Ops/s 151.6862 Ops/s $\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1098s 0.5917ms 1.6900 KOps/s 1.9286 KOps/s $\textbf{\color{#d91a1a}-12.37\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7463ms 0.4937ms 2.0255 KOps/s 2.0306 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8894ms 6.5429ms 152.8376 Ops/s 154.1314 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5560ms 0.5111ms 1.9567 KOps/s 1.9692 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7203ms 0.4891ms 2.0447 KOps/s 2.0580 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1735ms 1.9712ms 507.3112 Ops/s 509.1050 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1128ms 1.8784ms 532.3766 Ops/s 531.3304 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9806ms 6.7382ms 148.4075 Ops/s 148.5027 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1872ms 0.6663ms 1.5008 KOps/s 1.5101 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8556ms 0.6439ms 1.5530 KOps/s 1.5701 KOps/s $\color{#d91a1a}-1.09\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8067ms 6.6229ms 150.9915 Ops/s 152.4648 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4676ms 0.5154ms 1.9402 KOps/s 1.9368 KOps/s $\color{#35bf28}+0.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6735ms 0.4944ms 2.0228 KOps/s 2.0284 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9669ms 6.5448ms 152.7927 Ops/s 153.2142 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7257ms 0.5114ms 1.9553 KOps/s 1.9635 KOps/s $\color{#d91a1a}-0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.1760ms 0.4962ms 2.0155 KOps/s 2.0191 KOps/s $\color{#d91a1a}-0.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9355ms 6.7830ms 147.4275 Ops/s 148.2753 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1720ms 0.6707ms 1.4911 KOps/s 1.4939 KOps/s $\color{#d91a1a}-0.19\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7888ms 0.6485ms 1.5419 KOps/s 1.5513 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1444s 7.9296ms 126.1103 Ops/s 126.5520 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 20.4348ms 16.0599ms 62.2670 Ops/s 63.1001 Ops/s $\color{#d91a1a}-1.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.4940ms 1.2770ms 783.0567 Ops/s 788.7407 Ops/s $\color{#d91a1a}-0.72\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1268s 7.6088ms 131.4272 Ops/s 131.8763 Ops/s $\color{#d91a1a}-0.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1362s 18.2717ms 54.7296 Ops/s 63.3803 Ops/s $\textbf{\color{#d91a1a}-13.65\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.5269ms 1.2734ms 785.2687 Ops/s 794.2537 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1252s 7.7271ms 129.4149 Ops/s 129.2417 Ops/s $\color{#35bf28}+0.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.5694ms 15.9860ms 62.5546 Ops/s 62.5929 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.3865ms 1.4086ms 709.9319 Ops/s 697.9760 Ops/s $\color{#35bf28}+1.71\%$

@vmoens vmoens merged commit dd70b78 into main Jul 25, 2024
50 of 54 checks passed
@vmoens vmoens deleted the make-marl-module-apparent branch July 25, 2024 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] SACLoss module does not allow stochastic modules (i.e. Dropout, etc.) due to vmap
2 participants
@vmoens @facebook-github-bot