Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Minor] Hide params in ddpg actor-critic #1716

Merged
merged 3 commits into from
Nov 27, 2023
Merged

[Minor] Hide params in ddpg actor-critic #1716

merged 3 commits into from
Nov 27, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 27, 2023 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 27, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1716

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit 6b161ff with merge base aedcf29 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 27, 2023
@matteobettini
Copy link
Contributor

We need the same hiding in Qmixer

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 27, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}14$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 65.6397ms 64.6155ms 15.4762 Ops/s 14.8222 Ops/s $\color{#35bf28}+4.41\%$
test_sync 34.8245ms 34.4911ms 28.9930 Ops/s 28.3920 Ops/s $\color{#35bf28}+2.12\%$
test_async 59.2353ms 32.1699ms 31.0850 Ops/s 31.0037 Ops/s $\color{#35bf28}+0.26\%$
test_simple 0.5122s 0.4496s 2.2242 Ops/s 2.2630 Ops/s $\color{#d91a1a}-1.72\%$
test_transformed 0.6667s 0.6050s 1.6530 Ops/s 1.6333 Ops/s $\color{#35bf28}+1.20\%$
test_serial 1.3932s 1.3341s 0.7495 Ops/s 0.7390 Ops/s $\color{#35bf28}+1.42\%$
test_parallel 1.4095s 1.3595s 0.7356 Ops/s 0.7337 Ops/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-True-True-True] 0.1918ms 22.3492μs 44.7444 KOps/s 43.3342 KOps/s $\color{#35bf28}+3.25\%$
test_step_mdp_speed[True-True-True-True-False] 52.4980μs 13.5386μs 73.8630 KOps/s 71.6513 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[True-True-True-False-True] 37.4210μs 13.8024μs 72.4512 KOps/s 71.5091 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-False-False] 43.9120μs 8.2002μs 121.9484 KOps/s 117.8886 KOps/s $\color{#35bf28}+3.44\%$
test_step_mdp_speed[True-True-False-True-True] 48.8410μs 23.8115μs 41.9964 KOps/s 40.8427 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-True-False-True-False] 53.1500μs 14.7903μs 67.6119 KOps/s 64.6378 KOps/s $\color{#35bf28}+4.60\%$
test_step_mdp_speed[True-True-False-False-True] 36.7390μs 14.8864μs 67.1754 KOps/s 65.2436 KOps/s $\color{#35bf28}+2.96\%$
test_step_mdp_speed[True-True-False-False-False] 65.5920μs 9.5382μs 104.8415 KOps/s 101.4289 KOps/s $\color{#35bf28}+3.36\%$
test_step_mdp_speed[True-False-True-True-True] 49.7830μs 25.3281μs 39.4818 KOps/s 38.6317 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[True-False-True-True-False] 51.5970μs 16.1438μs 61.9434 KOps/s 59.1784 KOps/s $\color{#35bf28}+4.67\%$
test_step_mdp_speed[True-False-True-False-True] 41.1070μs 15.1278μs 66.1033 KOps/s 65.0065 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-True-False-False] 46.0060μs 9.4062μs 106.3130 KOps/s 101.7144 KOps/s $\color{#35bf28}+4.52\%$
test_step_mdp_speed[True-False-False-True-True] 0.1070ms 26.3444μs 37.9587 KOps/s 36.6261 KOps/s $\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-False-False-True-False] 57.1270μs 17.1997μs 58.1405 KOps/s 55.5301 KOps/s $\color{#35bf28}+4.70\%$
test_step_mdp_speed[True-False-False-False-True] 64.5910μs 16.1043μs 62.0953 KOps/s 59.9136 KOps/s $\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-False-False-False-False] 50.7450μs 10.4790μs 95.4286 KOps/s 89.5374 KOps/s $\textbf{\color{#35bf28}+6.58\%}$
test_step_mdp_speed[False-True-True-True-True] 72.2350μs 25.3318μs 39.4761 KOps/s 38.3315 KOps/s $\color{#35bf28}+2.99\%$
test_step_mdp_speed[False-True-True-True-False] 56.6860μs 16.0810μs 62.1854 KOps/s 58.5864 KOps/s $\textbf{\color{#35bf28}+6.14\%}$
test_step_mdp_speed[False-True-True-False-True] 44.9540μs 17.3160μs 57.7500 KOps/s 55.9757 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[False-True-True-False-False] 47.7790μs 10.5368μs 94.9057 KOps/s 89.2700 KOps/s $\textbf{\color{#35bf28}+6.31\%}$
test_step_mdp_speed[False-True-False-True-True] 61.3140μs 26.2450μs 38.1025 KOps/s 36.7272 KOps/s $\color{#35bf28}+3.74\%$
test_step_mdp_speed[False-True-False-True-False] 93.5850μs 17.2856μs 57.8516 KOps/s 55.4561 KOps/s $\color{#35bf28}+4.32\%$
test_step_mdp_speed[False-True-False-False-True] 39.9540μs 18.4082μs 54.3236 KOps/s 52.8517 KOps/s $\color{#35bf28}+2.78\%$
test_step_mdp_speed[False-True-False-False-False] 54.2420μs 11.8139μs 84.6462 KOps/s 81.1058 KOps/s $\color{#35bf28}+4.37\%$
test_step_mdp_speed[False-False-True-True-True] 91.3610μs 27.8093μs 35.9592 KOps/s 34.7352 KOps/s $\color{#35bf28}+3.52\%$
test_step_mdp_speed[False-False-True-True-False] 40.4750μs 18.4503μs 54.1996 KOps/s 51.5895 KOps/s $\textbf{\color{#35bf28}+5.06\%}$
test_step_mdp_speed[False-False-True-False-True] 55.6940μs 18.5241μs 53.9838 KOps/s 52.2370 KOps/s $\color{#35bf28}+3.34\%$
test_step_mdp_speed[False-False-True-False-False] 32.0400μs 11.8226μs 84.5841 KOps/s 81.1627 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[False-False-False-True-True] 76.6340μs 28.6466μs 34.9081 KOps/s 33.7619 KOps/s $\color{#35bf28}+3.39\%$
test_step_mdp_speed[False-False-False-True-False] 42.5800μs 19.7311μs 50.6815 KOps/s 49.2205 KOps/s $\color{#35bf28}+2.97\%$
test_step_mdp_speed[False-False-False-False-True] 39.0930μs 19.4391μs 51.4426 KOps/s 49.7494 KOps/s $\color{#35bf28}+3.40\%$
test_step_mdp_speed[False-False-False-False-False] 38.4920μs 13.0306μs 76.7427 KOps/s 74.8437 KOps/s $\color{#35bf28}+2.54\%$
test_values[generalized_advantage_estimate-True-True] 13.1719ms 12.3622ms 80.8918 Ops/s 81.3676 Ops/s $\color{#d91a1a}-0.58\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.2312ms 26.9719ms 37.0757 Ops/s 37.4697 Ops/s $\color{#d91a1a}-1.05\%$
test_values[td0_return_estimate-False-False] 0.2559ms 0.1883ms 5.3105 KOps/s 5.4707 KOps/s $\color{#d91a1a}-2.93\%$
test_values[td1_return_estimate-False-False] 27.1651ms 26.3231ms 37.9894 Ops/s 38.0013 Ops/s $\color{#d91a1a}-0.03\%$
test_values[vec_td1_return_estimate-False-False] 35.6315ms 27.5947ms 36.2388 Ops/s 37.3024 Ops/s $\color{#d91a1a}-2.85\%$
test_values[td_lambda_return_estimate-True-False] 38.2319ms 37.5039ms 26.6639 Ops/s 26.9041 Ops/s $\color{#d91a1a}-0.89\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.3907ms 27.2733ms 36.6659 Ops/s 36.5752 Ops/s $\color{#35bf28}+0.25\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7679ms 8.2633ms 121.0175 Ops/s 116.4240 Ops/s $\color{#35bf28}+3.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.7876ms 2.0013ms 499.6762 Ops/s 526.3680 Ops/s $\textbf{\color{#d91a1a}-5.07\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7951ms 0.4422ms 2.2615 KOps/s 2.2589 KOps/s $\color{#35bf28}+0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.2425ms 38.4801ms 25.9875 Ops/s 24.0749 Ops/s $\textbf{\color{#35bf28}+7.94\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 11.1191ms 2.5039ms 399.3790 Ops/s 396.2741 Ops/s $\color{#35bf28}+0.78\%$
test_dqn_speed 10.1014ms 1.6520ms 605.3126 Ops/s 542.9460 Ops/s $\textbf{\color{#35bf28}+11.49\%}$
test_ddpg_speed 13.0733ms 3.8380ms 260.5514 Ops/s 248.5761 Ops/s $\color{#35bf28}+4.82\%$
test_sac_speed 26.2373ms 11.4474ms 87.3563 Ops/s 86.9267 Ops/s $\color{#35bf28}+0.49\%$
test_redq_speed 29.3420ms 20.2750ms 49.3218 Ops/s 40.2667 Ops/s $\textbf{\color{#35bf28}+22.49\%}$
test_redq_deprec_speed 94.0265ms 17.2183ms 58.0778 Ops/s 58.6443 Ops/s $\color{#d91a1a}-0.97\%$
test_td3_speed 11.7028ms 10.7077ms 93.3904 Ops/s 88.8315 Ops/s $\textbf{\color{#35bf28}+5.13\%}$
test_cql_speed 0.1182s 41.9248ms 23.8523 Ops/s 24.4157 Ops/s $\color{#d91a1a}-2.31\%$
test_a2c_speed 16.6839ms 8.4180ms 118.7931 Ops/s 105.6732 Ops/s $\textbf{\color{#35bf28}+12.42\%}$
test_ppo_speed 17.3546ms 8.8051ms 113.5706 Ops/s 104.0387 Ops/s $\textbf{\color{#35bf28}+9.16\%}$
test_reinforce_speed 16.2708ms 7.4333ms 134.5298 Ops/s 120.6291 Ops/s $\textbf{\color{#35bf28}+11.52\%}$
test_iql_speed 37.2464ms 34.9042ms 28.6499 Ops/s 27.4508 Ops/s $\color{#35bf28}+4.37\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8576ms 1.9932ms 501.7006 Ops/s 486.0672 Ops/s $\color{#35bf28}+3.22\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.9584ms 2.0760ms 481.6942 Ops/s 455.6110 Ops/s $\textbf{\color{#35bf28}+5.72\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.2432ms 2.0512ms 487.5305 Ops/s 460.8549 Ops/s $\textbf{\color{#35bf28}+5.79\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.7662ms 1.9931ms 501.7287 Ops/s 489.0943 Ops/s $\color{#35bf28}+2.58\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.1605ms 2.1234ms 470.9510 Ops/s 462.3380 Ops/s $\color{#35bf28}+1.86\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8204ms 2.1257ms 470.4238 Ops/s 461.0455 Ops/s $\color{#35bf28}+2.03\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1478ms 2.0242ms 494.0124 Ops/s 483.8207 Ops/s $\color{#35bf28}+2.11\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.5340ms 2.1449ms 466.2250 Ops/s 456.8087 Ops/s $\color{#35bf28}+2.06\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6863ms 2.1491ms 465.3119 Ops/s 462.6810 Ops/s $\color{#35bf28}+0.57\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.5337ms 2.0537ms 486.9363 Ops/s 492.6713 Ops/s $\color{#d91a1a}-1.16\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.7338ms 2.1255ms 470.4691 Ops/s 462.2172 Ops/s $\color{#35bf28}+1.79\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8101ms 2.1461ms 465.9563 Ops/s 455.9630 Ops/s $\color{#35bf28}+2.19\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.2404ms 1.9844ms 503.9406 Ops/s 483.5743 Ops/s $\color{#35bf28}+4.21\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.3932ms 2.1508ms 464.9347 Ops/s 464.3798 Ops/s $\color{#35bf28}+0.12\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.9893ms 2.1587ms 463.2500 Ops/s 470.8118 Ops/s $\color{#d91a1a}-1.61\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8603ms 2.0298ms 492.6707 Ops/s 491.6936 Ops/s $\color{#35bf28}+0.20\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.4228ms 2.1598ms 462.9971 Ops/s 465.5399 Ops/s $\color{#d91a1a}-0.55\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0497ms 2.1233ms 470.9729 Ops/s 461.5189 Ops/s $\color{#35bf28}+2.05\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2323s 20.1930ms 49.5221 Ops/s 52.2340 Ops/s $\textbf{\color{#d91a1a}-5.19\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1367s 18.3767ms 54.4167 Ops/s 56.4620 Ops/s $\color{#d91a1a}-3.62\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1324s 17.9085ms 55.8395 Ops/s 57.1209 Ops/s $\color{#d91a1a}-2.24\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1321s 17.8189ms 56.1201 Ops/s 57.1078 Ops/s $\color{#d91a1a}-1.73\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1381s 18.2905ms 54.6733 Ops/s 57.0102 Ops/s $\color{#d91a1a}-4.10\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1282s 17.7125ms 56.4574 Ops/s 56.8650 Ops/s $\color{#d91a1a}-0.72\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1261s 17.5571ms 56.9570 Ops/s 56.5852 Ops/s $\color{#35bf28}+0.66\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1347s 18.0856ms 55.2928 Ops/s 58.0648 Ops/s $\color{#d91a1a}-4.77\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1232s 17.4546ms 57.2916 Ops/s 50.3648 Ops/s $\textbf{\color{#35bf28}+13.75\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1217s 0.1212s 8.2504 Ops/s 8.2542 Ops/s $\color{#d91a1a}-0.05\%$
test_sync 0.1020s 0.1016s 9.8449 Ops/s 9.8195 Ops/s $\color{#35bf28}+0.26\%$
test_async 0.2725s 99.5398ms 10.0462 Ops/s 10.0235 Ops/s $\color{#35bf28}+0.23\%$
test_single_pixels 0.1452s 0.1449s 6.9031 Ops/s 7.5886 Ops/s $\textbf{\color{#d91a1a}-9.03\%}$
test_sync_pixels 98.1113ms 94.5752ms 10.5736 Ops/s 10.3000 Ops/s $\color{#35bf28}+2.66\%$
test_async_pixels 0.2472s 92.4126ms 10.8210 Ops/s 10.9324 Ops/s $\color{#d91a1a}-1.02\%$
test_simple 0.9637s 0.8876s 1.1266 Ops/s 1.1355 Ops/s $\color{#d91a1a}-0.78\%$
test_transformed 1.2124s 1.1382s 0.8786 Ops/s 0.8825 Ops/s $\color{#d91a1a}-0.44\%$
test_serial 2.5286s 2.4688s 0.4051 Ops/s 0.4101 Ops/s $\color{#d91a1a}-1.24\%$
test_parallel 2.5952s 2.5214s 0.3966 Ops/s 0.3916 Ops/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-True-True-True-True] 96.3810μs 35.2966μs 28.3313 KOps/s 29.0417 KOps/s $\color{#d91a1a}-2.45\%$
test_step_mdp_speed[True-True-True-True-False] 38.1700μs 20.7823μs 48.1178 KOps/s 47.9135 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[True-True-True-False-True] 41.5900μs 20.2044μs 49.4941 KOps/s 48.8214 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-True-True-False-False] 29.6200μs 12.0992μs 82.6503 KOps/s 82.1407 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-False-True-True] 65.1500μs 36.2272μs 27.6036 KOps/s 27.8881 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-False-True-False] 41.0900μs 22.3640μs 44.7148 KOps/s 44.3235 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[True-True-False-False-True] 56.1910μs 22.1237μs 45.2003 KOps/s 44.2429 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-False-False-False] 39.2200μs 13.9861μs 71.4996 KOps/s 71.0565 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-False-True-True-True] 69.4710μs 38.7153μs 25.8296 KOps/s 26.1877 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-False-True-True-False] 46.0000μs 24.1466μs 41.4137 KOps/s 40.8280 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-True-False-True] 55.7910μs 22.0931μs 45.2629 KOps/s 44.7623 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-False-True-False-False] 35.1910μs 14.0708μs 71.0690 KOps/s 71.1810 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-False-False-True-True] 69.6210μs 39.5427μs 25.2891 KOps/s 24.6418 KOps/s $\color{#35bf28}+2.63\%$
test_step_mdp_speed[True-False-False-True-False] 42.8200μs 26.0440μs 38.3966 KOps/s 37.6966 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[True-False-False-False-True] 61.2100μs 23.9589μs 41.7382 KOps/s 41.4335 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-False-False-False] 50.5900μs 15.9009μs 62.8894 KOps/s 61.9572 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-True-True-True] 65.9900μs 38.0351μs 26.2915 KOps/s 25.6100 KOps/s $\color{#35bf28}+2.66\%$
test_step_mdp_speed[False-True-True-True-False] 46.6810μs 24.1038μs 41.4872 KOps/s 41.1560 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-True-True-False-True] 47.5000μs 26.4603μs 37.7924 KOps/s 37.4575 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-True-True-False-False] 45.0110μs 15.9564μs 62.6708 KOps/s 63.1066 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-False-True-True] 70.3410μs 40.2949μs 24.8170 KOps/s 24.4440 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-True-False-True-False] 64.4010μs 26.4655μs 37.7850 KOps/s 37.5817 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-False-False-True] 57.3500μs 28.3826μs 35.2329 KOps/s 34.7879 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-True-False-False-False] 42.5900μs 18.0095μs 55.5264 KOps/s 55.3042 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[False-False-True-True-True] 72.1810μs 41.6801μs 23.9923 KOps/s 23.7774 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-False-True-True-False] 69.9410μs 28.3748μs 35.2426 KOps/s 34.9852 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-False-True-False-True] 48.3900μs 28.3611μs 35.2595 KOps/s 34.9105 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-False-True-False-False] 47.7200μs 17.7915μs 56.2067 KOps/s 55.7731 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-False-False-True-True] 61.6510μs 43.3816μs 23.0512 KOps/s 22.7017 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[False-False-False-True-False] 51.5300μs 29.9450μs 33.3946 KOps/s 33.1392 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-False-False-False-True] 53.0100μs 30.8207μs 32.4458 KOps/s 33.3719 KOps/s $\color{#d91a1a}-2.78\%$
test_step_mdp_speed[False-False-False-False-False] 57.4410μs 19.5376μs 51.1835 KOps/s 51.3078 KOps/s $\color{#d91a1a}-0.24\%$
test_values[generalized_advantage_estimate-True-True] 26.1124ms 25.6289ms 39.0185 Ops/s 38.9100 Ops/s $\color{#35bf28}+0.28\%$
test_values[vec_generalized_advantage_estimate-True-True] 86.3606ms 3.2875ms 304.1824 Ops/s 290.1445 Ops/s $\color{#35bf28}+4.84\%$
test_values[td0_return_estimate-False-False] 98.9910μs 65.0309μs 15.3773 KOps/s 14.8547 KOps/s $\color{#35bf28}+3.52\%$
test_values[td1_return_estimate-False-False] 57.1026ms 55.0605ms 18.1619 Ops/s 18.0001 Ops/s $\color{#35bf28}+0.90\%$
test_values[vec_td1_return_estimate-False-False] 1.9730ms 1.7268ms 579.1031 Ops/s 576.4614 Ops/s $\color{#35bf28}+0.46\%$
test_values[td_lambda_return_estimate-True-False] 90.4509ms 87.5510ms 11.4219 Ops/s 11.2437 Ops/s $\color{#35bf28}+1.58\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.9988ms 1.7234ms 580.2366 Ops/s 578.1556 Ops/s $\color{#35bf28}+0.36\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.7876ms 25.5351ms 39.1617 Ops/s 40.6146 Ops/s $\color{#d91a1a}-3.58\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8984ms 0.7214ms 1.3862 KOps/s 1.3732 KOps/s $\color{#35bf28}+0.94\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7419ms 0.6803ms 1.4699 KOps/s 1.4583 KOps/s $\color{#35bf28}+0.79\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5284ms 1.4758ms 677.6032 Ops/s 675.1790 Ops/s $\color{#35bf28}+0.36\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9598ms 0.7092ms 1.4101 KOps/s 1.4051 KOps/s $\color{#35bf28}+0.36\%$
test_dqn_speed 8.0062ms 1.4493ms 689.9668 Ops/s 681.2444 Ops/s $\color{#35bf28}+1.28\%$
test_ddpg_speed 4.7333ms 3.2432ms 308.3391 Ops/s 279.0632 Ops/s $\textbf{\color{#35bf28}+10.49\%}$
test_sac_speed 10.2422ms 9.0973ms 109.9229 Ops/s 108.6031 Ops/s $\color{#35bf28}+1.22\%$
test_redq_speed 16.8574ms 16.4593ms 60.7558 Ops/s 60.3197 Ops/s $\color{#35bf28}+0.72\%$
test_redq_deprec_speed 13.8590ms 12.7091ms 78.6835 Ops/s 78.1097 Ops/s $\color{#35bf28}+0.73\%$
test_td3_speed 18.3739ms 9.3116ms 107.3924 Ops/s 106.8046 Ops/s $\color{#35bf28}+0.55\%$
test_cql_speed 32.4305ms 30.8733ms 32.3904 Ops/s 32.2356 Ops/s $\color{#35bf28}+0.48\%$
test_a2c_speed 8.3225ms 7.0211ms 142.4279 Ops/s 142.0143 Ops/s $\color{#35bf28}+0.29\%$
test_ppo_speed 8.6579ms 7.2967ms 137.0479 Ops/s 135.2497 Ops/s $\color{#35bf28}+1.33\%$
test_reinforce_speed 7.3618ms 5.9734ms 167.4101 Ops/s 164.1052 Ops/s $\color{#35bf28}+2.01\%$
test_iql_speed 29.1194ms 26.9658ms 37.0841 Ops/s 37.3305 Ops/s $\color{#d91a1a}-0.66\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9917ms 2.5128ms 397.9683 Ops/s 396.7482 Ops/s $\color{#35bf28}+0.31\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.6983ms 2.6769ms 373.5630 Ops/s 372.0878 Ops/s $\color{#35bf28}+0.40\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.1208s 2.9862ms 334.8714 Ops/s 373.2018 Ops/s $\textbf{\color{#d91a1a}-10.27\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9443ms 2.4596ms 406.5646 Ops/s 398.4561 Ops/s $\color{#35bf28}+2.03\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.4338ms 2.7038ms 369.8489 Ops/s 372.3547 Ops/s $\color{#d91a1a}-0.67\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1204s 3.0345ms 329.5394 Ops/s 373.8462 Ops/s $\textbf{\color{#d91a1a}-11.85\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0575ms 2.5104ms 398.3460 Ops/s 397.1998 Ops/s $\color{#35bf28}+0.29\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.7071ms 2.6924ms 371.4134 Ops/s 372.6456 Ops/s $\color{#d91a1a}-0.33\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0941ms 2.6615ms 375.7302 Ops/s 375.4501 Ops/s $\color{#35bf28}+0.07\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1486ms 2.5103ms 398.3604 Ops/s 398.2365 Ops/s $\color{#35bf28}+0.03\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.7449ms 2.6724ms 374.1973 Ops/s 373.1577 Ops/s $\color{#35bf28}+0.28\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.1858ms 2.6898ms 371.7800 Ops/s 373.3823 Ops/s $\color{#d91a1a}-0.43\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.7993ms 2.5174ms 397.2302 Ops/s 398.1468 Ops/s $\color{#d91a1a}-0.23\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.0984ms 2.6782ms 373.3834 Ops/s 371.3972 Ops/s $\color{#35bf28}+0.53\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.8497ms 2.6851ms 372.4195 Ops/s 371.4098 Ops/s $\color{#35bf28}+0.27\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7196ms 2.5142ms 397.7374 Ops/s 398.0470 Ops/s $\color{#d91a1a}-0.08\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.6568ms 2.6806ms 373.0574 Ops/s 371.9278 Ops/s $\color{#35bf28}+0.30\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.2345ms 2.6806ms 373.0536 Ops/s 372.7909 Ops/s $\color{#35bf28}+0.07\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1934s 19.1709ms 52.1625 Ops/s 51.5024 Ops/s $\color{#35bf28}+1.28\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1276s 17.7907ms 56.2092 Ops/s 56.5633 Ops/s $\color{#d91a1a}-0.63\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1288s 17.8705ms 55.9582 Ops/s 56.2893 Ops/s $\color{#d91a1a}-0.59\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1271s 17.7324ms 56.3941 Ops/s 65.1139 Ops/s $\textbf{\color{#d91a1a}-13.39\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1270s 15.4439ms 64.7506 Ops/s 56.7199 Ops/s $\textbf{\color{#35bf28}+14.16\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1275s 17.8155ms 56.1309 Ops/s 56.3916 Ops/s $\color{#d91a1a}-0.46\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1293s 17.8126ms 56.1400 Ops/s 56.2827 Ops/s $\color{#d91a1a}-0.25\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1277s 17.8386ms 56.0581 Ops/s 56.5464 Ops/s $\color{#d91a1a}-0.86\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1269s 15.4876ms 64.5676 Ops/s 64.9539 Ops/s $\color{#d91a1a}-0.59\%$

@vmoens vmoens added the bug Something isn't working label Nov 27, 2023
@vmoens
Copy link
Contributor Author

vmoens commented Nov 27, 2023

@matteobettini I think it's all good now

@vmoens vmoens marked this pull request as ready for review November 27, 2023 15:11
@vmoens vmoens merged commit 07fcfb1 into main Nov 27, 2023
57 of 61 checks passed
@vmoens vmoens deleted the fix_ddpg_actor_critic branch November 27, 2023 15:12
@albertbou92 albertbou92 mentioned this pull request Nov 28, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot