Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] serial_for_single arg in batched envs #1846

Merged
merged 2 commits into from
Jan 29, 2024
Merged

[Feature] serial_for_single arg in batched envs #1846

merged 2 commits into from
Jan 29, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 29, 2024

Makes it possible to avoid creating purposeless parallel envs when only one env is executed.

cc @albertbou92 @BY571 since we have a lot of these in the examples

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 29, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1846

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 2 Unrelated Failures

As of commit b4de825 with merge base 9da61f2 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 29, 2024
@vmoens vmoens added the enhancement New feature or request label Jan 29, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.5269ms 62.7518ms 15.9358 Ops/s 15.8975 Ops/s $\color{#35bf28}+0.24\%$
test_sync 40.9249ms 35.0968ms 28.4926 Ops/s 29.1570 Ops/s $\color{#d91a1a}-2.28\%$
test_async 80.5452ms 32.9473ms 30.3515 Ops/s 30.7391 Ops/s $\color{#d91a1a}-1.26\%$
test_simple 0.5080s 0.4421s 2.2617 Ops/s 2.2463 Ops/s $\color{#35bf28}+0.69\%$
test_transformed 0.6631s 0.6065s 1.6488 Ops/s 1.6279 Ops/s $\color{#35bf28}+1.28\%$
test_serial 1.4384s 1.3746s 0.7275 Ops/s 0.7000 Ops/s $\color{#35bf28}+3.94\%$
test_parallel 1.2830s 1.2201s 0.8196 Ops/s 0.8126 Ops/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-True-True-True] 0.1564ms 21.3585μs 46.8197 KOps/s 46.8423 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-True-True-True-False] 42.2390μs 13.2491μs 75.4768 KOps/s 76.9355 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-True-True-False-True] 42.0480μs 12.6090μs 79.3085 KOps/s 80.8698 KOps/s $\color{#d91a1a}-1.93\%$
test_step_mdp_speed[True-True-True-False-False] 50.7350μs 7.6961μs 129.9355 KOps/s 132.5014 KOps/s $\color{#d91a1a}-1.94\%$
test_step_mdp_speed[True-True-False-True-True] 57.3170μs 22.9924μs 43.4927 KOps/s 44.6018 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[True-True-False-True-False] 47.8700μs 14.5886μs 68.5467 KOps/s 70.3929 KOps/s $\color{#d91a1a}-2.62\%$
test_step_mdp_speed[True-True-False-False-True] 43.4120μs 13.8515μs 72.1943 KOps/s 73.9895 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-True-False-False-False] 47.2880μs 8.9052μs 112.2936 KOps/s 113.9435 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-True-True-True] 53.2590μs 24.2491μs 41.2386 KOps/s 42.4264 KOps/s $\color{#d91a1a}-2.80\%$
test_step_mdp_speed[True-False-True-True-False] 57.3370μs 15.8013μs 63.2861 KOps/s 64.4467 KOps/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-False-True-False-True] 69.3990μs 13.7787μs 72.5758 KOps/s 73.6458 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-True-False-False] 31.9800μs 8.9760μs 111.4085 KOps/s 113.8193 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[True-False-False-True-True] 92.9850μs 25.1276μs 39.7969 KOps/s 39.9613 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-False-False-True-False] 46.3860μs 16.8537μs 59.3340 KOps/s 59.4909 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-False-False-False-True] 40.8660μs 14.8856μs 67.1789 KOps/s 68.3840 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-False-False-False-False] 38.6320μs 10.1178μs 98.8356 KOps/s 100.5424 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[False-True-True-True-True] 50.9650μs 23.8978μs 41.8449 KOps/s 41.9744 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[False-True-True-True-False] 42.1790μs 15.7729μs 63.4000 KOps/s 64.4457 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[False-True-True-False-True] 47.7190μs 15.9804μs 62.5765 KOps/s 62.8575 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-True-False-False] 39.5140μs 10.2162μs 97.8838 KOps/s 101.4835 KOps/s $\color{#d91a1a}-3.55\%$
test_step_mdp_speed[False-True-False-True-True] 62.8670μs 25.0984μs 39.8432 KOps/s 40.3423 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-True-False-True-False] 53.5300μs 16.4681μs 60.7234 KOps/s 59.8342 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-True-False-False-True] 54.2910μs 17.0471μs 58.6609 KOps/s 58.8982 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-False-False] 45.6760μs 11.2877μs 88.5922 KOps/s 89.4185 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-False-True-True-True] 67.3550μs 26.5143μs 37.7155 KOps/s 38.2664 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-False-True-True-False] 53.0590μs 18.2150μs 54.8999 KOps/s 55.8420 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[False-False-True-False-True] 51.3050μs 17.1903μs 58.1725 KOps/s 59.4382 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[False-False-True-False-False] 38.3210μs 11.3793μs 87.8791 KOps/s 90.1818 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-False-False-True-True] 0.1475ms 28.0370μs 35.6671 KOps/s 36.7051 KOps/s $\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-False-False-True-False] 0.1316ms 19.7060μs 50.7459 KOps/s 52.1943 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-False-False-False-True] 46.2060μs 18.0645μs 55.3572 KOps/s 55.6918 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-False-False-False-False] 37.2300μs 12.4273μs 80.4679 KOps/s 81.8316 KOps/s $\color{#d91a1a}-1.67\%$
test_values[generalized_advantage_estimate-True-True] 20.0411ms 12.4160ms 80.5414 Ops/s 81.3389 Ops/s $\color{#d91a1a}-0.98\%$
test_values[vec_generalized_advantage_estimate-True-True] 64.7822ms 29.5420ms 33.8501 Ops/s 35.5813 Ops/s $\color{#d91a1a}-4.87\%$
test_values[td0_return_estimate-False-False] 0.2416ms 0.1789ms 5.5886 KOps/s 5.2726 KOps/s $\textbf{\color{#35bf28}+5.99\%}$
test_values[td1_return_estimate-False-False] 30.0216ms 27.0646ms 36.9486 Ops/s 37.2666 Ops/s $\color{#d91a1a}-0.85\%$
test_values[vec_td1_return_estimate-False-False] 35.8686ms 28.1081ms 35.5769 Ops/s 35.1499 Ops/s $\color{#35bf28}+1.21\%$
test_values[td_lambda_return_estimate-True-False] 40.5004ms 37.6466ms 26.5629 Ops/s 26.6064 Ops/s $\color{#d91a1a}-0.16\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.2531ms 27.9695ms 35.7532 Ops/s 35.5790 Ops/s $\color{#35bf28}+0.49\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7515ms 8.2896ms 120.6325 Ops/s 123.4224 Ops/s $\color{#d91a1a}-2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.3066ms 2.0431ms 489.4598 Ops/s 543.6249 Ops/s $\textbf{\color{#d91a1a}-9.96\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.6781ms 0.4509ms 2.2180 KOps/s 2.2577 KOps/s $\color{#d91a1a}-1.76\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.8353ms 41.6684ms 23.9990 Ops/s 23.4687 Ops/s $\color{#35bf28}+2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1541ms 2.6439ms 378.2344 Ops/s 374.6043 Ops/s $\color{#35bf28}+0.97\%$
test_dqn_speed 82.7800ms 8.2908ms 120.6152 Ops/s 130.1886 Ops/s $\textbf{\color{#d91a1a}-7.35\%}$
test_ddpg_speed 15.8519ms 14.7184ms 67.9422 Ops/s 63.1327 Ops/s $\textbf{\color{#35bf28}+7.62\%}$
test_sac_speed 36.3999ms 29.8380ms 33.5143 Ops/s 33.9066 Ops/s $\color{#d91a1a}-1.16\%$
test_redq_speed 24.2398ms 15.7346ms 63.5542 Ops/s 63.2474 Ops/s $\color{#35bf28}+0.49\%$
test_redq_deprec_speed 34.4870ms 26.2045ms 38.1614 Ops/s 38.3143 Ops/s $\color{#d91a1a}-0.40\%$
test_td3_speed 29.8561ms 20.6853ms 48.3434 Ops/s 48.4702 Ops/s $\color{#d91a1a}-0.26\%$
test_cql_speed 99.5474ms 90.7169ms 11.0233 Ops/s 11.2798 Ops/s $\color{#d91a1a}-2.27\%$
test_a2c_speed 37.9514ms 28.2687ms 35.3749 Ops/s 36.6575 Ops/s $\color{#d91a1a}-3.50\%$
test_ppo_speed 30.4181ms 27.9545ms 35.7725 Ops/s 36.8905 Ops/s $\color{#d91a1a}-3.03\%$
test_reinforce_speed 27.9730ms 26.7913ms 37.3256 Ops/s 38.3195 Ops/s $\color{#d91a1a}-2.59\%$
test_iql_speed 68.5414ms 64.7325ms 15.4482 Ops/s 15.4830 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.1482ms 2.9686ms 336.8541 Ops/s 357.4130 Ops/s $\textbf{\color{#d91a1a}-5.75\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 9.6568ms 0.5447ms 1.8357 KOps/s 1.8894 KOps/s $\color{#d91a1a}-2.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 8.8585ms 0.5062ms 1.9755 KOps/s 1.9913 KOps/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4860ms 2.9072ms 343.9757 Ops/s 346.3786 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 9.0688ms 0.5332ms 1.8753 KOps/s 1.8966 KOps/s $\color{#d91a1a}-1.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.6019ms 0.4983ms 2.0067 KOps/s 2.0179 KOps/s $\color{#d91a1a}-0.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9029ms 2.6694ms 374.6115 Ops/s 391.8058 Ops/s $\color{#d91a1a}-4.39\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 9.0742ms 0.6744ms 1.4828 KOps/s 1.4935 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.2378ms 0.6466ms 1.5466 KOps/s 1.5695 KOps/s $\color{#d91a1a}-1.46\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9915ms 2.7785ms 359.9108 Ops/s 357.8612 Ops/s $\color{#35bf28}+0.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.8177ms 0.5428ms 1.8423 KOps/s 1.9057 KOps/s $\color{#d91a1a}-3.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6335ms 0.4986ms 2.0057 KOps/s 2.0110 KOps/s $\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.2331ms 2.9204ms 342.4192 Ops/s 346.4341 Ops/s $\color{#d91a1a}-1.16\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.7859ms 0.5340ms 1.8728 KOps/s 1.9027 KOps/s $\color{#d91a1a}-1.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.8068ms 0.4953ms 2.0192 KOps/s 1.9744 KOps/s $\color{#35bf28}+2.27\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0634ms 2.6677ms 374.8597 Ops/s 387.7198 Ops/s $\color{#d91a1a}-3.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 9.1586ms 0.6768ms 1.4775 KOps/s 1.5186 KOps/s $\color{#d91a1a}-2.71\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7754ms 0.6369ms 1.5702 KOps/s 1.5552 KOps/s $\color{#35bf28}+0.97\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1112s 10.4088ms 96.0728 Ops/s 80.7143 Ops/s $\textbf{\color{#35bf28}+19.03\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.9057ms 13.7966ms 72.4817 Ops/s 73.8365 Ops/s $\color{#d91a1a}-1.83\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 11.6783ms 3.5259ms 283.6193 Ops/s 302.6958 Ops/s $\textbf{\color{#d91a1a}-6.30\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1110s 12.5392ms 79.7496 Ops/s 79.9680 Ops/s $\color{#d91a1a}-0.27\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.0348ms 13.7502ms 72.7260 Ops/s 73.6801 Ops/s $\color{#d91a1a}-1.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.6348ms 3.3568ms 297.9000 Ops/s 300.9624 Ops/s $\color{#d91a1a}-1.02\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1071s 10.5197ms 95.0596 Ops/s 95.3631 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.4624ms 14.0874ms 70.9854 Ops/s 72.1034 Ops/s $\color{#d91a1a}-1.55\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 11.8996ms 3.7359ms 267.6706 Ops/s 282.0505 Ops/s $\textbf{\color{#d91a1a}-5.10\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}20$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1176s 0.1172s 8.5300 Ops/s 8.9594 Ops/s $\color{#d91a1a}-4.79\%$
test_sync 96.0554ms 95.6793ms 10.4516 Ops/s 10.4177 Ops/s $\color{#35bf28}+0.32\%$
test_async 0.2564s 92.3694ms 10.8261 Ops/s 10.8688 Ops/s $\color{#d91a1a}-0.39\%$
test_single_pixels 0.1405s 0.1400s 7.1420 Ops/s 7.3393 Ops/s $\color{#d91a1a}-2.69\%$
test_sync_pixels 76.5849ms 75.0009ms 13.3332 Ops/s 12.0069 Ops/s $\textbf{\color{#35bf28}+11.05\%}$
test_async_pixels 0.1376s 71.3398ms 14.0174 Ops/s 14.2340 Ops/s $\color{#d91a1a}-1.52\%$
test_simple 0.9142s 0.8429s 1.1864 Ops/s 1.2583 Ops/s $\textbf{\color{#d91a1a}-5.72\%}$
test_transformed 1.1687s 1.0996s 0.9094 Ops/s 0.9517 Ops/s $\color{#d91a1a}-4.45\%$
test_serial 2.4077s 2.3382s 0.4277 Ops/s 0.4457 Ops/s $\color{#d91a1a}-4.05\%$
test_parallel 1.9890s 1.8844s 0.5307 Ops/s 0.5390 Ops/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[True-True-True-True-True] 0.1014ms 32.7827μs 30.5039 KOps/s 30.8179 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-True-True-False] 42.4010μs 19.8288μs 50.4317 KOps/s 52.5338 KOps/s $\color{#d91a1a}-4.00\%$
test_step_mdp_speed[True-True-True-False-True] 39.6610μs 19.2400μs 51.9751 KOps/s 55.2657 KOps/s $\textbf{\color{#d91a1a}-5.95\%}$
test_step_mdp_speed[True-True-True-False-False] 37.7810μs 11.4488μs 87.3457 KOps/s 91.1187 KOps/s $\color{#d91a1a}-4.14\%$
test_step_mdp_speed[True-True-False-True-True] 98.5820μs 34.6432μs 28.8657 KOps/s 29.8528 KOps/s $\color{#d91a1a}-3.31\%$
test_step_mdp_speed[True-True-False-True-False] 43.3310μs 21.7206μs 46.0392 KOps/s 48.0352 KOps/s $\color{#d91a1a}-4.16\%$
test_step_mdp_speed[True-True-False-False-True] 85.3010μs 21.0074μs 47.6023 KOps/s 50.2109 KOps/s $\textbf{\color{#d91a1a}-5.20\%}$
test_step_mdp_speed[True-True-False-False-False] 32.3100μs 13.3385μs 74.9707 KOps/s 79.0275 KOps/s $\textbf{\color{#d91a1a}-5.13\%}$
test_step_mdp_speed[True-False-True-True-True] 62.5600μs 36.8072μs 27.1686 KOps/s 28.3559 KOps/s $\color{#d91a1a}-4.19\%$
test_step_mdp_speed[True-False-True-True-False] 42.7510μs 23.6983μs 42.1971 KOps/s 44.0393 KOps/s $\color{#d91a1a}-4.18\%$
test_step_mdp_speed[True-False-True-False-True] 42.7410μs 20.7928μs 48.0936 KOps/s 50.1747 KOps/s $\color{#d91a1a}-4.15\%$
test_step_mdp_speed[True-False-True-False-False] 78.6410μs 13.3389μs 74.9687 KOps/s 79.1272 KOps/s $\textbf{\color{#d91a1a}-5.26\%}$
test_step_mdp_speed[True-False-False-True-True] 61.4710μs 38.3604μs 26.0686 KOps/s 26.8532 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[True-False-False-True-False] 50.7010μs 25.4978μs 39.2191 KOps/s 40.4729 KOps/s $\color{#d91a1a}-3.10\%$
test_step_mdp_speed[True-False-False-False-True] 39.1010μs 22.4646μs 44.5144 KOps/s 46.1422 KOps/s $\color{#d91a1a}-3.53\%$
test_step_mdp_speed[True-False-False-False-False] 38.3410μs 15.2227μs 65.6916 KOps/s 70.0821 KOps/s $\textbf{\color{#d91a1a}-6.26\%}$
test_step_mdp_speed[False-True-True-True-True] 0.1124ms 36.6175μs 27.3094 KOps/s 28.2482 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[False-True-True-True-False] 47.7120μs 23.5087μs 42.5374 KOps/s 43.7927 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-True-True-False-True] 48.0810μs 25.0038μs 39.9939 KOps/s 41.4513 KOps/s $\color{#d91a1a}-3.52\%$
test_step_mdp_speed[False-True-True-False-False] 43.0300μs 15.5787μs 64.1901 KOps/s 68.9407 KOps/s $\textbf{\color{#d91a1a}-6.89\%}$
test_step_mdp_speed[False-True-False-True-True] 66.9420μs 38.8918μs 25.7124 KOps/s 26.6267 KOps/s $\color{#d91a1a}-3.43\%$
test_step_mdp_speed[False-True-False-True-False] 45.7800μs 25.3155μs 39.5014 KOps/s 40.6279 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-True-False-False-True] 78.8120μs 26.5582μs 37.6532 KOps/s 39.5592 KOps/s $\color{#d91a1a}-4.82\%$
test_step_mdp_speed[False-True-False-False-False] 37.7010μs 17.0308μs 58.7172 KOps/s 61.5330 KOps/s $\color{#d91a1a}-4.58\%$
test_step_mdp_speed[False-False-True-True-True] 66.4120μs 40.4747μs 24.7068 KOps/s 25.4999 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[False-False-True-True-False] 50.3210μs 27.5116μs 36.3483 KOps/s 37.6643 KOps/s $\color{#d91a1a}-3.49\%$
test_step_mdp_speed[False-False-True-False-True] 48.1800μs 26.7567μs 37.3738 KOps/s 39.5046 KOps/s $\textbf{\color{#d91a1a}-5.39\%}$
test_step_mdp_speed[False-False-True-False-False] 90.2420μs 17.0641μs 58.6024 KOps/s 61.3889 KOps/s $\color{#d91a1a}-4.54\%$
test_step_mdp_speed[False-False-False-True-True] 65.2110μs 41.5288μs 24.0797 KOps/s 24.6063 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-False-True-False] 56.6310μs 29.2394μs 34.2005 KOps/s 35.1394 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[False-False-False-False-True] 47.6810μs 28.2362μs 35.4156 KOps/s 37.5386 KOps/s $\textbf{\color{#d91a1a}-5.66\%}$
test_step_mdp_speed[False-False-False-False-False] 39.5210μs 18.9778μs 52.6931 KOps/s 55.8760 KOps/s $\textbf{\color{#d91a1a}-5.70\%}$
test_values[generalized_advantage_estimate-True-True] 25.4077ms 24.8428ms 40.2531 Ops/s 42.0023 Ops/s $\color{#d91a1a}-4.16\%$
test_values[vec_generalized_advantage_estimate-True-True] 86.1467ms 3.2886ms 304.0807 Ops/s 299.2936 Ops/s $\color{#35bf28}+1.60\%$
test_values[td0_return_estimate-False-False] 91.9320μs 61.5059μs 16.2586 KOps/s 17.0817 KOps/s $\color{#d91a1a}-4.82\%$
test_values[td1_return_estimate-False-False] 55.4585ms 53.4668ms 18.7032 Ops/s 19.4626 Ops/s $\color{#d91a1a}-3.90\%$
test_values[vec_td1_return_estimate-False-False] 1.9939ms 1.7585ms 568.6795 Ops/s 572.5841 Ops/s $\color{#d91a1a}-0.68\%$
test_values[td_lambda_return_estimate-True-False] 88.4812ms 85.0215ms 11.7617 Ops/s 12.0657 Ops/s $\color{#d91a1a}-2.52\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.1483ms 1.8256ms 547.7704 Ops/s 560.2441 Ops/s $\color{#d91a1a}-2.23\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.1491ms 23.4377ms 42.6663 Ops/s 43.5434 Ops/s $\color{#d91a1a}-2.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8911ms 0.7005ms 1.4276 KOps/s 1.4762 KOps/s $\color{#d91a1a}-3.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7112ms 0.6705ms 1.4914 KOps/s 1.5391 KOps/s $\color{#d91a1a}-3.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5206ms 1.4595ms 685.1567 Ops/s 697.4362 Ops/s $\color{#d91a1a}-1.76\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9647ms 0.6701ms 1.4922 KOps/s 1.5258 KOps/s $\color{#d91a1a}-2.20\%$
test_dqn_speed 14.7826ms 7.4940ms 133.4396 Ops/s 144.2672 Ops/s $\textbf{\color{#d91a1a}-7.51\%}$
test_ddpg_speed 15.6478ms 14.6309ms 68.3486 Ops/s 73.9509 Ops/s $\textbf{\color{#d91a1a}-7.58\%}$
test_sac_speed 30.7518ms 29.6760ms 33.6973 Ops/s 33.4416 Ops/s $\color{#35bf28}+0.76\%$
test_redq_speed 14.5513ms 13.5221ms 73.9530 Ops/s 76.2330 Ops/s $\color{#d91a1a}-2.99\%$
test_redq_deprec_speed 25.5355ms 24.3101ms 41.1352 Ops/s 43.1036 Ops/s $\color{#d91a1a}-4.57\%$
test_td3_speed 30.1918ms 19.9100ms 50.2259 Ops/s 53.5896 Ops/s $\textbf{\color{#d91a1a}-6.28\%}$
test_cql_speed 85.7814ms 84.1946ms 11.8772 Ops/s 12.5907 Ops/s $\textbf{\color{#d91a1a}-5.67\%}$
test_a2c_speed 28.8514ms 27.4667ms 36.4077 Ops/s 38.7862 Ops/s $\textbf{\color{#d91a1a}-6.13\%}$
test_ppo_speed 28.3354ms 27.2428ms 36.7069 Ops/s 38.9032 Ops/s $\textbf{\color{#d91a1a}-5.65\%}$
test_reinforce_speed 28.2917ms 26.5034ms 37.7311 Ops/s 40.1435 Ops/s $\textbf{\color{#d91a1a}-6.01\%}$
test_iql_speed 59.1849ms 57.8508ms 17.2859 Ops/s 18.0573 Ops/s $\color{#d91a1a}-4.27\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7830ms 3.6812ms 271.6536 Ops/s 281.0156 Ops/s $\color{#d91a1a}-3.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9575ms 0.8479ms 1.1793 KOps/s 1.1888 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9348ms 0.8231ms 1.2149 KOps/s 1.2228 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9650ms 3.7043ms 269.9597 Ops/s 279.3758 Ops/s $\color{#d91a1a}-3.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9808ms 0.8349ms 1.1977 KOps/s 1.2092 KOps/s $\color{#d91a1a}-0.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9348ms 0.8107ms 1.2335 KOps/s 1.2414 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4627ms 3.3773ms 296.0936 Ops/s 302.9201 Ops/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0787ms 0.9669ms 1.0342 KOps/s 1.0505 KOps/s $\color{#d91a1a}-1.55\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0548ms 0.9430ms 1.0605 KOps/s 1.0769 KOps/s $\color{#d91a1a}-1.52\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7626ms 3.6615ms 273.1144 Ops/s 282.8066 Ops/s $\color{#d91a1a}-3.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9836ms 0.8488ms 1.1782 KOps/s 1.1912 KOps/s $\color{#d91a1a}-1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9635ms 0.8224ms 1.2160 KOps/s 1.2200 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8862ms 3.6834ms 271.4919 Ops/s 277.9813 Ops/s $\color{#d91a1a}-2.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9673ms 0.8366ms 1.1952 KOps/s 1.2066 KOps/s $\color{#d91a1a}-0.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9630ms 0.8130ms 1.2300 KOps/s 1.2410 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.7695ms 3.3725ms 296.5173 Ops/s 307.4484 Ops/s $\color{#d91a1a}-3.56\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0751ms 0.9654ms 1.0359 KOps/s 1.0511 KOps/s $\color{#d91a1a}-1.45\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0692ms 0.9424ms 1.0612 KOps/s 1.0742 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1260s 10.3468ms 96.6484 Ops/s 96.1580 Ops/s $\color{#35bf28}+0.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.5733ms 14.1145ms 70.8489 Ops/s 75.0740 Ops/s $\textbf{\color{#d91a1a}-5.63\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.9898ms 3.3733ms 296.4491 Ops/s 308.3368 Ops/s $\color{#d91a1a}-3.86\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1263s 10.1982ms 98.0567 Ops/s 99.8005 Ops/s $\color{#d91a1a}-1.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.8720ms 14.4266ms 69.3163 Ops/s 75.0531 Ops/s $\textbf{\color{#d91a1a}-7.64\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.4239ms 3.3899ms 294.9967 Ops/s 306.7954 Ops/s $\color{#d91a1a}-3.85\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1234s 10.2908ms 97.1740 Ops/s 78.5440 Ops/s $\textbf{\color{#35bf28}+23.72\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1270s 16.5228ms 60.5225 Ops/s 73.8866 Ops/s $\textbf{\color{#d91a1a}-18.09\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.8177ms 3.5777ms 279.5115 Ops/s 290.5752 Ops/s $\color{#d91a1a}-3.81\%$

@vmoens vmoens merged commit 156a668 into main Jan 29, 2024
62 of 64 checks passed
@vmoens vmoens deleted the parallel-1-notparallel branch January 29, 2024 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot