Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix mp_start_method for ParallelEnv with single_for_serial #2007

Merged
merged 1 commit into from
Mar 11, 2024
Merged

[BugFix] Fix mp_start_method for ParallelEnv with single_for_serial #2007

merged 1 commit into from
Mar 11, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 11, 2024 •
edited
Loading

cc @teopir

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 11, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2007

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 11 Unrelated Failures

As of commit e9f9847 with merge base 358475a (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 11, 2024
@vmoens vmoens marked this pull request as ready for review March 11, 2024 17:36
@vmoens vmoens self-assigned this Mar 11, 2024
@vmoens vmoens added the bug Something isn't working label Mar 11, 2024
@vmoens vmoens merged commit 2b8450c into main Mar 11, 2024
37 of 52 checks passed
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 72.1802ms 54.5336ms 18.3373 Ops/s 17.1009 Ops/s $\textbf{\color{#35bf28}+7.23\%}$
test_sync 49.5344ms 33.5764ms 29.7829 Ops/s 32.6242 Ops/s $\textbf{\color{#d91a1a}-8.71\%}$
test_async 51.6539ms 27.0623ms 36.9518 Ops/s 34.7207 Ops/s $\textbf{\color{#35bf28}+6.43\%}$
test_simple 0.4007s 0.3401s 2.9403 Ops/s 2.8959 Ops/s $\color{#35bf28}+1.53\%$
test_transformed 0.5250s 0.4734s 2.1125 Ops/s 2.1185 Ops/s $\color{#d91a1a}-0.28\%$
test_serial 1.2371s 1.1884s 0.8415 Ops/s 0.8227 Ops/s $\color{#35bf28}+2.28\%$
test_parallel 1.0885s 1.0490s 0.9533 Ops/s 0.9588 Ops/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-True-True-True] 0.1772ms 20.9251μs 47.7894 KOps/s 47.4667 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-True-True-True-False] 62.9070μs 12.6883μs 78.8130 KOps/s 77.8190 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-True-True-False-True] 75.8070μs 12.1974μs 81.9847 KOps/s 80.7083 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-True-True-False-False] 30.6370μs 7.5110μs 133.1372 KOps/s 132.8840 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-True-False-True-True] 50.9040μs 22.2205μs 45.0035 KOps/s 44.9015 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-True-False-True-False] 57.3160μs 14.0757μs 71.0444 KOps/s 71.5830 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-False-False-True] 42.3590μs 13.4911μs 74.1228 KOps/s 73.8422 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-False-False-False] 50.4240μs 8.6489μs 115.6218 KOps/s 114.1503 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[True-False-True-True-True] 45.3450μs 23.5635μs 42.4385 KOps/s 42.1442 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-False-True-True-False] 62.0360μs 15.3914μs 64.9714 KOps/s 64.9391 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-False-True] 41.7680μs 13.4624μs 74.2811 KOps/s 73.7583 KOps/s $\color{#35bf28}+0.71\%$
test_step_mdp_speed[True-False-True-False-False] 60.3720μs 8.6441μs 115.6852 KOps/s 114.6237 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-False-False-True-True] 55.1830μs 24.6004μs 40.6497 KOps/s 40.6576 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-False-True-False] 75.3100μs 16.4757μs 60.6956 KOps/s 60.9091 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-False-False-False-True] 65.8220μs 14.6432μs 68.2910 KOps/s 69.0427 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-False-False-False-False] 35.0750μs 9.8112μs 101.9243 KOps/s 102.8798 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-True-True-True-True] 82.8640μs 23.8685μs 41.8961 KOps/s 42.6304 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-True-True-True-False] 51.7260μs 15.3557μs 65.1223 KOps/s 65.3105 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[False-True-True-False-True] 69.5400μs 15.6754μs 63.7942 KOps/s 63.4605 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-True-False-False] 37.8300μs 9.9029μs 100.9807 KOps/s 100.7541 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-False-True-True] 45.2840μs 25.2089μs 39.6685 KOps/s 38.8598 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-True-False-True-False] 70.8310μs 16.4931μs 60.6313 KOps/s 60.9306 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[False-True-False-False-True] 57.3060μs 16.9998μs 58.8243 KOps/s 59.6737 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[False-True-False-False-False] 59.3100μs 11.0018μs 90.8944 KOps/s 90.0996 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-True-True] 55.6940μs 25.9406μs 38.5496 KOps/s 38.3949 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[False-False-True-True-False] 76.4420μs 17.7652μs 56.2897 KOps/s 56.3824 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-False-True-False-True] 38.0200μs 16.8666μs 59.2887 KOps/s 60.2324 KOps/s $\color{#d91a1a}-1.57\%$
test_step_mdp_speed[False-False-True-False-False] 64.9110μs 11.0486μs 90.5096 KOps/s 90.4401 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-False-True-True] 82.4040μs 27.0961μs 36.9056 KOps/s 37.6200 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-False-False-True-False] 47.6690μs 18.6783μs 53.5380 KOps/s 54.0216 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-False-False-True] 70.9920μs 17.7460μs 56.3507 KOps/s 56.2688 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-False-False-False] 48.3800μs 11.9798μs 83.4740 KOps/s 84.2659 KOps/s $\color{#d91a1a}-0.94\%$
test_values[generalized_advantage_estimate-True-True] 10.7009ms 9.5255ms 104.9814 Ops/s 99.6261 Ops/s $\textbf{\color{#35bf28}+5.38\%}$
test_values[vec_generalized_advantage_estimate-True-True] 36.5454ms 35.1691ms 28.4340 Ops/s 28.4049 Ops/s $\color{#35bf28}+0.10\%$
test_values[td0_return_estimate-False-False] 0.2181ms 0.1743ms 5.7369 KOps/s 5.9743 KOps/s $\color{#d91a1a}-3.97\%$
test_values[td1_return_estimate-False-False] 27.1976ms 23.5853ms 42.3993 Ops/s 42.7746 Ops/s $\color{#d91a1a}-0.88\%$
test_values[vec_td1_return_estimate-False-False] 36.4076ms 35.2856ms 28.3402 Ops/s 28.2902 Ops/s $\color{#35bf28}+0.18\%$
test_values[td_lambda_return_estimate-True-False] 35.0322ms 34.1295ms 29.3001 Ops/s 29.9620 Ops/s $\color{#d91a1a}-2.21\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.8216ms 35.3296ms 28.3048 Ops/s 28.3047 Ops/s $+0.00\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.3935ms 8.2805ms 120.7650 Ops/s 122.3046 Ops/s $\color{#d91a1a}-1.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.7781ms 2.0553ms 486.5466 Ops/s 505.7116 Ops/s $\color{#d91a1a}-3.79\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5491ms 0.3466ms 2.8852 KOps/s 2.7932 KOps/s $\color{#35bf28}+3.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9615ms 45.7045ms 21.8797 Ops/s 21.5931 Ops/s $\color{#35bf28}+1.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.1717ms 3.0279ms 330.2646 Ops/s 329.2958 Ops/s $\color{#35bf28}+0.29\%$
test_dqn_speed 2.2581ms 1.3368ms 748.0662 Ops/s 734.7142 Ops/s $\color{#35bf28}+1.82\%$
test_ddpg_speed 3.3118ms 2.6377ms 379.1150 Ops/s 369.1248 Ops/s $\color{#35bf28}+2.71\%$
test_sac_speed 9.6730ms 8.1998ms 121.9545 Ops/s 119.5234 Ops/s $\color{#35bf28}+2.03\%$
test_redq_speed 14.2247ms 13.1639ms 75.9652 Ops/s 74.6718 Ops/s $\color{#35bf28}+1.73\%$
test_redq_deprec_speed 14.4595ms 13.3112ms 75.1247 Ops/s 74.6646 Ops/s $\color{#35bf28}+0.62\%$
test_td3_speed 8.6345ms 8.1238ms 123.0944 Ops/s 119.4877 Ops/s $\color{#35bf28}+3.02\%$
test_cql_speed 0.1097s 39.2851ms 25.4550 Ops/s 26.9028 Ops/s $\textbf{\color{#d91a1a}-5.38\%}$
test_a2c_speed 8.6519ms 7.5252ms 132.8861 Ops/s 131.8947 Ops/s $\color{#35bf28}+0.75\%$
test_ppo_speed 9.3764ms 7.8083ms 128.0688 Ops/s 127.7026 Ops/s $\color{#35bf28}+0.29\%$
test_reinforce_speed 7.7587ms 6.6305ms 150.8180 Ops/s 151.2656 Ops/s $\color{#d91a1a}-0.30\%$
test_iql_speed 34.0351ms 32.5406ms 30.7308 Ops/s 30.5122 Ops/s $\color{#35bf28}+0.72\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.4906ms 2.3722ms 421.5519 Ops/s 425.4015 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6809ms 0.5055ms 1.9781 KOps/s 1.9754 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.4965ms 0.4816ms 2.0765 KOps/s 2.0785 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4565ms 2.4122ms 414.5595 Ops/s 423.8965 Ops/s $\color{#d91a1a}-2.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7988ms 0.5003ms 1.9988 KOps/s 2.0383 KOps/s $\color{#d91a1a}-1.94\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6878ms 0.4740ms 2.1096 KOps/s 2.0955 KOps/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9743ms 1.2668ms 789.4072 Ops/s 771.9602 Ops/s $\color{#35bf28}+2.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4011ms 1.1988ms 834.1368 Ops/s 810.0449 Ops/s $\color{#35bf28}+2.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.6172ms 2.4812ms 403.0322 Ops/s 405.3870 Ops/s $\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7827ms 0.6194ms 1.6144 KOps/s 1.6093 KOps/s $\color{#35bf28}+0.32\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1050s 0.6727ms 1.4865 KOps/s 1.6584 KOps/s $\textbf{\color{#d91a1a}-10.36\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8823ms 2.3704ms 421.8704 Ops/s 426.2407 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6371ms 0.5018ms 1.9927 KOps/s 1.9877 KOps/s $\color{#35bf28}+0.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5202ms 0.4853ms 2.0605 KOps/s 2.0412 KOps/s $\color{#35bf28}+0.95\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1608ms 2.3657ms 422.7136 Ops/s 424.5282 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 92.8763ms 0.6184ms 1.6171 KOps/s 2.0123 KOps/s $\textbf{\color{#d91a1a}-19.64\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6803ms 0.4705ms 2.1254 KOps/s 2.1029 KOps/s $\color{#35bf28}+1.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6479ms 2.4948ms 400.8408 Ops/s 406.9434 Ops/s $\color{#d91a1a}-1.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8343ms 0.6210ms 1.6103 KOps/s 1.5873 KOps/s $\color{#35bf28}+1.45\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.5558ms 0.5914ms 1.6908 KOps/s 1.6654 KOps/s $\color{#35bf28}+1.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 93.2193ms 7.1248ms 140.3549 Ops/s 132.7274 Ops/s $\textbf{\color{#35bf28}+5.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.9201ms 12.3972ms 80.6636 Ops/s 82.5493 Ops/s $\color{#d91a1a}-2.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.4904ms 1.0584ms 944.8003 Ops/s 924.1269 Ops/s $\color{#35bf28}+2.24\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 93.3436ms 7.2428ms 138.0674 Ops/s 137.4167 Ops/s $\color{#35bf28}+0.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.4521ms 12.2985ms 81.3105 Ops/s 83.0068 Ops/s $\color{#d91a1a}-2.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.4910ms 1.0247ms 975.9042 Ops/s 950.6103 Ops/s $\color{#35bf28}+2.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 91.4367ms 5.7655ms 173.4455 Ops/s 128.0692 Ops/s $\textbf{\color{#35bf28}+35.43\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.2716ms 12.9894ms 76.9856 Ops/s 80.0593 Ops/s $\color{#d91a1a}-3.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.2577ms 1.5118ms 661.4507 Ops/s 723.7791 Ops/s $\textbf{\color{#d91a1a}-8.61\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1005s 98.6745ms 10.1343 Ops/s 9.4407 Ops/s $\textbf{\color{#35bf28}+7.35\%}$
test_sync 88.7858ms 87.5902ms 11.4168 Ops/s 11.5960 Ops/s $\color{#d91a1a}-1.55\%$
test_async 0.1697s 85.6218ms 11.6793 Ops/s 13.9573 Ops/s $\textbf{\color{#d91a1a}-16.32\%}$
test_single_pixels 0.1970s 0.1176s 8.5064 Ops/s 9.0495 Ops/s $\textbf{\color{#d91a1a}-6.00\%}$
test_sync_pixels 67.2201ms 66.1594ms 15.1150 Ops/s 15.2626 Ops/s $\color{#d91a1a}-0.97\%$
test_async_pixels 0.1234s 56.4532ms 17.7138 Ops/s 17.6595 Ops/s $\color{#35bf28}+0.31\%$
test_simple 0.6399s 0.6381s 1.5671 Ops/s 1.4889 Ops/s $\textbf{\color{#35bf28}+5.26\%}$
test_transformed 0.9324s 0.8544s 1.1704 Ops/s 1.1498 Ops/s $\color{#35bf28}+1.79\%$
test_serial 2.0702s 2.0159s 0.4961 Ops/s 0.4886 Ops/s $\color{#35bf28}+1.53\%$
test_parallel 1.8543s 1.7718s 0.5644 Ops/s 0.5504 Ops/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-True-True-True-True] 92.3910μs 31.7079μs 31.5379 KOps/s 31.2064 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-True-True-False] 41.6400μs 18.8537μs 53.0400 KOps/s 52.3376 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-False-True] 40.7310μs 17.7048μs 56.4817 KOps/s 55.0257 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-True-True-False-False] 37.5300μs 10.7385μs 93.1231 KOps/s 90.7044 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[True-True-False-True-True] 62.4600μs 33.6913μs 29.6812 KOps/s 29.9610 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-False-True-False] 43.5010μs 20.7192μs 48.2645 KOps/s 47.3007 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[True-True-False-False-True] 43.5910μs 19.7439μs 50.6487 KOps/s 50.3327 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-False-False-False] 33.3110μs 12.5507μs 79.6771 KOps/s 78.1185 KOps/s $\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-False-True-True-True] 64.8810μs 35.4708μs 28.1922 KOps/s 28.1887 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-False-True-True-False] 49.7110μs 22.4148μs 44.6134 KOps/s 43.6193 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-False-True-False-True] 48.8200μs 19.5893μs 51.0482 KOps/s 50.1014 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[True-False-True-False-False] 44.4200μs 12.4354μs 80.4157 KOps/s 78.1347 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-False-False-True-True] 74.8520μs 37.2188μs 26.8682 KOps/s 27.0161 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-False-False-True-False] 47.5100μs 24.2117μs 41.3023 KOps/s 40.2590 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-False-False-False-True] 46.7910μs 21.0481μs 47.5102 KOps/s 45.8462 KOps/s $\color{#35bf28}+3.63\%$
test_step_mdp_speed[True-False-False-False-False] 35.6510μs 14.2341μs 70.2538 KOps/s 68.0383 KOps/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[False-True-True-True-True] 66.1910μs 35.3157μs 28.3160 KOps/s 28.1590 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-True-True-True-False] 47.3400μs 22.7183μs 44.0173 KOps/s 43.5310 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-True-True-False-True] 49.2010μs 23.1339μs 43.2267 KOps/s 41.4965 KOps/s $\color{#35bf28}+4.17\%$
test_step_mdp_speed[False-True-True-False-False] 35.8010μs 14.2308μs 70.2703 KOps/s 68.0494 KOps/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[False-True-False-True-True] 76.3710μs 37.7187μs 26.5120 KOps/s 26.6772 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-True-False-True-False] 52.8000μs 24.3946μs 40.9928 KOps/s 40.4453 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[False-True-False-False-True] 98.4230μs 24.7799μs 40.3554 KOps/s 38.8521 KOps/s $\color{#35bf28}+3.87\%$
test_step_mdp_speed[False-True-False-False-False] 38.6400μs 16.0368μs 62.3567 KOps/s 60.8757 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-False-True-True-True] 73.2810μs 39.0942μs 25.5792 KOps/s 25.6423 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-False-True-True-False] 65.6810μs 26.2511μs 38.0937 KOps/s 37.4884 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-False-True-False-True] 51.5400μs 25.0825μs 39.8684 KOps/s 38.8588 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[False-False-True-False-False] 89.8210μs 15.9571μs 62.6680 KOps/s 61.7069 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-False-False-True-True] 72.6110μs 40.2397μs 24.8511 KOps/s 24.5914 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-False-True-False] 52.3610μs 27.8376μs 35.9227 KOps/s 35.3827 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-False-False-False-True] 51.5310μs 26.6311μs 37.5501 KOps/s 36.7745 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-False-False-False-False] 38.9710μs 17.6877μs 56.5364 KOps/s 56.0475 KOps/s $\color{#35bf28}+0.87\%$
test_values[generalized_advantage_estimate-True-True] 23.2342ms 22.6761ms 44.0993 Ops/s 43.3181 Ops/s $\color{#35bf28}+1.80\%$
test_values[vec_generalized_advantage_estimate-True-True] 86.0714ms 3.2566ms 307.0726 Ops/s 308.6323 Ops/s $\color{#d91a1a}-0.51\%$
test_values[td0_return_estimate-False-False] 86.8220μs 60.5525μs 16.5146 KOps/s 16.0720 KOps/s $\color{#35bf28}+2.75\%$
test_values[td1_return_estimate-False-False] 49.6129ms 48.9930ms 20.4111 Ops/s 20.6601 Ops/s $\color{#d91a1a}-1.21\%$
test_values[vec_td1_return_estimate-False-False] 2.1420ms 1.7242ms 579.9707 Ops/s 577.9686 Ops/s $\color{#35bf28}+0.35\%$
test_values[td_lambda_return_estimate-True-False] 81.0356ms 78.4367ms 12.7491 Ops/s 12.7330 Ops/s $\color{#35bf28}+0.13\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0854ms 1.7248ms 579.7698 Ops/s 577.2980 Ops/s $\color{#35bf28}+0.43\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 21.3879ms 21.0494ms 47.5074 Ops/s 46.1041 Ops/s $\color{#35bf28}+3.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8449ms 0.6622ms 1.5101 KOps/s 1.4596 KOps/s $\color{#35bf28}+3.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7485ms 0.6135ms 1.6299 KOps/s 1.5661 KOps/s $\color{#35bf28}+4.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6122ms 1.4175ms 705.4532 Ops/s 699.3499 Ops/s $\color{#35bf28}+0.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.0873ms 0.6378ms 1.5679 KOps/s 1.5683 KOps/s $\color{#d91a1a}-0.03\%$
test_dqn_speed 1.6814ms 1.3731ms 728.2654 Ops/s 717.9727 Ops/s $\color{#35bf28}+1.43\%$
test_ddpg_speed 2.9973ms 2.6338ms 379.6770 Ops/s 378.5791 Ops/s $\color{#35bf28}+0.29\%$
test_sac_speed 8.4025ms 7.6849ms 130.1254 Ops/s 129.3033 Ops/s $\color{#35bf28}+0.64\%$
test_redq_speed 12.7338ms 10.7239ms 93.2495 Ops/s 94.5880 Ops/s $\color{#d91a1a}-1.42\%$
test_redq_deprec_speed 12.0007ms 11.1756ms 89.4805 Ops/s 87.2181 Ops/s $\color{#35bf28}+2.59\%$
test_td3_speed 7.8486ms 7.6962ms 129.9339 Ops/s 130.1823 Ops/s $\color{#d91a1a}-0.19\%$
test_cql_speed 26.6092ms 25.5855ms 39.0846 Ops/s 35.5454 Ops/s $\textbf{\color{#35bf28}+9.96\%}$
test_a2c_speed 5.9707ms 5.4503ms 183.4745 Ops/s 179.5092 Ops/s $\color{#35bf28}+2.21\%$
test_ppo_speed 6.2253ms 5.7988ms 172.4508 Ops/s 170.8488 Ops/s $\color{#35bf28}+0.94\%$
test_reinforce_speed 4.7608ms 4.4741ms 223.5081 Ops/s 219.1282 Ops/s $\color{#35bf28}+2.00\%$
test_iql_speed 21.1038ms 19.8982ms 50.2558 Ops/s 50.5007 Ops/s $\color{#d91a1a}-0.48\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0631ms 2.8842ms 346.7157 Ops/s 345.5045 Ops/s $\color{#35bf28}+0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3053ms 0.5504ms 1.8169 KOps/s 1.8046 KOps/s $\color{#35bf28}+0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7547ms 0.5283ms 1.8928 KOps/s 1.8897 KOps/s $\color{#35bf28}+0.16\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.4225ms 3.0019ms 333.1226 Ops/s 333.5271 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4413ms 0.5484ms 1.8233 KOps/s 1.8317 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6689ms 0.5265ms 1.8993 KOps/s 1.9009 KOps/s $\color{#d91a1a}-0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 0.1239s 1.7081ms 585.4399 Ops/s 675.2522 Ops/s $\textbf{\color{#d91a1a}-13.30\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5956ms 1.3959ms 716.3754 Ops/s 706.6079 Ops/s $\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2504ms 3.0275ms 330.3070 Ops/s 327.2150 Ops/s $\color{#35bf28}+0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0393ms 0.6706ms 1.4912 KOps/s 1.4878 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.7798ms 0.6572ms 1.5215 KOps/s 1.2921 KOps/s $\textbf{\color{#35bf28}+17.76\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0556ms 2.8780ms 347.4600 Ops/s 342.2653 Ops/s $\color{#35bf28}+1.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7916ms 0.5458ms 1.8320 KOps/s 1.8111 KOps/s $\color{#35bf28}+1.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6767ms 0.5348ms 1.8699 KOps/s 1.8791 KOps/s $\color{#d91a1a}-0.49\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8766ms 2.9963ms 333.7479 Ops/s 336.3854 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.4135ms 0.5400ms 1.8517 KOps/s 1.8367 KOps/s $\color{#35bf28}+0.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6789ms 0.5184ms 1.9289 KOps/s 1.3770 KOps/s $\textbf{\color{#35bf28}+40.08\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1730ms 2.9729ms 336.3721 Ops/s 330.4594 Ops/s $\color{#35bf28}+1.79\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7976ms 0.6687ms 1.4954 KOps/s 1.4787 KOps/s $\color{#35bf28}+1.13\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6758ms 0.6524ms 1.5329 KOps/s 1.5249 KOps/s $\color{#35bf28}+0.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1599s 7.9128ms 126.3772 Ops/s 105.8376 Ops/s $\textbf{\color{#35bf28}+19.41\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.2516ms 13.7801ms 72.5686 Ops/s 71.9114 Ops/s $\color{#35bf28}+0.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3632ms 1.0715ms 933.3115 Ops/s 867.1827 Ops/s $\textbf{\color{#35bf28}+7.63\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1375s 10.0368ms 99.6334 Ops/s 147.2595 Ops/s $\textbf{\color{#d91a1a}-32.34\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.1431ms 13.8332ms 72.2900 Ops/s 71.3095 Ops/s $\color{#35bf28}+1.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1083ms 1.0467ms 955.3508 Ops/s 782.9709 Ops/s $\textbf{\color{#35bf28}+22.02\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1373s 7.7977ms 128.2426 Ops/s 101.6025 Ops/s $\textbf{\color{#35bf28}+26.22\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.5426ms 14.2771ms 70.0422 Ops/s 69.4706 Ops/s $\color{#35bf28}+0.82\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.5001ms 1.3739ms 727.8647 Ops/s 686.5712 Ops/s $\textbf{\color{#35bf28}+6.01\%}$

@vmoens vmoens deleted the fix-fork-singlepenv branch April 3, 2024 06:04
vmoens added a commit that referenced this pull request Apr 7, 2024
@vmoens
[BugFix] Fix mp_start_method for ParallelEnv with single_for_serial (#…
11512c9 
…2007)

(cherry picked from commit 2b8450c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@vmoens vmoens

Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot