Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix async gym env with non-sync resets #2170

Merged
merged 8 commits into from
May 28, 2024
Merged

[BugFix] Fix async gym env with non-sync resets #2170

merged 8 commits into from
May 28, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented May 23, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented May 23, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2170

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unrelated Failures

As of commit ee2bab5 with merge base 78b4717 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 23, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented May 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1002s 55.7119ms 17.9495 Ops/s 18.4183 Ops/s $\color{#d91a1a}-2.55\%$
test_sync 30.8612ms 29.5630ms 33.8260 Ops/s 33.9603 Ops/s $\color{#d91a1a}-0.40\%$
test_async 56.3676ms 28.4222ms 35.1838 Ops/s 35.5096 Ops/s $\color{#d91a1a}-0.92\%$
test_simple 0.4316s 0.3536s 2.8277 Ops/s 2.9443 Ops/s $\color{#d91a1a}-3.96\%$
test_transformed 0.4842s 0.4818s 2.0757 Ops/s 2.0021 Ops/s $\color{#35bf28}+3.67\%$
test_serial 1.2533s 1.2056s 0.8295 Ops/s 0.8126 Ops/s $\color{#35bf28}+2.07\%$
test_parallel 1.0856s 1.0222s 0.9783 Ops/s 0.9804 Ops/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[True-True-True-True-True] 0.1699ms 21.5064μs 46.4977 KOps/s 45.6254 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[True-True-True-True-False] 46.4560μs 13.0580μs 76.5816 KOps/s 74.4752 KOps/s $\color{#35bf28}+2.83\%$
test_step_mdp_speed[True-True-True-False-True] 46.7270μs 12.9862μs 77.0049 KOps/s 77.1633 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-True-True-False-False] 31.8490μs 7.7408μs 129.1863 KOps/s 126.1562 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[True-True-False-True-True] 61.8150μs 22.7808μs 43.8967 KOps/s 42.7651 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-True-False-True-False] 50.3330μs 14.2841μs 70.0080 KOps/s 67.5723 KOps/s $\color{#35bf28}+3.60\%$
test_step_mdp_speed[True-True-False-False-True] 76.5950μs 13.8517μs 72.1931 KOps/s 70.3717 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-True-False-False-False] 29.4950μs 8.8358μs 113.1754 KOps/s 110.0841 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-False-True-True-True] 63.3180μs 24.0299μs 41.6148 KOps/s 40.5109 KOps/s $\color{#35bf28}+2.72\%$
test_step_mdp_speed[True-False-True-True-False] 37.5500μs 15.4741μs 64.6239 KOps/s 62.6179 KOps/s $\color{#35bf28}+3.20\%$
test_step_mdp_speed[True-False-True-False-True] 41.8570μs 13.7586μs 72.6820 KOps/s 69.5463 KOps/s $\color{#35bf28}+4.51\%$
test_step_mdp_speed[True-False-True-False-False] 32.0200μs 8.8074μs 113.5403 KOps/s 110.5904 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[True-False-False-True-True] 64.2500μs 25.0701μs 39.8882 KOps/s 38.6987 KOps/s $\color{#35bf28}+3.07\%$
test_step_mdp_speed[True-False-False-True-False] 45.6650μs 16.6817μs 59.9458 KOps/s 57.8161 KOps/s $\color{#35bf28}+3.68\%$
test_step_mdp_speed[True-False-False-False-True] 53.3590μs 14.8210μs 67.4716 KOps/s 65.5640 KOps/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[True-False-False-False-False] 39.6540μs 9.9739μs 100.2614 KOps/s 98.1075 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-True-True-True-True] 61.0630μs 23.6610μs 42.2636 KOps/s 40.6986 KOps/s $\color{#35bf28}+3.85\%$
test_step_mdp_speed[False-True-True-True-False] 34.9950μs 15.4374μs 64.7779 KOps/s 62.9664 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[False-True-True-False-True] 47.3280μs 16.0052μs 62.4798 KOps/s 61.4017 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-True-True-False-False] 43.5610μs 9.9449μs 100.5537 KOps/s 97.1678 KOps/s $\color{#35bf28}+3.48\%$
test_step_mdp_speed[False-True-False-True-True] 54.1110μs 24.8044μs 40.3154 KOps/s 38.9592 KOps/s $\color{#35bf28}+3.48\%$
test_step_mdp_speed[False-True-False-True-False] 39.0530μs 16.6098μs 60.2055 KOps/s 57.8150 KOps/s $\color{#35bf28}+4.13\%$
test_step_mdp_speed[False-True-False-False-True] 64.2590μs 16.9784μs 58.8985 KOps/s 56.6057 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[False-True-False-False-False] 33.3820μs 11.1347μs 89.8093 KOps/s 87.1999 KOps/s $\color{#35bf28}+2.99\%$
test_step_mdp_speed[False-False-True-True-True] 57.1660μs 26.1595μs 38.2271 KOps/s 36.9114 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[False-False-True-True-False] 45.1340μs 17.8453μs 56.0370 KOps/s 53.8911 KOps/s $\color{#35bf28}+3.98\%$
test_step_mdp_speed[False-False-True-False-True] 46.7670μs 16.9313μs 59.0620 KOps/s 57.0533 KOps/s $\color{#35bf28}+3.52\%$
test_step_mdp_speed[False-False-True-False-False] 31.8800μs 11.1330μs 89.8234 KOps/s 87.3441 KOps/s $\color{#35bf28}+2.84\%$
test_step_mdp_speed[False-False-False-True-True] 48.3200μs 27.5202μs 36.3370 KOps/s 35.1808 KOps/s $\color{#35bf28}+3.29\%$
test_step_mdp_speed[False-False-False-True-False] 0.2600ms 19.7514μs 50.6293 KOps/s 51.0895 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-False-False-True] 52.4670μs 17.9345μs 55.7585 KOps/s 53.9493 KOps/s $\color{#35bf28}+3.35\%$
test_step_mdp_speed[False-False-False-False-False] 53.5290μs 12.2542μs 81.6048 KOps/s 77.7984 KOps/s $\color{#35bf28}+4.89\%$
test_values[generalized_advantage_estimate-True-True] 15.8609ms 9.4828ms 105.4545 Ops/s 107.6857 Ops/s $\color{#d91a1a}-2.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.2416ms 33.4158ms 29.9259 Ops/s 28.3784 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_values[td0_return_estimate-False-False] 0.2283ms 0.1706ms 5.8629 KOps/s 5.9242 KOps/s $\color{#d91a1a}-1.04\%$
test_values[td1_return_estimate-False-False] 23.8485ms 23.0128ms 43.4541 Ops/s 42.4916 Ops/s $\color{#35bf28}+2.27\%$
test_values[vec_td1_return_estimate-False-False] 35.0070ms 33.5920ms 29.7690 Ops/s 28.3152 Ops/s $\textbf{\color{#35bf28}+5.13\%}$
test_values[td_lambda_return_estimate-True-False] 36.9205ms 33.5881ms 29.7724 Ops/s 29.7025 Ops/s $\color{#35bf28}+0.24\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.8846ms 33.5194ms 29.8334 Ops/s 28.1808 Ops/s $\textbf{\color{#35bf28}+5.86\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.3109ms 8.1794ms 122.2580 Ops/s 121.7915 Ops/s $\color{#35bf28}+0.38\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9499ms 1.8528ms 539.7243 Ops/s 559.4591 Ops/s $\color{#d91a1a}-3.53\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4712ms 0.3545ms 2.8211 KOps/s 2.8472 KOps/s $\color{#d91a1a}-0.92\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.0150ms 46.2016ms 21.6443 Ops/s 22.5388 Ops/s $\color{#d91a1a}-3.97\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8109ms 3.0467ms 328.2272 Ops/s 325.0145 Ops/s $\color{#35bf28}+0.99\%$
test_dqn_speed 2.5405ms 1.3474ms 742.1529 Ops/s 734.4932 Ops/s $\color{#35bf28}+1.04\%$
test_ddpg_speed 3.5220ms 2.8588ms 349.7920 Ops/s 351.4139 Ops/s $\color{#d91a1a}-0.46\%$
test_sac_speed 8.9782ms 8.3572ms 119.6567 Ops/s 118.7701 Ops/s $\color{#35bf28}+0.75\%$
test_redq_speed 15.0132ms 13.2586ms 75.4227 Ops/s 76.4335 Ops/s $\color{#d91a1a}-1.32\%$
test_redq_deprec_speed 14.5351ms 13.3837ms 74.7180 Ops/s 69.3327 Ops/s $\textbf{\color{#35bf28}+7.77\%}$
test_td3_speed 15.9380ms 8.3837ms 119.2795 Ops/s 118.9158 Ops/s $\color{#35bf28}+0.31\%$
test_cql_speed 55.2582ms 37.7431ms 26.4949 Ops/s 27.4557 Ops/s $\color{#d91a1a}-3.50\%$
test_a2c_speed 8.6692ms 7.5170ms 133.0326 Ops/s 130.2821 Ops/s $\color{#35bf28}+2.11\%$
test_ppo_speed 8.1609ms 7.7045ms 129.7945 Ops/s 127.9465 Ops/s $\color{#35bf28}+1.44\%$
test_reinforce_speed 7.6429ms 6.6584ms 150.1859 Ops/s 150.0764 Ops/s $\color{#35bf28}+0.07\%$
test_iql_speed 33.9712ms 32.9696ms 30.3310 Ops/s 30.2616 Ops/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3801ms 2.2932ms 436.0654 Ops/s 455.5286 Ops/s $\color{#d91a1a}-4.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0987ms 0.5234ms 1.9107 KOps/s 2.0108 KOps/s $\color{#d91a1a}-4.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6825ms 0.4740ms 2.1096 KOps/s 2.1301 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3739ms 2.3243ms 430.2355 Ops/s 474.0699 Ops/s $\textbf{\color{#d91a1a}-9.25\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0910ms 0.4950ms 2.0201 KOps/s 2.0599 KOps/s $\color{#d91a1a}-1.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7714ms 0.4692ms 2.1313 KOps/s 2.1634 KOps/s $\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8184ms 1.2256ms 815.8966 Ops/s 803.9242 Ops/s $\color{#35bf28}+1.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.1541ms 1.1642ms 858.9276 Ops/s 848.7758 Ops/s $\color{#35bf28}+1.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.3700ms 2.3334ms 428.5676 Ops/s 428.6161 Ops/s $\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1372ms 0.6134ms 1.6301 KOps/s 1.6178 KOps/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1091s 0.6682ms 1.4965 KOps/s 1.6329 KOps/s $\textbf{\color{#d91a1a}-8.35\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3105ms 2.1732ms 460.1601 Ops/s 451.3320 Ops/s $\color{#35bf28}+1.96\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0288ms 0.4947ms 2.0215 KOps/s 1.9831 KOps/s $\color{#35bf28}+1.94\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8183ms 0.4753ms 2.1040 KOps/s 2.0726 KOps/s $\color{#35bf28}+1.52\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4526ms 2.1499ms 465.1479 Ops/s 446.4225 Ops/s $\color{#35bf28}+4.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7747ms 0.4857ms 2.0589 KOps/s 2.0326 KOps/s $\color{#35bf28}+1.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1022s 0.5971ms 1.6748 KOps/s 2.1194 KOps/s $\textbf{\color{#d91a1a}-20.98\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5626ms 2.2816ms 438.2911 Ops/s 434.3418 Ops/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9259ms 0.6147ms 1.6267 KOps/s 1.6051 KOps/s $\color{#35bf28}+1.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8259ms 0.5851ms 1.7091 KOps/s 1.6607 KOps/s $\color{#35bf28}+2.92\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1070s 7.7606ms 128.8567 Ops/s 115.1991 Ops/s $\textbf{\color{#35bf28}+11.86\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.2834ms 12.5879ms 79.4412 Ops/s 78.5477 Ops/s $\color{#35bf28}+1.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8166ms 1.0728ms 932.1781 Ops/s 921.0171 Ops/s $\color{#35bf28}+1.21\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1133s 6.0921ms 164.1461 Ops/s 174.7255 Ops/s $\textbf{\color{#d91a1a}-6.05\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.0747ms 12.2972ms 81.3193 Ops/s 79.0269 Ops/s $\color{#35bf28}+2.90\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.1197ms 1.0440ms 957.8172 Ops/s 927.6466 Ops/s $\color{#35bf28}+3.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1092s 6.0963ms 164.0338 Ops/s 125.9543 Ops/s $\textbf{\color{#35bf28}+30.23\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1191s 14.8280ms 67.4401 Ops/s 76.1771 Ops/s $\textbf{\color{#d91a1a}-11.47\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8996ms 1.3813ms 723.9695 Ops/s 726.3743 Ops/s $\color{#d91a1a}-0.33\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented May 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1173s 0.1165s 8.5809 Ops/s 8.7189 Ops/s $\color{#d91a1a}-1.58\%$
test_sync 0.1008s 99.4566ms 10.0546 Ops/s 9.6996 Ops/s $\color{#35bf28}+3.66\%$
test_async 0.1921s 97.2860ms 10.2790 Ops/s 10.1670 Ops/s $\color{#35bf28}+1.10\%$
test_single_pixels 0.1297s 0.1284s 7.7875 Ops/s 7.9743 Ops/s $\color{#d91a1a}-2.34\%$
test_sync_pixels 83.3231ms 81.7269ms 12.2359 Ops/s 12.3681 Ops/s $\color{#d91a1a}-1.07\%$
test_async_pixels 0.1618s 66.6587ms 15.0018 Ops/s 14.7976 Ops/s $\color{#35bf28}+1.38\%$
test_simple 0.7925s 0.7796s 1.2827 Ops/s 1.2928 Ops/s $\color{#d91a1a}-0.78\%$
test_transformed 1.0316s 1.0271s 0.9737 Ops/s 0.9867 Ops/s $\color{#d91a1a}-1.33\%$
test_serial 2.5473s 2.4817s 0.4030 Ops/s 0.4117 Ops/s $\color{#d91a1a}-2.12\%$
test_parallel 2.3797s 2.3082s 0.4332 Ops/s 0.4306 Ops/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[True-True-True-True-True] 92.3120μs 33.7265μs 29.6503 KOps/s 29.7946 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-True-True-False] 40.5310μs 20.0476μs 49.8813 KOps/s 50.2000 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-True-False-True] 37.8700μs 19.1465μs 52.2290 KOps/s 52.3302 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-True-True-False-False] 36.7820μs 11.4125μs 87.6232 KOps/s 86.9242 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-False-True-True] 56.6310μs 34.9129μs 28.6427 KOps/s 28.3820 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-True-False] 42.3600μs 21.6868μs 46.1109 KOps/s 46.1685 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-False-False-True] 43.7910μs 20.7538μs 48.1839 KOps/s 47.4673 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-False-False-False] 62.0610μs 13.2553μs 75.4415 KOps/s 74.8467 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[True-False-True-True-True] 61.9910μs 36.6264μs 27.3027 KOps/s 27.0203 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-False-True-True-False] 51.8610μs 23.6077μs 42.3591 KOps/s 41.8335 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-False-True-False-True] 61.9210μs 21.0826μs 47.4324 KOps/s 47.8901 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-True-False-False] 48.5800μs 13.3429μs 74.9462 KOps/s 74.3625 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-False-False-True-True] 63.1910μs 38.5067μs 25.9695 KOps/s 25.6046 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-False-True-False] 60.9210μs 25.2346μs 39.6282 KOps/s 39.0002 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-False-False-False-True] 53.9410μs 23.1166μs 43.2589 KOps/s 43.6532 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-False-False] 33.0000μs 15.0695μs 66.3591 KOps/s 65.3397 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-True-True-True] 55.7510μs 36.9122μs 27.0913 KOps/s 26.7392 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-True-True-True-False] 57.5520μs 23.7686μs 42.0723 KOps/s 42.2113 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-True-False-True] 49.6300μs 24.8390μs 40.2592 KOps/s 39.9552 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-True-False-False] 39.7110μs 15.0697μs 66.3582 KOps/s 65.1176 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-True-False-True-True] 63.5810μs 38.9658μs 25.6635 KOps/s 25.4471 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-True-False-True-False] 43.0510μs 25.3334μs 39.4735 KOps/s 39.1129 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-True-False-False-True] 61.0510μs 26.6124μs 37.5764 KOps/s 37.0291 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-True-False-False-False] 36.4900μs 17.1864μs 58.1856 KOps/s 58.5744 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-True-True-True] 65.7110μs 40.2975μs 24.8154 KOps/s 24.7064 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-False-True-True-False] 45.5220μs 27.5174μs 36.3407 KOps/s 35.7315 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-True-False-True] 45.1800μs 26.3354μs 37.9716 KOps/s 37.1105 KOps/s $\color{#35bf28}+2.32\%$
test_step_mdp_speed[False-False-True-False-False] 41.1600μs 17.0149μs 58.7721 KOps/s 59.2332 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-False-False-True-True] 55.4100μs 42.5751μs 23.4879 KOps/s 23.1432 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-False-False-True-False] 53.4910μs 29.3982μs 34.0157 KOps/s 33.4755 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-False-False-False-True] 60.0910μs 28.0555μs 35.6437 KOps/s 35.4710 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-False-False-False-False] 40.9010μs 18.4611μs 54.1680 KOps/s 53.6822 KOps/s $\color{#35bf28}+0.91\%$
test_values[generalized_advantage_estimate-True-True] 25.4523ms 24.8729ms 40.2044 Ops/s 40.2145 Ops/s $\color{#d91a1a}-0.03\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.9926ms 3.2111ms 311.4242 Ops/s 299.2362 Ops/s $\color{#35bf28}+4.07\%$
test_values[td0_return_estimate-False-False] 91.2910μs 64.4155μs 15.5242 KOps/s 15.5568 KOps/s $\color{#d91a1a}-0.21\%$
test_values[td1_return_estimate-False-False] 53.7171ms 53.0337ms 18.8559 Ops/s 18.8187 Ops/s $\color{#35bf28}+0.20\%$
test_values[vec_td1_return_estimate-False-False] 2.0644ms 1.7738ms 563.7698 Ops/s 565.2890 Ops/s $\color{#d91a1a}-0.27\%$
test_values[td_lambda_return_estimate-True-False] 85.1759ms 84.4379ms 11.8430 Ops/s 11.8241 Ops/s $\color{#35bf28}+0.16\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0731ms 1.7666ms 566.0568 Ops/s 566.2285 Ops/s $\color{#d91a1a}-0.03\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.9051ms 23.5973ms 42.3778 Ops/s 42.5030 Ops/s $\color{#d91a1a}-0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9237ms 0.7072ms 1.4140 KOps/s 1.4265 KOps/s $\color{#d91a1a}-0.88\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7205ms 0.6546ms 1.5276 KOps/s 1.5278 KOps/s $\color{#d91a1a}-0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5147ms 1.4574ms 686.1581 Ops/s 685.3174 Ops/s $\color{#35bf28}+0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9426ms 0.6714ms 1.4895 KOps/s 1.4782 KOps/s $\color{#35bf28}+0.77\%$
test_dqn_speed 78.2010ms 1.5938ms 627.4235 Ops/s 696.2557 Ops/s $\textbf{\color{#d91a1a}-9.89\%}$
test_ddpg_speed 3.2201ms 2.9504ms 338.9323 Ops/s 343.0322 Ops/s $\color{#d91a1a}-1.20\%$
test_sac_speed 9.4596ms 8.5215ms 117.3507 Ops/s 119.1986 Ops/s $\color{#d91a1a}-1.55\%$
test_redq_speed 11.6622ms 10.5104ms 95.1440 Ops/s 85.2750 Ops/s $\textbf{\color{#35bf28}+11.57\%}$
test_redq_deprec_speed 12.1778ms 11.6981ms 85.4838 Ops/s 86.6372 Ops/s $\color{#d91a1a}-1.33\%$
test_td3_speed 17.5458ms 8.5480ms 116.9871 Ops/s 119.7737 Ops/s $\color{#d91a1a}-2.33\%$
test_cql_speed 27.6644ms 26.0695ms 38.3590 Ops/s 39.1105 Ops/s $\color{#d91a1a}-1.92\%$
test_a2c_speed 6.0189ms 5.7439ms 174.0973 Ops/s 180.5788 Ops/s $\color{#d91a1a}-3.59\%$
test_ppo_speed 6.3928ms 6.0216ms 166.0695 Ops/s 170.2577 Ops/s $\color{#d91a1a}-2.46\%$
test_reinforce_speed 4.9469ms 4.6985ms 212.8347 Ops/s 222.2074 Ops/s $\color{#d91a1a}-4.22\%$
test_iql_speed 21.3043ms 20.1997ms 49.5057 Ops/s 51.4042 Ops/s $\color{#d91a1a}-3.69\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0060ms 2.8565ms 350.0812 Ops/s 349.7636 Ops/s $\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2450ms 0.6018ms 1.6618 KOps/s 1.6782 KOps/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7639ms 0.5821ms 1.7178 KOps/s 1.7353 KOps/s $\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0920ms 2.8539ms 350.3927 Ops/s 348.8197 Ops/s $\color{#35bf28}+0.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7298ms 0.5922ms 1.6886 KOps/s 1.7133 KOps/s $\color{#d91a1a}-1.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.8938ms 0.5742ms 1.7417 KOps/s 1.7711 KOps/s $\color{#d91a1a}-1.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6743ms 1.5470ms 646.4211 Ops/s 654.5390 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.7875ms 1.4833ms 674.1673 Ops/s 684.1929 Ops/s $\color{#d91a1a}-1.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0864ms 2.9734ms 336.3165 Ops/s 334.6113 Ops/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7264ms 0.7274ms 1.3748 KOps/s 1.3915 KOps/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8609ms 0.7040ms 1.4204 KOps/s 1.4237 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9617ms 2.8425ms 351.8038 Ops/s 351.4966 Ops/s $\color{#35bf28}+0.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4245ms 0.6023ms 1.6603 KOps/s 1.6889 KOps/s $\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7330ms 0.5811ms 1.7210 KOps/s 1.7446 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1306ms 2.8396ms 352.1683 Ops/s 348.1546 Ops/s $\color{#35bf28}+1.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7182ms 0.5920ms 1.6893 KOps/s 1.6993 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.9742ms 0.5807ms 1.7222 KOps/s 1.7655 KOps/s $\color{#d91a1a}-2.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0892ms 2.9918ms 334.2512 Ops/s 332.2514 Ops/s $\color{#35bf28}+0.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6355ms 0.7347ms 1.3611 KOps/s 1.3866 KOps/s $\color{#d91a1a}-1.84\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8807ms 0.7021ms 1.4243 KOps/s 1.4154 KOps/s $\color{#35bf28}+0.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1306s 7.4600ms 134.0482 Ops/s 132.3480 Ops/s $\color{#35bf28}+1.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 22.8148ms 15.6777ms 63.7849 Ops/s 54.9789 Ops/s $\textbf{\color{#35bf28}+16.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.3567ms 1.1935ms 837.8568 Ops/s 857.1143 Ops/s $\color{#d91a1a}-2.25\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1179s 7.2079ms 138.7362 Ops/s 136.7714 Ops/s $\color{#35bf28}+1.44\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1303s 18.0476ms 55.4090 Ops/s 62.7851 Ops/s $\textbf{\color{#d91a1a}-11.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2641ms 1.1962ms 835.9600 Ops/s 849.8542 Ops/s $\color{#d91a1a}-1.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1195s 7.6098ms 131.4097 Ops/s 131.3397 Ops/s $\color{#35bf28}+0.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.7086ms 15.9839ms 62.5628 Ops/s 61.4290 Ops/s $\color{#35bf28}+1.85\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.2335ms 1.6856ms 593.2719 Ops/s 650.9411 Ops/s $\textbf{\color{#d91a1a}-8.86\%}$

@vmoens vmoens added the bug Something isn't working label May 27, 2024
@vmoens vmoens merged commit 2ba9b4d into main May 28, 2024
53 of 58 checks passed
@vmoens vmoens deleted the fix-async-gym branch May 28, 2024 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot