Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix async gym when all reset #2144

Merged
merged 2 commits into from
Apr 30, 2024
Merged

[BugFix] Fix async gym when all reset #2144

merged 2 commits into from
Apr 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 30, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2144

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Unrelated Failure

As of commit b6214a2 with merge base 7b9305d (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 30, 2024
@vmoens vmoens added bug Something isn't working Environments Adds or modifies an environment wrapper labels Apr 30, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 54.0296ms 53.5940ms 18.6588 Ops/s 18.8667 Ops/s $\color{#d91a1a}-1.10\%$
test_sync 41.3322ms 30.0404ms 33.2885 Ops/s 29.5937 Ops/s $\textbf{\color{#35bf28}+12.49\%}$
test_async 53.0370ms 26.9145ms 37.1547 Ops/s 37.9150 Ops/s $\color{#d91a1a}-2.01\%$
test_simple 0.4045s 0.3533s 2.8307 Ops/s 2.9606 Ops/s $\color{#d91a1a}-4.39\%$
test_transformed 0.5404s 0.4909s 2.0370 Ops/s 2.0364 Ops/s $\color{#35bf28}+0.03\%$
test_serial 1.2591s 1.2089s 0.8272 Ops/s 0.8157 Ops/s $\color{#35bf28}+1.41\%$
test_parallel 1.0481s 1.0009s 0.9991 Ops/s 0.9809 Ops/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-True-True-True] 0.2224ms 21.5859μs 46.3266 KOps/s 46.8031 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-True-True-False] 47.1980μs 13.2836μs 75.2806 KOps/s 76.7974 KOps/s $\color{#d91a1a}-1.98\%$
test_step_mdp_speed[True-True-True-False-True] 43.6420μs 12.5841μs 79.4654 KOps/s 79.0289 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-True-True-False-False] 48.6800μs 7.7396μs 129.2054 KOps/s 128.9446 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-False-True-True] 52.9590μs 23.1111μs 43.2692 KOps/s 43.9978 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[True-True-False-True-False] 41.4880μs 14.5641μs 68.6619 KOps/s 69.0099 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-True-False-False-True] 42.8610μs 13.9914μs 71.4723 KOps/s 72.8458 KOps/s $\color{#d91a1a}-1.89\%$
test_step_mdp_speed[True-True-False-False-False] 62.3900μs 8.9347μs 111.9227 KOps/s 111.5583 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-False-True-True-True] 53.7210μs 24.5759μs 40.6903 KOps/s 41.6122 KOps/s $\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-False-True-True-False] 48.3900μs 15.9643μs 62.6400 KOps/s 63.2684 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[True-False-True-False-True] 37.2500μs 13.8578μs 72.1614 KOps/s 72.0223 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-True-False-False] 34.4450μs 8.9049μs 112.2973 KOps/s 112.3308 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-False-False-True-True] 51.2060μs 25.4444μs 39.3014 KOps/s 39.2540 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-False-False-True-False] 49.1720μs 17.0481μs 58.6576 KOps/s 58.6605 KOps/s $-0.00\%$
test_step_mdp_speed[True-False-False-False-True] 58.5800μs 15.2792μs 65.4486 KOps/s 66.0657 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-False-False-False-False] 42.3690μs 10.2089μs 97.9536 KOps/s 98.5113 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-True-True-True] 57.0970μs 24.3951μs 40.9919 KOps/s 41.4451 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-True-True-True-False] 38.0210μs 16.0511μs 62.3010 KOps/s 64.0293 KOps/s $\color{#d91a1a}-2.70\%$
test_step_mdp_speed[False-True-True-False-True] 45.5750μs 16.2491μs 61.5418 KOps/s 61.7077 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-True-True-False-False] 35.7970μs 10.2489μs 97.5714 KOps/s 97.5147 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-False-True-True] 62.5270μs 25.7667μs 38.8097 KOps/s 39.4987 KOps/s $\color{#d91a1a}-1.74\%$
test_step_mdp_speed[False-True-False-True-False] 53.3190μs 17.2475μs 57.9793 KOps/s 58.6220 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-True-False-False-True] 90.7300μs 17.8465μs 56.0335 KOps/s 58.0356 KOps/s $\color{#d91a1a}-3.45\%$
test_step_mdp_speed[False-True-False-False-False] 42.9800μs 11.4245μs 87.5314 KOps/s 87.8643 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[False-False-True-True-True] 53.7200μs 27.0363μs 36.9873 KOps/s 37.5672 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-False-True-True-False] 49.5330μs 18.6863μs 53.5151 KOps/s 54.8212 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[False-False-True-False-True] 43.3720μs 17.5336μs 57.0334 KOps/s 57.3322 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-True-False-False] 35.8880μs 11.5014μs 86.9458 KOps/s 86.9428 KOps/s $+0.00\%$
test_step_mdp_speed[False-False-False-True-True] 40.8770μs 28.3377μs 35.2886 KOps/s 35.5781 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-False-False-True-False] 47.8800μs 19.6668μs 50.8470 KOps/s 51.4941 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-False-False-False-True] 50.0330μs 18.5041μs 54.0420 KOps/s 54.4596 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-False-False-False] 36.7890μs 12.6678μs 78.9401 KOps/s 80.1554 KOps/s $\color{#d91a1a}-1.52\%$
test_values[generalized_advantage_estimate-True-True] 10.1162ms 9.6634ms 103.4828 Ops/s 107.4497 Ops/s $\color{#d91a1a}-3.69\%$
test_values[vec_generalized_advantage_estimate-True-True] 44.8390ms 35.5866ms 28.1004 Ops/s 28.4521 Ops/s $\color{#d91a1a}-1.24\%$
test_values[td0_return_estimate-False-False] 0.2367ms 0.1781ms 5.6150 KOps/s 5.8502 KOps/s $\color{#d91a1a}-4.02\%$
test_values[td1_return_estimate-False-False] 24.0583ms 23.7335ms 42.1346 Ops/s 43.2182 Ops/s $\color{#d91a1a}-2.51\%$
test_values[vec_td1_return_estimate-False-False] 36.3505ms 35.3595ms 28.2810 Ops/s 27.9670 Ops/s $\color{#35bf28}+1.12\%$
test_values[td_lambda_return_estimate-True-False] 34.5988ms 34.0579ms 29.3617 Ops/s 29.7441 Ops/s $\color{#d91a1a}-1.29\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.4207ms 35.3544ms 28.2851 Ops/s 28.4968 Ops/s $\color{#d91a1a}-0.74\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.7988ms 8.3949ms 119.1203 Ops/s 123.0996 Ops/s $\color{#d91a1a}-3.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4171ms 1.9909ms 502.2912 Ops/s 540.1919 Ops/s $\textbf{\color{#d91a1a}-7.02\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4388ms 0.3509ms 2.8496 KOps/s 2.8552 KOps/s $\color{#d91a1a}-0.20\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.3715ms 45.2062ms 22.1209 Ops/s 22.1334 Ops/s $\color{#d91a1a}-0.06\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.3808ms 3.1293ms 319.5558 Ops/s 330.2096 Ops/s $\color{#d91a1a}-3.23\%$
test_dqn_speed 7.1725ms 1.3866ms 721.1830 Ops/s 736.5834 Ops/s $\color{#d91a1a}-2.09\%$
test_ddpg_speed 3.5222ms 2.8611ms 349.5135 Ops/s 349.4191 Ops/s $\color{#35bf28}+0.03\%$
test_sac_speed 12.7912ms 8.5587ms 116.8397 Ops/s 108.8861 Ops/s $\textbf{\color{#35bf28}+7.30\%}$
test_redq_speed 14.4041ms 13.3347ms 74.9924 Ops/s 75.9098 Ops/s $\color{#d91a1a}-1.21\%$
test_redq_deprec_speed 0.1019s 14.6532ms 68.2446 Ops/s 74.4227 Ops/s $\textbf{\color{#d91a1a}-8.30\%}$
test_td3_speed 9.1081ms 8.4244ms 118.7026 Ops/s 118.1597 Ops/s $\color{#35bf28}+0.46\%$
test_cql_speed 39.9284ms 37.0280ms 27.0066 Ops/s 27.0968 Ops/s $\color{#d91a1a}-0.33\%$
test_a2c_speed 8.7329ms 7.5928ms 131.7042 Ops/s 133.3324 Ops/s $\color{#d91a1a}-1.22\%$
test_ppo_speed 8.1834ms 7.7226ms 129.4904 Ops/s 130.5495 Ops/s $\color{#d91a1a}-0.81\%$
test_reinforce_speed 7.6479ms 6.6856ms 149.5742 Ops/s 149.4877 Ops/s $\color{#35bf28}+0.06\%$
test_iql_speed 34.3969ms 32.9695ms 30.3310 Ops/s 30.1347 Ops/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2559ms 2.2087ms 452.7493 Ops/s 449.7194 Ops/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9688ms 0.4952ms 2.0193 KOps/s 2.0281 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7721ms 0.4691ms 2.1317 KOps/s 2.1174 KOps/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1805ms 2.1842ms 457.8282 Ops/s 461.1606 Ops/s $\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0383ms 0.4916ms 2.0343 KOps/s 2.0514 KOps/s $\color{#d91a1a}-0.83\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6537ms 0.4670ms 2.1412 KOps/s 2.1396 KOps/s $\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7948ms 1.2494ms 800.4066 Ops/s 810.0369 Ops/s $\color{#d91a1a}-1.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3994ms 1.1748ms 851.2252 Ops/s 856.9577 Ops/s $\color{#d91a1a}-0.67\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4474ms 2.3064ms 433.5708 Ops/s 445.0663 Ops/s $\color{#d91a1a}-2.58\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8896ms 0.6159ms 1.6236 KOps/s 1.6334 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8450ms 0.5896ms 1.6961 KOps/s 1.6961 KOps/s $+0.00\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3552ms 2.1445ms 466.3122 Ops/s 469.9984 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8371ms 0.4990ms 2.0040 KOps/s 2.0320 KOps/s $\color{#d91a1a}-1.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.5476ms 0.4742ms 2.1087 KOps/s 2.1111 KOps/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4546ms 2.1437ms 466.4800 Ops/s 459.5586 Ops/s $\color{#35bf28}+1.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5936ms 0.4895ms 2.0427 KOps/s 2.0713 KOps/s $\color{#d91a1a}-1.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6209ms 0.4640ms 2.1550 KOps/s 2.1624 KOps/s $\color{#d91a1a}-0.34\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9695ms 2.3536ms 424.8736 Ops/s 449.3071 Ops/s $\textbf{\color{#d91a1a}-5.44\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1393ms 0.6136ms 1.6298 KOps/s 1.6314 KOps/s $\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8329ms 0.5923ms 1.6885 KOps/s 1.6952 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1147s 7.9639ms 125.5672 Ops/s 125.0554 Ops/s $\color{#35bf28}+0.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.8430ms 12.2939ms 81.3414 Ops/s 83.9506 Ops/s $\color{#d91a1a}-3.11\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6843ms 1.0681ms 936.2301 Ops/s 970.0346 Ops/s $\color{#d91a1a}-3.48\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 99.8988ms 5.6253ms 177.7675 Ops/s 169.5532 Ops/s $\color{#35bf28}+4.84\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.5365ms 12.1303ms 82.4379 Ops/s 71.1826 Ops/s $\textbf{\color{#35bf28}+15.81\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8326ms 1.0794ms 926.4705 Ops/s 909.7204 Ops/s $\color{#35bf28}+1.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1094s 8.0519ms 124.1948 Ops/s 162.6931 Ops/s $\textbf{\color{#d91a1a}-23.66\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.4481ms 12.5642ms 79.5909 Ops/s 81.9417 Ops/s $\color{#d91a1a}-2.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0639ms 1.3872ms 720.8540 Ops/s 704.0187 Ops/s $\color{#35bf28}+2.39\%$

@vmoens vmoens merged commit 711a4ee into main Apr 30, 2024
52 of 57 checks passed
@vmoens vmoens deleted the fix-async-gym branch April 30, 2024 17:02
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1156s 0.1151s 8.6890 Ops/s 8.6496 Ops/s $\color{#35bf28}+0.46\%$
test_sync 0.1032s 0.1007s 9.9315 Ops/s 9.9644 Ops/s $\color{#d91a1a}-0.33\%$
test_async 0.1917s 96.6415ms 10.3475 Ops/s 10.4902 Ops/s $\color{#d91a1a}-1.36\%$
test_single_pixels 0.1268s 0.1261s 7.9303 Ops/s 7.9211 Ops/s $\color{#35bf28}+0.12\%$
test_sync_pixels 0.1380s 84.4772ms 11.8375 Ops/s 11.2954 Ops/s $\color{#35bf28}+4.80\%$
test_async_pixels 0.1591s 68.3316ms 14.6345 Ops/s 12.9195 Ops/s $\textbf{\color{#35bf28}+13.27\%}$
test_simple 0.8319s 0.7665s 1.3047 Ops/s 1.2631 Ops/s $\color{#35bf28}+3.29\%$
test_transformed 1.0787s 1.0164s 0.9839 Ops/s 0.9669 Ops/s $\color{#35bf28}+1.75\%$
test_serial 2.4822s 2.4205s 0.4131 Ops/s 0.3977 Ops/s $\color{#35bf28}+3.87\%$
test_parallel 2.3681s 2.3072s 0.4334 Ops/s 0.4331 Ops/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-True-True-True-True] 0.1019ms 32.9900μs 30.3122 KOps/s 29.6799 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-True-True-True-False] 40.8000μs 19.7117μs 50.7313 KOps/s 48.2659 KOps/s $\textbf{\color{#35bf28}+5.11\%}$
test_step_mdp_speed[True-True-True-False-True] 34.4200μs 18.8540μs 53.0392 KOps/s 52.3751 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-True-True-False-False] 35.4600μs 11.4093μs 87.6477 KOps/s 87.2174 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[True-True-False-True-True] 71.0610μs 35.2445μs 28.3732 KOps/s 27.9886 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-False-True-False] 40.9400μs 21.7502μs 45.9766 KOps/s 45.0136 KOps/s $\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-False-False-True] 40.0410μs 20.8681μs 47.9200 KOps/s 47.3492 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[True-True-False-False-False] 30.2410μs 13.2057μs 75.7249 KOps/s 73.9368 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-False-True-True-True] 0.1134ms 36.7967μs 27.1763 KOps/s 26.4743 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[True-False-True-True-False] 54.1420μs 23.7533μs 42.0994 KOps/s 41.8786 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[True-False-True-False-True] 43.7800μs 20.4262μs 48.9567 KOps/s 46.9356 KOps/s $\color{#35bf28}+4.31\%$
test_step_mdp_speed[True-False-True-False-False] 31.2910μs 13.2840μs 75.2787 KOps/s 73.6674 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[True-False-False-True-True] 74.8010μs 38.4468μs 26.0099 KOps/s 25.6753 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-False-False-True-False] 46.2910μs 25.1418μs 39.7743 KOps/s 38.9089 KOps/s $\color{#35bf28}+2.22\%$
test_step_mdp_speed[True-False-False-False-True] 47.4310μs 22.4424μs 44.5586 KOps/s 43.8580 KOps/s $\color{#35bf28}+1.60\%$
test_step_mdp_speed[True-False-False-False-False] 33.5120μs 15.0856μs 66.2884 KOps/s 66.0718 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-True-True-True] 72.6820μs 36.7452μs 27.2144 KOps/s 26.8168 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-True-True-True-False] 46.4510μs 23.7865μs 42.0406 KOps/s 41.6602 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-True-False-True] 47.5410μs 24.7935μs 40.3331 KOps/s 40.0722 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-True-False-False] 35.0800μs 14.9643μs 66.8256 KOps/s 65.1498 KOps/s $\color{#35bf28}+2.57\%$
test_step_mdp_speed[False-True-False-True-True] 62.7210μs 38.2543μs 26.1409 KOps/s 25.5022 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-True-False-True-False] 66.7200μs 25.2545μs 39.5968 KOps/s 39.1097 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-True-False-False-True] 44.2500μs 26.2406μs 38.1089 KOps/s 37.2119 KOps/s $\color{#35bf28}+2.41\%$
test_step_mdp_speed[False-True-False-False-False] 39.5810μs 16.8704μs 59.2753 KOps/s 58.6275 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-True-True-True] 74.1710μs 40.8356μs 24.4884 KOps/s 24.7553 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-True-True-False] 42.9510μs 27.5318μs 36.3217 KOps/s 36.3944 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-True-False-True] 51.6710μs 26.2460μs 38.1010 KOps/s 36.9524 KOps/s $\color{#35bf28}+3.11\%$
test_step_mdp_speed[False-False-True-False-False] 40.9210μs 16.8691μs 59.2801 KOps/s 58.9766 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-False-True-True] 59.8210μs 42.5897μs 23.4799 KOps/s 23.0451 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-False-True-False] 55.3100μs 29.2458μs 34.1930 KOps/s 33.3644 KOps/s $\color{#35bf28}+2.48\%$
test_step_mdp_speed[False-False-False-False-True] 49.8210μs 27.9422μs 35.7882 KOps/s 35.5755 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-False-False-False] 43.6800μs 18.7398μs 53.3625 KOps/s 52.7604 KOps/s $\color{#35bf28}+1.14\%$
test_values[generalized_advantage_estimate-True-True] 24.3171ms 23.8376ms 41.9505 Ops/s 42.4993 Ops/s $\color{#d91a1a}-1.29\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.7256ms 3.1913ms 313.3550 Ops/s 302.1901 Ops/s $\color{#35bf28}+3.69\%$
test_values[td0_return_estimate-False-False] 90.9910μs 63.2660μs 15.8063 KOps/s 16.0901 KOps/s $\color{#d91a1a}-1.76\%$
test_values[td1_return_estimate-False-False] 54.4375ms 51.5957ms 19.3814 Ops/s 19.8914 Ops/s $\color{#d91a1a}-2.56\%$
test_values[vec_td1_return_estimate-False-False] 2.0436ms 1.7468ms 572.4809 Ops/s 572.8198 Ops/s $\color{#d91a1a}-0.06\%$
test_values[td_lambda_return_estimate-True-False] 87.0665ms 82.1878ms 12.1673 Ops/s 12.3164 Ops/s $\color{#d91a1a}-1.21\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1163ms 1.7443ms 573.2816 Ops/s 573.5764 Ops/s $\color{#d91a1a}-0.05\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.5633ms 22.4564ms 44.5308 Ops/s 42.5371 Ops/s $\color{#35bf28}+4.69\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8783ms 0.6801ms 1.4703 KOps/s 1.4746 KOps/s $\color{#d91a1a}-0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6887ms 0.6333ms 1.5790 KOps/s 1.5795 KOps/s $\color{#d91a1a}-0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4619ms 1.4371ms 695.8511 Ops/s 695.5921 Ops/s $\color{#35bf28}+0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9443ms 0.6507ms 1.5368 KOps/s 1.5364 KOps/s $\color{#35bf28}+0.03\%$
test_dqn_speed 1.8269ms 1.4818ms 674.8761 Ops/s 683.3311 Ops/s $\color{#d91a1a}-1.24\%$
test_ddpg_speed 3.2064ms 3.0026ms 333.0434 Ops/s 340.4802 Ops/s $\color{#d91a1a}-2.18\%$
test_sac_speed 9.1623ms 8.4595ms 118.2104 Ops/s 117.2278 Ops/s $\color{#35bf28}+0.84\%$
test_redq_speed 99.5279ms 11.2957ms 88.5293 Ops/s 94.9856 Ops/s $\textbf{\color{#d91a1a}-6.80\%}$
test_redq_deprec_speed 12.1175ms 11.5699ms 86.4310 Ops/s 86.4154 Ops/s $\color{#35bf28}+0.02\%$
test_td3_speed 8.4556ms 8.3429ms 119.8617 Ops/s 119.8482 Ops/s $\color{#35bf28}+0.01\%$
test_cql_speed 27.3992ms 26.1762ms 38.2026 Ops/s 38.3692 Ops/s $\color{#d91a1a}-0.43\%$
test_a2c_speed 6.0107ms 5.7622ms 173.5450 Ops/s 174.0194 Ops/s $\color{#d91a1a}-0.27\%$
test_ppo_speed 6.2929ms 6.0402ms 165.5566 Ops/s 165.2716 Ops/s $\color{#35bf28}+0.17\%$
test_reinforce_speed 4.9763ms 4.7137ms 212.1460 Ops/s 211.3517 Ops/s $\color{#35bf28}+0.38\%$
test_iql_speed 20.9247ms 20.2339ms 49.4221 Ops/s 48.8982 Ops/s $\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8770ms 2.7818ms 359.4832 Ops/s 361.7090 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7091ms 0.6003ms 1.6659 KOps/s 1.6562 KOps/s $\color{#35bf28}+0.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5633ms 0.5804ms 1.7229 KOps/s 1.7366 KOps/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9752ms 2.8240ms 354.1134 Ops/s 356.1284 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7113ms 0.5925ms 1.6876 KOps/s 1.6866 KOps/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.6653ms 0.5713ms 1.7504 KOps/s 1.7662 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6586ms 1.4995ms 666.8736 Ops/s 667.6608 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.2755ms 1.4489ms 690.1980 Ops/s 700.7227 Ops/s $\color{#d91a1a}-1.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0039ms 2.9231ms 342.1014 Ops/s 343.7656 Ops/s $\color{#d91a1a}-0.48\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6586ms 0.7337ms 1.3630 KOps/s 1.3743 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8450ms 0.7088ms 1.4108 KOps/s 1.4173 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9233ms 2.7948ms 357.8078 Ops/s 358.5110 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4651ms 0.6006ms 1.6649 KOps/s 1.6687 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7340ms 0.5784ms 1.7290 KOps/s 1.7214 KOps/s $\color{#35bf28}+0.45\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1080ms 2.8316ms 353.1619 Ops/s 354.2552 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7149ms 0.5929ms 1.6865 KOps/s 1.6822 KOps/s $\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7635ms 0.5742ms 1.7416 KOps/s 1.7304 KOps/s $\color{#35bf28}+0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0410ms 2.9359ms 340.6059 Ops/s 344.1338 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6266ms 0.7328ms 1.3646 KOps/s 1.3834 KOps/s $\color{#d91a1a}-1.36\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9108ms 0.7152ms 1.3982 KOps/s 1.4095 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1326s 9.8419ms 101.6064 Ops/s 101.8250 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3427ms 14.9594ms 66.8478 Ops/s 67.4860 Ops/s $\color{#d91a1a}-0.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.9776ms 1.3616ms 734.4236 Ops/s 758.2900 Ops/s $\color{#d91a1a}-3.15\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1220s 7.2250ms 138.4092 Ops/s 140.0894 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.3384ms 14.9390ms 66.9390 Ops/s 67.8993 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2910ms 1.2167ms 821.8676 Ops/s 788.1544 Ops/s $\color{#35bf28}+4.28\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1253s 7.6027ms 131.5329 Ops/s 133.9629 Ops/s $\color{#d91a1a}-1.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.5660ms 15.2846ms 65.4255 Ops/s 57.4462 Ops/s $\textbf{\color{#35bf28}+13.89\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5393ms 1.5481ms 645.9507 Ops/s 578.5240 Ops/s $\textbf{\color{#35bf28}+11.65\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Environments Adds or modifies an environment wrapper
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot