Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Clearer separation between single_task and share_individual_td #2026

Merged
merged 7 commits into from
Mar 20, 2024
Merged

[Refactor] Clearer separation between single_task and share_individual_td #2026

merged 7 commits into from
Mar 20, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Mar 20, 2024

Tests if the data that results from each env is stackable when share_individual_td is not specified. If the result is stackable, the variable is set to False.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 20, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2026

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 21 Unrelated Failures

As of commit 3dbeb45 with merge base d937478 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 20, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 20, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.8495ms 54.6425ms 18.3008 Ops/s 17.6398 Ops/s $\color{#35bf28}+3.75\%$
test_sync 35.1739ms 30.3752ms 32.9216 Ops/s 33.5245 Ops/s $\color{#d91a1a}-1.80\%$
test_async 57.4463ms 28.3164ms 35.3152 Ops/s 35.5681 Ops/s $\color{#d91a1a}-0.71\%$
test_simple 0.4005s 0.3417s 2.9261 Ops/s 3.0168 Ops/s $\color{#d91a1a}-3.01\%$
test_transformed 0.5285s 0.4773s 2.0952 Ops/s 2.1104 Ops/s $\color{#d91a1a}-0.72\%$
test_serial 1.2569s 1.2016s 0.8322 Ops/s 0.8348 Ops/s $\color{#d91a1a}-0.31\%$
test_parallel 1.0773s 1.0318s 0.9692 Ops/s 0.9650 Ops/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-True-True-True-True] 0.1585ms 20.9724μs 47.6818 KOps/s 46.7763 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-True-True-True-False] 40.8360μs 12.7353μs 78.5222 KOps/s 75.7820 KOps/s $\color{#35bf28}+3.62\%$
test_step_mdp_speed[True-True-True-False-True] 34.0530μs 12.3343μs 81.0749 KOps/s 78.4129 KOps/s $\color{#35bf28}+3.39\%$
test_step_mdp_speed[True-True-True-False-False] 34.1840μs 7.4481μs 134.2617 KOps/s 128.4317 KOps/s $\color{#35bf28}+4.54\%$
test_step_mdp_speed[True-True-False-True-True] 48.3000μs 22.1984μs 45.0482 KOps/s 44.2555 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-False-True-False] 38.0410μs 14.0798μs 71.0236 KOps/s 69.9796 KOps/s $\color{#35bf28}+1.49\%$
test_step_mdp_speed[True-True-False-False-True] 39.2040μs 13.5458μs 73.8236 KOps/s 71.9234 KOps/s $\color{#35bf28}+2.64\%$
test_step_mdp_speed[True-True-False-False-False] 54.1910μs 8.6025μs 116.2452 KOps/s 110.4682 KOps/s $\textbf{\color{#35bf28}+5.23\%}$
test_step_mdp_speed[True-False-True-True-True] 50.0640μs 23.5798μs 42.4093 KOps/s 40.8562 KOps/s $\color{#35bf28}+3.80\%$
test_step_mdp_speed[True-False-True-True-False] 38.5720μs 15.3324μs 65.2214 KOps/s 62.2770 KOps/s $\color{#35bf28}+4.73\%$
test_step_mdp_speed[True-False-True-False-True] 43.1200μs 13.5063μs 74.0393 KOps/s 72.0730 KOps/s $\color{#35bf28}+2.73\%$
test_step_mdp_speed[True-False-True-False-False] 30.4370μs 8.6482μs 115.6312 KOps/s 110.8350 KOps/s $\color{#35bf28}+4.33\%$
test_step_mdp_speed[True-False-False-True-True] 53.6500μs 24.7449μs 40.4124 KOps/s 39.1005 KOps/s $\color{#35bf28}+3.36\%$
test_step_mdp_speed[True-False-False-True-False] 45.7060μs 16.4941μs 60.6276 KOps/s 58.8637 KOps/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-False-False-False-True] 41.1770μs 14.5951μs 68.5160 KOps/s 66.9100 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[True-False-False-False-False] 32.0890μs 9.8265μs 101.7655 KOps/s 98.3189 KOps/s $\color{#35bf28}+3.51\%$
test_step_mdp_speed[False-True-True-True-True] 64.4200μs 23.3615μs 42.8055 KOps/s 41.4751 KOps/s $\color{#35bf28}+3.21\%$
test_step_mdp_speed[False-True-True-True-False] 39.2340μs 15.2676μs 65.4984 KOps/s 63.2654 KOps/s $\color{#35bf28}+3.53\%$
test_step_mdp_speed[False-True-True-False-True] 58.3090μs 15.5549μs 64.2883 KOps/s 62.2076 KOps/s $\color{#35bf28}+3.34\%$
test_step_mdp_speed[False-True-True-False-False] 37.4400μs 9.8551μs 101.4706 KOps/s 99.3336 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-True-False-True-True] 48.5610μs 25.0594μs 39.9052 KOps/s 38.5574 KOps/s $\color{#35bf28}+3.50\%$
test_step_mdp_speed[False-True-False-True-False] 49.2920μs 16.4782μs 60.6862 KOps/s 58.3928 KOps/s $\color{#35bf28}+3.93\%$
test_step_mdp_speed[False-True-False-False-True] 41.5870μs 16.6245μs 60.1521 KOps/s 57.3820 KOps/s $\color{#35bf28}+4.83\%$
test_step_mdp_speed[False-True-False-False-False] 36.6580μs 11.0272μs 90.6848 KOps/s 87.9857 KOps/s $\color{#35bf28}+3.07\%$
test_step_mdp_speed[False-False-True-True-True] 64.9610μs 25.8804μs 38.6393 KOps/s 37.4532 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[False-False-True-True-False] 48.5410μs 17.7135μs 56.4542 KOps/s 55.3210 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[False-False-True-False-True] 46.5570μs 16.6624μs 60.0155 KOps/s 58.4947 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[False-False-True-False-False] 27.9020μs 10.9958μs 90.9439 KOps/s 86.6845 KOps/s $\color{#35bf28}+4.91\%$
test_step_mdp_speed[False-False-False-True-True] 53.5300μs 26.7783μs 37.3436 KOps/s 36.4512 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[False-False-False-True-False] 53.4000μs 18.8508μs 53.0481 KOps/s 51.0882 KOps/s $\color{#35bf28}+3.84\%$
test_step_mdp_speed[False-False-False-False-True] 55.6730μs 17.6162μs 56.7659 KOps/s 53.7851 KOps/s $\textbf{\color{#35bf28}+5.54\%}$
test_step_mdp_speed[False-False-False-False-False] 38.0110μs 12.0732μs 82.8281 KOps/s 80.7128 KOps/s $\color{#35bf28}+2.62\%$
test_values[generalized_advantage_estimate-True-True] 9.8743ms 9.3130ms 107.3772 Ops/s 108.4929 Ops/s $\color{#d91a1a}-1.03\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.0662ms 35.7682ms 27.9578 Ops/s 29.8799 Ops/s $\textbf{\color{#d91a1a}-6.43\%}$
test_values[td0_return_estimate-False-False] 0.2227ms 0.1835ms 5.4505 KOps/s 5.8157 KOps/s $\textbf{\color{#d91a1a}-6.28\%}$
test_values[td1_return_estimate-False-False] 26.3835ms 23.7039ms 42.1871 Ops/s 43.1059 Ops/s $\color{#d91a1a}-2.13\%$
test_values[vec_td1_return_estimate-False-False] 37.2618ms 35.7200ms 27.9955 Ops/s 29.8817 Ops/s $\textbf{\color{#d91a1a}-6.31\%}$
test_values[td_lambda_return_estimate-True-False] 39.0514ms 33.7328ms 29.6448 Ops/s 29.8007 Ops/s $\color{#d91a1a}-0.52\%$
test_values[vec_td_lambda_return_estimate-True-False] 39.0098ms 35.6029ms 28.0876 Ops/s 29.7833 Ops/s $\textbf{\color{#d91a1a}-5.69\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.3037ms 8.1190ms 123.1680 Ops/s 125.8812 Ops/s $\color{#d91a1a}-2.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.5407ms 2.0354ms 491.2975 Ops/s 518.3191 Ops/s $\textbf{\color{#d91a1a}-5.21\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4666ms 0.3536ms 2.8283 KOps/s 2.8659 KOps/s $\color{#d91a1a}-1.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 54.2128ms 45.4418ms 22.0062 Ops/s 24.6598 Ops/s $\textbf{\color{#d91a1a}-10.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8572ms 3.0536ms 327.4843 Ops/s 328.5616 Ops/s $\color{#d91a1a}-0.33\%$
test_dqn_speed 7.2560ms 1.3757ms 726.9078 Ops/s 736.3662 Ops/s $\color{#d91a1a}-1.28\%$
test_ddpg_speed 3.3988ms 2.7325ms 365.9614 Ops/s 371.4989 Ops/s $\color{#d91a1a}-1.49\%$
test_sac_speed 9.9906ms 8.3929ms 119.1486 Ops/s 121.5215 Ops/s $\color{#d91a1a}-1.95\%$
test_redq_speed 20.3793ms 13.7215ms 72.8784 Ops/s 75.5967 Ops/s $\color{#d91a1a}-3.60\%$
test_redq_deprec_speed 14.2910ms 13.5226ms 73.9503 Ops/s 71.1053 Ops/s $\color{#35bf28}+4.00\%$
test_td3_speed 15.4221ms 8.2629ms 121.0223 Ops/s 119.3265 Ops/s $\color{#35bf28}+1.42\%$
test_cql_speed 37.0740ms 36.4869ms 27.4071 Ops/s 27.3832 Ops/s $\color{#35bf28}+0.09\%$
test_a2c_speed 8.3533ms 7.3805ms 135.4926 Ops/s 129.8079 Ops/s $\color{#35bf28}+4.38\%$
test_ppo_speed 8.3432ms 7.6811ms 130.1903 Ops/s 127.0827 Ops/s $\color{#35bf28}+2.45\%$
test_reinforce_speed 7.7421ms 6.6278ms 150.8797 Ops/s 149.2999 Ops/s $\color{#35bf28}+1.06\%$
test_iql_speed 34.0009ms 33.0808ms 30.2290 Ops/s 30.2823 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.5340ms 2.3103ms 432.8432 Ops/s 437.1043 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9636ms 0.5014ms 1.9942 KOps/s 1.7813 KOps/s $\textbf{\color{#35bf28}+11.95\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 83.8499ms 0.5322ms 1.8791 KOps/s 2.1086 KOps/s $\textbf{\color{#d91a1a}-10.88\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6484ms 2.3687ms 422.1729 Ops/s 428.9774 Ops/s $\color{#d91a1a}-1.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9167ms 0.4929ms 2.0286 KOps/s 2.0564 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7876ms 0.4675ms 2.1392 KOps/s 2.1144 KOps/s $\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0460ms 1.2961ms 771.5530 Ops/s 786.0894 Ops/s $\color{#d91a1a}-1.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6563ms 1.2299ms 813.1039 Ops/s 834.1266 Ops/s $\color{#d91a1a}-2.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5997ms 2.3966ms 417.2612 Ops/s 410.5704 Ops/s $\color{#35bf28}+1.63\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7242ms 0.6108ms 1.6371 KOps/s 1.6397 KOps/s $\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.8167ms 0.5956ms 1.6789 KOps/s 1.6939 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6198ms 2.2814ms 438.3369 Ops/s 436.1631 Ops/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7949ms 0.5008ms 1.9968 KOps/s 1.9976 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8046ms 0.4756ms 2.1026 KOps/s 2.1055 KOps/s $\color{#d91a1a}-0.14\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6407ms 2.3115ms 432.6105 Ops/s 420.0349 Ops/s $\color{#35bf28}+2.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5956ms 0.4904ms 2.0390 KOps/s 2.0356 KOps/s $\color{#35bf28}+0.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7775ms 0.4758ms 2.1016 KOps/s 2.0748 KOps/s $\color{#35bf28}+1.29\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6191ms 2.4116ms 414.6566 Ops/s 404.7612 Ops/s $\color{#35bf28}+2.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1608ms 0.6238ms 1.6031 KOps/s 1.6228 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9568ms 0.5970ms 1.6749 KOps/s 1.6755 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1087s 7.5271ms 132.8537 Ops/s 133.8149 Ops/s $\color{#d91a1a}-0.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.9317ms 11.9776ms 83.4893 Ops/s 84.2747 Ops/s $\color{#d91a1a}-0.93\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8331ms 1.0741ms 930.9751 Ops/s 916.1364 Ops/s $\color{#35bf28}+1.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 94.2029ms 7.2097ms 138.7012 Ops/s 139.6950 Ops/s $\color{#d91a1a}-0.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.7433ms 11.9506ms 83.6779 Ops/s 84.2477 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7717ms 1.0523ms 950.2700 Ops/s 884.1586 Ops/s $\textbf{\color{#35bf28}+7.48\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 94.8328ms 5.7989ms 172.4474 Ops/s 173.5321 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.7224ms 12.2365ms 81.7227 Ops/s 81.1929 Ops/s $\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8542ms 1.3511ms 740.1255 Ops/s 316.6271 Ops/s $\textbf{\color{#35bf28}+133.75\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 20, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 98.2250ms 97.8886ms 10.2157 Ops/s 9.3653 Ops/s $\textbf{\color{#35bf28}+9.08\%}$
test_sync 91.4547ms 88.8329ms 11.2571 Ops/s 11.3223 Ops/s $\color{#d91a1a}-0.58\%$
test_async 0.1721s 83.7904ms 11.9345 Ops/s 11.5040 Ops/s $\color{#35bf28}+3.74\%$
test_single_pixels 0.1822s 0.1163s 8.5990 Ops/s 8.8405 Ops/s $\color{#d91a1a}-2.73\%$
test_sync_pixels 69.9734ms 68.4098ms 14.6178 Ops/s 15.0495 Ops/s $\color{#d91a1a}-2.87\%$
test_async_pixels 0.1085s 61.6915ms 16.2097 Ops/s 17.5789 Ops/s $\textbf{\color{#d91a1a}-7.79\%}$
test_simple 0.7113s 0.6531s 1.5312 Ops/s 1.4801 Ops/s $\color{#35bf28}+3.45\%$
test_transformed 0.9071s 0.8518s 1.1739 Ops/s 1.1227 Ops/s $\color{#35bf28}+4.57\%$
test_serial 2.0797s 2.0167s 0.4958 Ops/s 0.4513 Ops/s $\textbf{\color{#35bf28}+9.88\%}$
test_parallel 1.8373s 1.8181s 0.5500 Ops/s 0.5502 Ops/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-True-True-True-True] 0.1114ms 34.1754μs 29.2608 KOps/s 29.7112 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-True-True-True-False] 41.5610μs 19.3981μs 51.5513 KOps/s 49.8406 KOps/s $\color{#35bf28}+3.43\%$
test_step_mdp_speed[True-True-True-False-True] 37.0500μs 18.4137μs 54.3074 KOps/s 53.2314 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[True-True-True-False-False] 31.8800μs 11.2881μs 88.5889 KOps/s 88.7981 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-False-True-True] 62.5100μs 34.2721μs 29.1783 KOps/s 28.8034 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-True-False-True-False] 49.5100μs 20.9571μs 47.7164 KOps/s 46.2563 KOps/s $\color{#35bf28}+3.16\%$
test_step_mdp_speed[True-True-False-False-True] 37.6300μs 19.7428μs 50.6514 KOps/s 49.3963 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[True-True-False-False-False] 33.9810μs 12.6809μs 78.8587 KOps/s 75.9146 KOps/s $\color{#35bf28}+3.88\%$
test_step_mdp_speed[True-False-True-True-True] 62.9300μs 35.0702μs 28.5142 KOps/s 27.0663 KOps/s $\textbf{\color{#35bf28}+5.35\%}$
test_step_mdp_speed[True-False-True-True-False] 46.3710μs 22.6704μs 44.1104 KOps/s 42.4427 KOps/s $\color{#35bf28}+3.93\%$
test_step_mdp_speed[True-False-True-False-True] 39.6610μs 19.7296μs 50.6852 KOps/s 49.4487 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[True-False-True-False-False] 38.0700μs 12.5388μs 79.7525 KOps/s 75.9971 KOps/s $\color{#35bf28}+4.94\%$
test_step_mdp_speed[True-False-False-True-True] 0.1029ms 37.4023μs 26.7363 KOps/s 26.1142 KOps/s $\color{#35bf28}+2.38\%$
test_step_mdp_speed[True-False-False-True-False] 46.9110μs 24.3310μs 41.0998 KOps/s 39.2938 KOps/s $\color{#35bf28}+4.60\%$
test_step_mdp_speed[True-False-False-False-True] 40.3010μs 21.4746μs 46.5666 KOps/s 45.2630 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[True-False-False-False-False] 80.9300μs 14.2398μs 70.2258 KOps/s 65.7401 KOps/s $\textbf{\color{#35bf28}+6.82\%}$
test_step_mdp_speed[False-True-True-True-True] 63.9900μs 35.2127μs 28.3989 KOps/s 27.3613 KOps/s $\color{#35bf28}+3.79\%$
test_step_mdp_speed[False-True-True-True-False] 57.7410μs 22.7347μs 43.9855 KOps/s 42.2609 KOps/s $\color{#35bf28}+4.08\%$
test_step_mdp_speed[False-True-True-False-True] 0.1150ms 23.6412μs 42.2991 KOps/s 41.4299 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-True-True-False-False] 94.2510μs 14.3270μs 69.7982 KOps/s 66.0164 KOps/s $\textbf{\color{#35bf28}+5.73\%}$
test_step_mdp_speed[False-True-False-True-True] 66.2810μs 38.1213μs 26.2320 KOps/s 25.7126 KOps/s $\color{#35bf28}+2.02\%$
test_step_mdp_speed[False-True-False-True-False] 55.9800μs 26.0352μs 38.4095 KOps/s 39.1768 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[False-True-False-False-True] 50.7100μs 25.0832μs 39.8673 KOps/s 38.3030 KOps/s $\color{#35bf28}+4.08\%$
test_step_mdp_speed[False-True-False-False-False] 40.9910μs 16.4677μs 60.7248 KOps/s 59.5958 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-True-True-True] 67.3510μs 39.9410μs 25.0369 KOps/s 24.6506 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[False-False-True-True-False] 47.3700μs 26.0357μs 38.4088 KOps/s 36.4558 KOps/s $\textbf{\color{#35bf28}+5.36\%}$
test_step_mdp_speed[False-False-True-False-True] 93.5010μs 25.4545μs 39.2859 KOps/s 38.6770 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[False-False-True-False-False] 33.9800μs 16.2932μs 61.3753 KOps/s 59.2461 KOps/s $\color{#35bf28}+3.59\%$
test_step_mdp_speed[False-False-False-True-True] 65.1100μs 40.3537μs 24.7809 KOps/s 23.6710 KOps/s $\color{#35bf28}+4.69\%$
test_step_mdp_speed[False-False-False-True-False] 51.1110μs 28.3432μs 35.2818 KOps/s 34.2593 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-False-False-False-True] 48.9600μs 26.7675μs 37.3588 KOps/s 37.1637 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-False-False-False] 87.6310μs 18.1986μs 54.9492 KOps/s 54.0978 KOps/s $\color{#35bf28}+1.57\%$
test_values[generalized_advantage_estimate-True-True] 24.4229ms 23.7513ms 42.1029 Ops/s 41.5176 Ops/s $\color{#35bf28}+1.41\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.8627ms 3.1807ms 314.3937 Ops/s 310.2315 Ops/s $\color{#35bf28}+1.34\%$
test_values[td0_return_estimate-False-False] 88.4700μs 60.5713μs 16.5095 KOps/s 15.9193 KOps/s $\color{#35bf28}+3.71\%$
test_values[td1_return_estimate-False-False] 52.0009ms 50.6681ms 19.7363 Ops/s 19.0909 Ops/s $\color{#35bf28}+3.38\%$
test_values[vec_td1_return_estimate-False-False] 2.1005ms 1.7306ms 577.8185 Ops/s 571.3169 Ops/s $\color{#35bf28}+1.14\%$
test_values[td_lambda_return_estimate-True-False] 83.2912ms 80.6218ms 12.4036 Ops/s 12.0564 Ops/s $\color{#35bf28}+2.88\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0578ms 1.7408ms 574.4451 Ops/s 572.3835 Ops/s $\color{#35bf28}+0.36\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.0750ms 22.7318ms 43.9912 Ops/s 42.3148 Ops/s $\color{#35bf28}+3.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8475ms 0.6732ms 1.4854 KOps/s 1.4492 KOps/s $\color{#35bf28}+2.49\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7208ms 0.6453ms 1.5496 KOps/s 1.5794 KOps/s $\color{#d91a1a}-1.89\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5147ms 1.4401ms 694.3777 Ops/s 693.2999 Ops/s $\color{#35bf28}+0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9074ms 0.6532ms 1.5309 KOps/s 1.5289 KOps/s $\color{#35bf28}+0.13\%$
test_dqn_speed 8.4375ms 1.4011ms 713.7149 Ops/s 699.9483 Ops/s $\color{#35bf28}+1.97\%$
test_ddpg_speed 3.2313ms 2.6452ms 378.0470 Ops/s 372.6036 Ops/s $\color{#35bf28}+1.46\%$
test_sac_speed 8.4413ms 7.9098ms 126.4257 Ops/s 125.1210 Ops/s $\color{#35bf28}+1.04\%$
test_redq_speed 11.0190ms 10.1912ms 98.1236 Ops/s 97.7864 Ops/s $\color{#35bf28}+0.34\%$
test_redq_deprec_speed 11.5582ms 10.9130ms 91.6338 Ops/s 93.6940 Ops/s $\color{#d91a1a}-2.20\%$
test_td3_speed 8.0410ms 7.7444ms 129.1255 Ops/s 127.1789 Ops/s $\color{#35bf28}+1.53\%$
test_cql_speed 25.6728ms 24.7065ms 40.4752 Ops/s 40.0649 Ops/s $\color{#35bf28}+1.02\%$
test_a2c_speed 83.5441ms 5.7941ms 172.5892 Ops/s 185.3097 Ops/s $\textbf{\color{#d91a1a}-6.86\%}$
test_ppo_speed 6.1223ms 5.7129ms 175.0433 Ops/s 176.1572 Ops/s $\color{#d91a1a}-0.63\%$
test_reinforce_speed 4.6513ms 4.3990ms 227.3262 Ops/s 228.3693 Ops/s $\color{#d91a1a}-0.46\%$
test_iql_speed 19.5888ms 19.0379ms 52.5267 Ops/s 53.4811 Ops/s $\color{#d91a1a}-1.78\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9582ms 2.8393ms 352.1976 Ops/s 350.4192 Ops/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6755ms 0.5382ms 1.8580 KOps/s 1.8657 KOps/s $\color{#d91a1a}-0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7239ms 0.5143ms 1.9444 KOps/s 1.9411 KOps/s $\color{#35bf28}+0.17\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1540ms 2.8560ms 350.1361 Ops/s 344.5063 Ops/s $\color{#35bf28}+1.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6524ms 0.5288ms 1.8912 KOps/s 1.8877 KOps/s $\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.1510ms 0.5069ms 1.9726 KOps/s 1.9598 KOps/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5323ms 1.4509ms 689.2358 Ops/s 679.9804 Ops/s $\color{#35bf28}+1.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5124ms 1.3837ms 722.6913 Ops/s 716.6717 Ops/s $\color{#35bf28}+0.84\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1245ms 2.9658ms 337.1792 Ops/s 333.4413 Ops/s $\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2414ms 0.6520ms 1.5337 KOps/s 1.5168 KOps/s $\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7471ms 0.6283ms 1.5915 KOps/s 1.5710 KOps/s $\color{#35bf28}+1.31\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9543ms 2.8231ms 354.2178 Ops/s 351.9838 Ops/s $\color{#35bf28}+0.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6644ms 0.5369ms 1.8626 KOps/s 1.8633 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5054ms 0.5175ms 1.9324 KOps/s 1.9538 KOps/s $\color{#d91a1a}-1.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9807ms 2.8186ms 354.7866 Ops/s 347.1389 Ops/s $\color{#35bf28}+2.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6101ms 0.5272ms 1.8969 KOps/s 1.8855 KOps/s $\color{#35bf28}+0.60\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6621ms 0.5106ms 1.9583 KOps/s 1.9377 KOps/s $\color{#35bf28}+1.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0703ms 2.9608ms 337.7423 Ops/s 334.1597 Ops/s $\color{#35bf28}+1.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2951ms 0.6536ms 1.5300 KOps/s 1.4927 KOps/s $\color{#35bf28}+2.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7746ms 0.6317ms 1.5831 KOps/s 1.5621 KOps/s $\color{#35bf28}+1.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1031s 8.6069ms 116.1857 Ops/s 112.6537 Ops/s $\color{#35bf28}+3.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.4967ms 14.1651ms 70.5962 Ops/s 67.9225 Ops/s $\color{#35bf28}+3.94\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2372ms 1.0793ms 926.5110 Ops/s 828.9754 Ops/s $\textbf{\color{#35bf28}+11.77\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1001s 6.6048ms 151.4061 Ops/s 149.0337 Ops/s $\color{#35bf28}+1.59\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.3807ms 14.1293ms 70.7747 Ops/s 60.2319 Ops/s $\textbf{\color{#35bf28}+17.50\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.0841ms 1.0914ms 916.2163 Ops/s 775.4136 Ops/s $\textbf{\color{#35bf28}+18.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1030s 8.9523ms 111.7030 Ops/s 141.0410 Ops/s $\textbf{\color{#d91a1a}-20.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.8396ms 14.6912ms 68.0677 Ops/s 66.5862 Ops/s $\color{#35bf28}+2.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5123ms 1.4200ms 704.2501 Ops/s 619.8572 Ops/s $\textbf{\color{#35bf28}+13.61\%}$

@vmoens vmoens added the Refactoring Refactoring of an existing feature label Mar 20, 2024
@vmoens vmoens merged commit effd868 into main Mar 20, 2024
6 checks passed
@vmoens vmoens deleted the refactor-single-task branch March 20, 2024 15:19
vmoens added a commit that referenced this pull request Mar 25, 2024
@vmoens
[Refactor] Clearer separation between single_task and share_individua…
0efc380 
…l_td (#2026)

(cherry picked from commit effd868)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot