Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Use a RL-specific NO_DEFAULT instead of TD's one #2367

Merged
merged 1 commit into from
Aug 6, 2024
Merged

[BugFix] Use a RL-specific NO_DEFAULT instead of TD's one #2367

merged 1 commit into from
Aug 6, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 6, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 6, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2367

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Cancelled Job, 14 Unrelated Failures

As of commit fdb8e12 with merge base 76213f7 (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 6, 2024
@vmoens vmoens added the bug Something isn't working label Aug 6, 2024
@vmoens vmoens merged commit 788710f into main Aug 6, 2024
45 of 58 checks passed
@vmoens vmoens deleted the fix-no-default branch August 6, 2024 00:06
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 6, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.5066ms 59.7428ms 16.7384 Ops/s 16.9149 Ops/s $\color{#d91a1a}-1.04\%$
test_sync 42.0402ms 32.3902ms 30.8735 Ops/s 31.4339 Ops/s $\color{#d91a1a}-1.78\%$
test_async 61.6865ms 30.3722ms 32.9248 Ops/s 32.9698 Ops/s $\color{#d91a1a}-0.14\%$
test_simple 0.4810s 0.4168s 2.3993 Ops/s 2.3858 Ops/s $\color{#35bf28}+0.57\%$
test_transformed 0.6409s 0.5759s 1.7363 Ops/s 1.7132 Ops/s $\color{#35bf28}+1.35\%$
test_serial 1.3466s 1.2832s 0.7793 Ops/s 0.7932 Ops/s $\color{#d91a1a}-1.75\%$
test_parallel 1.1773s 1.1105s 0.9005 Ops/s 0.9009 Ops/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-True-True-True-True] 0.1996ms 25.3100μs 39.5101 KOps/s 39.6547 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[True-True-True-True-False] 41.8190μs 14.4920μs 69.0035 KOps/s 69.0893 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-True-True-False-True] 56.6660μs 14.2887μs 69.9855 KOps/s 70.0164 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-True-False-False] 32.0200μs 8.2447μs 121.2896 KOps/s 119.9604 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[True-True-False-True-True] 71.1630μs 26.8380μs 37.2606 KOps/s 37.2068 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-True-False-True-False] 66.4980μs 15.9785μs 62.5843 KOps/s 62.5074 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-False-False-True] 42.0090μs 15.9415μs 62.7295 KOps/s 63.0122 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-True-False-False-False] 37.5910μs 9.8362μs 101.6657 KOps/s 101.2552 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-False-True-True-True] 62.1070μs 28.5541μs 35.0212 KOps/s 35.1781 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-True-True-False] 58.4090μs 17.6581μs 56.6312 KOps/s 56.4351 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-False-True-False-True] 48.0200μs 15.8892μs 62.9358 KOps/s 63.2910 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-False-True-False-False] 44.1930μs 9.8143μs 101.8924 KOps/s 100.3907 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-False-False-True-True] 77.5560μs 29.9828μs 33.3525 KOps/s 33.4867 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-False-False-True-False] 82.7760μs 18.9737μs 52.7044 KOps/s 52.1666 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-False-False-False-True] 40.6160μs 17.4671μs 57.2506 KOps/s 57.3567 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-False-False-False] 38.2210μs 11.3680μs 87.9662 KOps/s 87.9409 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-True-True-True] 74.8000μs 28.5021μs 35.0851 KOps/s 34.9741 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-True-True-True-False] 41.0170μs 17.6416μs 56.6842 KOps/s 56.1380 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-True-True-False-True] 42.1090μs 18.5240μs 53.9841 KOps/s 53.6340 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-True-False-False] 33.4030μs 11.2213μs 89.1162 KOps/s 89.9499 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-True-False-True-True] 72.3050μs 30.1382μs 33.1805 KOps/s 33.1553 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-False-True-False] 58.9110μs 19.1398μs 52.2471 KOps/s 51.8131 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[False-True-False-False-True] 57.1470μs 20.1926μs 49.5230 KOps/s 49.3786 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-False-False-False] 85.2800μs 12.5845μs 79.4625 KOps/s 79.2919 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-False-True-True-True] 3.5811ms 31.5172μs 31.7287 KOps/s 31.4573 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-True-True-False] 55.0030μs 20.5740μs 48.6051 KOps/s 47.6344 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-False-True-False-True] 55.5440μs 19.7676μs 50.5879 KOps/s 50.0536 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-True-False-False] 73.7810μs 12.6079μs 79.3152 KOps/s 78.6412 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-False-True-True] 65.2520μs 32.7535μs 30.5311 KOps/s 30.4371 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-True-False] 53.7710μs 22.0896μs 45.2701 KOps/s 45.1467 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-False-False-True] 57.3670μs 21.2640μs 47.0279 KOps/s 47.0643 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-False-False-False-False] 45.7260μs 13.9848μs 71.5060 KOps/s 70.9166 KOps/s $\color{#35bf28}+0.83\%$
test_values[generalized_advantage_estimate-True-True] 14.2944ms 9.6208ms 103.9411 Ops/s 107.1815 Ops/s $\color{#d91a1a}-3.02\%$
test_values[vec_generalized_advantage_estimate-True-True] 43.3619ms 35.8281ms 27.9111 Ops/s 28.1655 Ops/s $\color{#d91a1a}-0.90\%$
test_values[td0_return_estimate-False-False] 0.2541ms 0.1682ms 5.9443 KOps/s 6.1172 KOps/s $\color{#d91a1a}-2.83\%$
test_values[td1_return_estimate-False-False] 24.3058ms 23.7934ms 42.0285 Ops/s 42.3218 Ops/s $\color{#d91a1a}-0.69\%$
test_values[vec_td1_return_estimate-False-False] 38.5411ms 35.5897ms 28.0980 Ops/s 28.0243 Ops/s $\color{#35bf28}+0.26\%$
test_values[td_lambda_return_estimate-True-False] 35.4032ms 34.1322ms 29.2979 Ops/s 29.9394 Ops/s $\color{#d91a1a}-2.14\%$
test_values[vec_td_lambda_return_estimate-True-False] 39.4459ms 35.6103ms 28.0818 Ops/s 27.9417 Ops/s $\color{#35bf28}+0.50\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.9232ms 8.3815ms 119.3106 Ops/s 120.7647 Ops/s $\color{#d91a1a}-1.20\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1418ms 1.9267ms 519.0334 Ops/s 506.1954 Ops/s $\color{#35bf28}+2.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6534ms 0.3595ms 2.7815 KOps/s 2.8238 KOps/s $\color{#d91a1a}-1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.0758ms 46.2119ms 21.6394 Ops/s 21.6993 Ops/s $\color{#d91a1a}-0.28\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0171ms 3.0362ms 329.3545 Ops/s 330.2871 Ops/s $\color{#d91a1a}-0.28\%$
test_dqn_speed 1.9487ms 1.2930ms 773.3693 Ops/s 771.5438 Ops/s $\color{#35bf28}+0.24\%$
test_ddpg_speed 3.7880ms 2.6972ms 370.7533 Ops/s 369.2180 Ops/s $\color{#35bf28}+0.42\%$
test_sac_speed 8.8834ms 7.9857ms 125.2244 Ops/s 125.2541 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_speed 15.5165ms 12.5557ms 79.6454 Ops/s 78.4816 Ops/s $\color{#35bf28}+1.48\%$
test_redq_deprec_speed 17.2409ms 12.8342ms 77.9166 Ops/s 79.4277 Ops/s $\color{#d91a1a}-1.90\%$
test_td3_speed 11.5912ms 8.1658ms 122.4613 Ops/s 127.5640 Ops/s $\color{#d91a1a}-4.00\%$
test_cql_speed 37.4551ms 35.5305ms 28.1448 Ops/s 28.0035 Ops/s $\color{#35bf28}+0.50\%$
test_a2c_speed 9.0524ms 7.3074ms 136.8483 Ops/s 137.6464 Ops/s $\color{#d91a1a}-0.58\%$
test_ppo_speed 8.9407ms 7.5829ms 131.8759 Ops/s 133.3108 Ops/s $\color{#d91a1a}-1.08\%$
test_reinforce_speed 7.9405ms 6.4550ms 154.9176 Ops/s 155.8438 Ops/s $\color{#d91a1a}-0.59\%$
test_iql_speed 34.3570ms 31.9123ms 31.3359 Ops/s 31.4978 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6162ms 4.8280ms 207.1245 Ops/s 211.3321 Ops/s $\color{#d91a1a}-1.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5600ms 0.4804ms 2.0815 KOps/s 2.1099 KOps/s $\color{#d91a1a}-1.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7546ms 0.4520ms 2.2123 KOps/s 2.1995 KOps/s $\color{#35bf28}+0.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2590ms 4.7640ms 209.9079 Ops/s 210.3263 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2592ms 0.4725ms 2.1164 KOps/s 2.1384 KOps/s $\color{#d91a1a}-1.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6708ms 0.4482ms 2.2312 KOps/s 2.2733 KOps/s $\color{#d91a1a}-1.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.3837ms 1.6949ms 589.9996 Ops/s 595.4666 Ops/s $\color{#d91a1a}-0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0122ms 1.5990ms 625.3892 Ops/s 627.2803 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.0206ms 4.9348ms 202.6420 Ops/s 204.8038 Ops/s $\color{#d91a1a}-1.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8649ms 0.6159ms 1.6235 KOps/s 1.6447 KOps/s $\color{#d91a1a}-1.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8481ms 0.5893ms 1.6970 KOps/s 1.7181 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7489ms 4.7607ms 210.0542 Ops/s 210.4124 Ops/s $\color{#d91a1a}-0.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1312ms 0.4823ms 2.0732 KOps/s 1.6616 KOps/s $\textbf{\color{#35bf28}+24.77\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6298ms 0.4522ms 2.2113 KOps/s 2.1895 KOps/s $\color{#35bf28}+1.00\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8174ms 4.7836ms 209.0466 Ops/s 212.6581 Ops/s $\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6798ms 0.4826ms 2.0721 KOps/s 2.1391 KOps/s $\color{#d91a1a}-3.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7144ms 0.4580ms 2.1833 KOps/s 2.2152 KOps/s $\color{#d91a1a}-1.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.7703ms 4.9755ms 200.9851 Ops/s 205.7781 Ops/s $\color{#d91a1a}-2.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9788ms 0.6225ms 1.6065 KOps/s 1.6333 KOps/s $\color{#d91a1a}-1.64\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8068ms 0.5877ms 1.7015 KOps/s 1.7172 KOps/s $\color{#d91a1a}-0.91\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1252s 8.1331ms 122.9549 Ops/s 163.7532 Ops/s $\textbf{\color{#d91a1a}-24.91\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.8088ms 12.7935ms 78.1644 Ops/s 78.4135 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8469ms 1.1395ms 877.5724 Ops/s 900.3424 Ops/s $\color{#d91a1a}-2.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1021s 5.7592ms 173.6342 Ops/s 172.9480 Ops/s $\color{#35bf28}+0.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6428ms 12.8168ms 78.0227 Ops/s 78.6484 Ops/s $\color{#d91a1a}-0.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7129ms 1.1163ms 895.8494 Ops/s 908.3577 Ops/s $\color{#d91a1a}-1.38\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1083s 5.9988ms 166.6991 Ops/s 126.5538 Ops/s $\textbf{\color{#35bf28}+31.72\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.1018ms 12.9713ms 77.0933 Ops/s 77.1800 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9921ms 1.2903ms 775.0306 Ops/s 801.0750 Ops/s $\color{#d91a1a}-3.25\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 6, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1049s 0.1034s 9.6713 Ops/s 9.2013 Ops/s $\textbf{\color{#35bf28}+5.11\%}$
test_sync 94.9432ms 91.2840ms 10.9548 Ops/s 11.2300 Ops/s $\color{#d91a1a}-2.45\%$
test_async 0.2572s 86.8813ms 11.5100 Ops/s 11.3809 Ops/s $\color{#35bf28}+1.13\%$
test_single_pixels 0.1146s 0.1142s 8.7555 Ops/s 8.4206 Ops/s $\color{#35bf28}+3.98\%$
test_sync_pixels 78.3823ms 76.7172ms 13.0349 Ops/s 12.9692 Ops/s $\color{#35bf28}+0.51\%$
test_async_pixels 0.1290s 70.1010ms 14.2651 Ops/s 14.0671 Ops/s $\color{#35bf28}+1.41\%$
test_simple 0.7409s 0.7387s 1.3538 Ops/s 1.2693 Ops/s $\textbf{\color{#35bf28}+6.65\%}$
test_transformed 1.0718s 0.9906s 1.0094 Ops/s 0.9965 Ops/s $\color{#35bf28}+1.30\%$
test_serial 2.2018s 2.1220s 0.4712 Ops/s 0.4554 Ops/s $\color{#35bf28}+3.47\%$
test_parallel 1.9865s 1.9113s 0.5232 Ops/s 0.5317 Ops/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[True-True-True-True-True] 0.1942ms 38.7849μs 25.7832 KOps/s 25.9917 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-True-True-False] 55.9010μs 21.1405μs 47.3027 KOps/s 47.2071 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-True-True-False-True] 0.1992ms 21.5548μs 46.3933 KOps/s 46.7902 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-True-True-False-False] 0.1962ms 12.0380μs 83.0706 KOps/s 83.5244 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-False-True-True] 0.2374ms 40.1874μs 24.8834 KOps/s 24.3564 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-False-True-False] 61.5710μs 23.2723μs 42.9695 KOps/s 42.6731 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-True-False-False-True] 0.1995ms 23.8226μs 41.9769 KOps/s 41.8353 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-True-False-False-False] 30.7710μs 14.3016μs 69.9223 KOps/s 69.8595 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[True-False-True-True-True] 66.1210μs 42.6190μs 23.4637 KOps/s 23.0660 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-False-True-True-False] 0.2125ms 25.4952μs 39.2231 KOps/s 38.5752 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-False-True-False-True] 42.0200μs 23.7299μs 42.1409 KOps/s 41.6543 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-False-False] 30.5210μs 14.2843μs 70.0069 KOps/s 69.6995 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-False-False-True-True] 75.7220μs 44.2840μs 22.5815 KOps/s 22.1000 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-False-False-True-False] 0.1359ms 27.9389μs 35.7923 KOps/s 35.0348 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-False-False-True] 44.3610μs 25.6756μs 38.9474 KOps/s 38.5098 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-False-False-False] 32.8310μs 16.5665μs 60.3628 KOps/s 59.8626 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[False-True-True-True-True] 60.1010μs 42.3354μs 23.6209 KOps/s 23.1165 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-True-True-True-False] 0.2114ms 25.7518μs 38.8322 KOps/s 38.3612 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-True-True-False-True] 0.1947ms 27.9370μs 35.7948 KOps/s 35.7589 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-False-False] 0.1870ms 16.1631μs 61.8694 KOps/s 61.2373 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-True-False-True-True] 76.9520μs 44.5750μs 22.4341 KOps/s 22.0525 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-True-False-True-False] 47.9810μs 27.7911μs 35.9827 KOps/s 35.7612 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-False-False-True] 49.2810μs 30.2671μs 33.0391 KOps/s 33.3497 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[False-True-False-False-False] 48.6910μs 18.3984μs 54.3526 KOps/s 54.3448 KOps/s $\color{#35bf28}+0.01\%$
test_step_mdp_speed[False-False-True-True-True] 4.3245ms 47.7546μs 20.9404 KOps/s 20.6982 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-True-True-False] 71.0410μs 30.3306μs 32.9700 KOps/s 32.4992 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-False-True-False-True] 50.1510μs 30.3726μs 32.9244 KOps/s 33.0522 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-False-True-False-False] 38.0600μs 18.4269μs 54.2685 KOps/s 54.1096 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-False-False-True-True] 76.9910μs 48.2997μs 20.7041 KOps/s 20.0146 KOps/s $\color{#35bf28}+3.44\%$
test_step_mdp_speed[False-False-False-True-False] 57.9110μs 32.3084μs 30.9517 KOps/s 30.6422 KOps/s $\color{#35bf28}+1.01\%$
test_step_mdp_speed[False-False-False-False-True] 50.1310μs 31.8277μs 31.4192 KOps/s 31.4899 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-False-False-False] 44.6810μs 20.3830μs 49.0606 KOps/s 48.4761 KOps/s $\color{#35bf28}+1.21\%$
test_values[generalized_advantage_estimate-True-True] 24.1522ms 23.7710ms 42.0681 Ops/s 41.4203 Ops/s $\color{#35bf28}+1.56\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.2076ms 2.8165ms 355.0462 Ops/s 363.4231 Ops/s $\color{#d91a1a}-2.30\%$
test_values[td0_return_estimate-False-False] 89.1220μs 64.0974μs 15.6013 KOps/s 15.2979 KOps/s $\color{#35bf28}+1.98\%$
test_values[td1_return_estimate-False-False] 54.1378ms 53.5604ms 18.6705 Ops/s 18.5058 Ops/s $\color{#35bf28}+0.89\%$
test_values[vec_td1_return_estimate-False-False] 1.4046ms 1.0745ms 930.6367 Ops/s 929.7190 Ops/s $\color{#35bf28}+0.10\%$
test_values[td_lambda_return_estimate-True-False] 86.9997ms 85.3683ms 11.7139 Ops/s 11.6610 Ops/s $\color{#35bf28}+0.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4449ms 1.0767ms 928.7947 Ops/s 931.9822 Ops/s $\color{#d91a1a}-0.34\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0906ms 23.5861ms 42.3979 Ops/s 42.0039 Ops/s $\color{#35bf28}+0.94\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9713ms 0.7246ms 1.3801 KOps/s 1.4016 KOps/s $\color{#d91a1a}-1.53\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8162ms 0.6573ms 1.5213 KOps/s 1.5206 KOps/s $\color{#35bf28}+0.04\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6860ms 1.4559ms 686.8799 Ops/s 685.7336 Ops/s $\color{#35bf28}+0.17\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8542ms 0.6693ms 1.4940 KOps/s 1.4942 KOps/s $-0.01\%$
test_dqn_speed 8.3156ms 1.4086ms 709.9000 Ops/s 737.7073 Ops/s $\color{#d91a1a}-3.77\%$
test_ddpg_speed 3.0518ms 2.7669ms 361.4097 Ops/s 356.8910 Ops/s $\color{#35bf28}+1.27\%$
test_sac_speed 8.2021ms 7.8277ms 127.7512 Ops/s 124.2544 Ops/s $\color{#35bf28}+2.81\%$
test_redq_speed 12.1623ms 10.4161ms 96.0048 Ops/s 92.3970 Ops/s $\color{#35bf28}+3.90\%$
test_redq_deprec_speed 11.4300ms 10.8846ms 91.8730 Ops/s 89.1715 Ops/s $\color{#35bf28}+3.03\%$
test_td3_speed 7.9850ms 7.7754ms 128.6115 Ops/s 125.4443 Ops/s $\color{#35bf28}+2.52\%$
test_cql_speed 26.8673ms 25.3367ms 39.4685 Ops/s 38.0293 Ops/s $\color{#35bf28}+3.78\%$
test_a2c_speed 6.4272ms 5.4655ms 182.9662 Ops/s 180.9056 Ops/s $\color{#35bf28}+1.14\%$
test_ppo_speed 6.2533ms 5.8518ms 170.8871 Ops/s 170.3402 Ops/s $\color{#35bf28}+0.32\%$
test_reinforce_speed 4.9359ms 4.5462ms 219.9621 Ops/s 221.3653 Ops/s $\color{#d91a1a}-0.63\%$
test_iql_speed 21.7934ms 20.1599ms 49.6034 Ops/s 49.5640 Ops/s $\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8806ms 6.5959ms 151.6084 Ops/s 147.5501 Ops/s $\color{#35bf28}+2.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1186s 0.6031ms 1.6581 KOps/s 1.9024 KOps/s $\textbf{\color{#d91a1a}-12.84\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6821ms 0.5017ms 1.9932 KOps/s 1.9725 KOps/s $\color{#35bf28}+1.05\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1158ms 6.5436ms 152.8201 Ops/s 149.5852 Ops/s $\color{#35bf28}+2.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9307ms 0.5150ms 1.9417 KOps/s 1.9287 KOps/s $\color{#35bf28}+0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7013ms 0.5000ms 2.0001 KOps/s 1.9986 KOps/s $\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 3.0742ms 2.0347ms 491.4633 Ops/s 500.7693 Ops/s $\color{#d91a1a}-1.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.8484ms 1.9163ms 521.8377 Ops/s 521.9678 Ops/s $\color{#d91a1a}-0.02\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9785ms 6.6910ms 149.4536 Ops/s 145.0167 Ops/s $\color{#35bf28}+3.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1406s 0.8027ms 1.2458 KOps/s 1.4792 KOps/s $\textbf{\color{#d91a1a}-15.78\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9775ms 0.6519ms 1.5339 KOps/s 1.5260 KOps/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8900ms 6.5834ms 151.8966 Ops/s 148.2279 Ops/s $\color{#35bf28}+2.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.8332ms 0.5218ms 1.9163 KOps/s 1.8957 KOps/s $\color{#35bf28}+1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6854ms 0.5021ms 1.9915 KOps/s 1.9712 KOps/s $\color{#35bf28}+1.03\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9839ms 6.5132ms 153.5334 Ops/s 150.4498 Ops/s $\color{#35bf28}+2.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0091ms 0.5148ms 1.9424 KOps/s 1.9197 KOps/s $\color{#35bf28}+1.18\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7057ms 0.4990ms 2.0038 KOps/s 1.9978 KOps/s $\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0694ms 6.7163ms 148.8925 Ops/s 145.8456 Ops/s $\color{#35bf28}+2.09\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8920ms 0.6741ms 1.4836 KOps/s 1.4866 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8820ms 0.6531ms 1.5311 KOps/s 1.5149 KOps/s $\color{#35bf28}+1.07\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1569s 8.2011ms 121.9341 Ops/s 122.6299 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.6632ms 15.5119ms 64.4667 Ops/s 61.3307 Ops/s $\textbf{\color{#35bf28}+5.11\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.7089ms 1.3370ms 747.9238 Ops/s 770.0369 Ops/s $\color{#d91a1a}-2.87\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1328s 7.7032ms 129.8156 Ops/s 96.0197 Ops/s $\textbf{\color{#35bf28}+35.20\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6125ms 15.4946ms 64.5384 Ops/s 61.5919 Ops/s $\color{#35bf28}+4.78\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2104ms 1.2622ms 792.2894 Ops/s 721.4652 Ops/s $\textbf{\color{#35bf28}+9.82\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1399s 10.6427ms 93.9612 Ops/s 127.4012 Ops/s $\textbf{\color{#d91a1a}-26.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.1932ms 15.7306ms 63.5704 Ops/s 60.9223 Ops/s $\color{#35bf28}+4.35\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4151ms 1.4264ms 701.0531 Ops/s 651.7579 Ops/s $\textbf{\color{#35bf28}+7.56\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot