Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix reward2go for nd tensors #2087

Merged
merged 3 commits into from
Apr 18, 2024
Merged

[BugFix] Fix reward2go for nd tensors #2087

merged 3 commits into from
Apr 18, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 18, 2024

Fixes #2086

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 18, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2087

Note: Links to docs will display an error until the docs builds have been completed.

❌ 13 New Failures

As of commit 5152b9d with merge base a740559 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 18, 2024
@vmoens vmoens added the bug Something isn't working label Apr 18, 2024
@vmoens vmoens merged commit 61c42e4 into main Apr 18, 2024
54 of 67 checks passed
@vmoens vmoens deleted the fix-2086 branch April 18, 2024 12:42
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 52.7315ms 52.4933ms 19.0501 Ops/s 17.5480 Ops/s $\textbf{\color{#35bf28}+8.56\%}$
test_sync 40.1583ms 35.1467ms 28.4522 Ops/s 33.6809 Ops/s $\textbf{\color{#d91a1a}-15.52\%}$
test_async 61.7025ms 26.8920ms 37.1858 Ops/s 37.4505 Ops/s $\color{#d91a1a}-0.71\%$
test_simple 0.4017s 0.3449s 2.8991 Ops/s 2.9656 Ops/s $\color{#d91a1a}-2.24\%$
test_transformed 0.5484s 0.4937s 2.0254 Ops/s 1.9883 Ops/s $\color{#35bf28}+1.87\%$
test_serial 1.2476s 1.1960s 0.8361 Ops/s 0.8313 Ops/s $\color{#35bf28}+0.58\%$
test_parallel 1.0675s 1.0087s 0.9914 Ops/s 0.9958 Ops/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-True-True-True] 0.1667ms 21.4389μs 46.6442 KOps/s 46.1566 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-True-True-False] 47.7690μs 13.0129μs 76.8465 KOps/s 75.4965 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-True-False-True] 36.9990μs 12.4321μs 80.4370 KOps/s 79.1242 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-True-True-False-False] 46.7460μs 7.5870μs 131.8041 KOps/s 130.6975 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-True-True] 68.2860μs 22.6696μs 44.1120 KOps/s 43.7371 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[True-True-False-True-False] 50.0730μs 14.2600μs 70.1263 KOps/s 69.0780 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-True-False-False-True] 41.1060μs 13.7525μs 72.7142 KOps/s 72.2950 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-True-False-False-False] 45.1840μs 8.8763μs 112.6601 KOps/s 111.1958 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-False-True-True-True] 74.6580μs 24.1355μs 41.4327 KOps/s 41.5504 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-False-True-True-False] 52.8780μs 15.5606μs 64.2647 KOps/s 63.8219 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-True-False-True] 41.9580μs 13.6502μs 73.2591 KOps/s 71.9414 KOps/s $\color{#35bf28}+1.83\%$
test_step_mdp_speed[True-False-True-False-False] 47.2180μs 8.8403μs 113.1177 KOps/s 111.5936 KOps/s $\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-False-False-True-True] 0.1079ms 25.0403μs 39.9356 KOps/s 39.4169 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-False-False-True-False] 51.9770μs 16.7110μs 59.8409 KOps/s 59.1387 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-False-False-False-True] 39.7130μs 14.8403μs 67.3841 KOps/s 66.6008 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-False-False-False] 37.7000μs 9.9309μs 100.6955 KOps/s 99.0985 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-True-True-True-True] 59.5110μs 24.3188μs 41.1205 KOps/s 41.1216 KOps/s $-0.00\%$
test_step_mdp_speed[False-True-True-True-False] 38.4310μs 15.7581μs 63.4594 KOps/s 63.4997 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-True-False-True] 54.5610μs 16.0475μs 62.3148 KOps/s 62.3115 KOps/s $+0.01\%$
test_step_mdp_speed[False-True-True-False-False] 34.9150μs 10.0599μs 99.4044 KOps/s 98.2036 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-True-True] 58.1780μs 25.8880μs 38.6280 KOps/s 38.7917 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-True-False-True-False] 55.5530μs 16.9969μs 58.8344 KOps/s 58.9838 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[False-True-False-False-True] 63.7080μs 16.7858μs 59.5740 KOps/s 58.3836 KOps/s $\color{#35bf28}+2.04\%$
test_step_mdp_speed[False-True-False-False-False] 49.2710μs 11.2281μs 89.0621 KOps/s 88.6409 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-False-True-True-True] 51.5960μs 26.2548μs 38.0883 KOps/s 37.6371 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-False-True-True-False] 56.0440μs 17.8112μs 56.1443 KOps/s 54.6034 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[False-False-True-False-True] 43.6610μs 16.9203μs 59.1005 KOps/s 55.6818 KOps/s $\textbf{\color{#35bf28}+6.14\%}$
test_step_mdp_speed[False-False-True-False-False] 48.2290μs 11.0881μs 90.1870 KOps/s 87.9838 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-False-False-True-True] 57.8380μs 27.3754μs 36.5291 KOps/s 36.3691 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-False-False-True-False] 55.9940μs 18.9874μs 52.6664 KOps/s 51.4305 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-False-False-False-True] 56.5750μs 18.1629μs 55.0573 KOps/s 55.0806 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-False-False-False-False] 54.9020μs 12.2814μs 81.4240 KOps/s 80.0301 KOps/s $\color{#35bf28}+1.74\%$
test_values[generalized_advantage_estimate-True-True] 9.5879ms 9.3345ms 107.1291 Ops/s 106.8849 Ops/s $\color{#35bf28}+0.23\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.6571ms 35.7163ms 27.9984 Ops/s 28.2570 Ops/s $\color{#d91a1a}-0.92\%$
test_values[td0_return_estimate-False-False] 0.2191ms 0.1681ms 5.9502 KOps/s 5.9147 KOps/s $\color{#35bf28}+0.60\%$
test_values[td1_return_estimate-False-False] 26.5139ms 23.4129ms 42.7116 Ops/s 42.5465 Ops/s $\color{#35bf28}+0.39\%$
test_values[vec_td1_return_estimate-False-False] 36.5636ms 35.3292ms 28.3052 Ops/s 28.1482 Ops/s $\color{#35bf28}+0.56\%$
test_values[td_lambda_return_estimate-True-False] 33.6848ms 33.2921ms 30.0371 Ops/s 29.4872 Ops/s $\color{#35bf28}+1.87\%$
test_values[vec_td_lambda_return_estimate-True-False] 39.5230ms 35.3958ms 28.2519 Ops/s 28.2366 Ops/s $\color{#35bf28}+0.05\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2914ms 8.1470ms 122.7448 Ops/s 122.2703 Ops/s $\color{#35bf28}+0.39\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3059ms 1.9710ms 507.3634 Ops/s 562.8275 Ops/s $\textbf{\color{#d91a1a}-9.85\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4377ms 0.3396ms 2.9445 KOps/s 2.8256 KOps/s $\color{#35bf28}+4.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.8109ms 44.1169ms 22.6670 Ops/s 21.6961 Ops/s $\color{#35bf28}+4.48\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5092ms 3.0156ms 331.6050 Ops/s 330.4553 Ops/s $\color{#35bf28}+0.35\%$
test_dqn_speed 7.0309ms 1.3320ms 750.7298 Ops/s 739.7330 Ops/s $\color{#35bf28}+1.49\%$
test_ddpg_speed 3.4389ms 2.6603ms 375.8972 Ops/s 372.1826 Ops/s $\color{#35bf28}+1.00\%$
test_sac_speed 8.4838ms 8.0550ms 124.1465 Ops/s 109.8008 Ops/s $\textbf{\color{#35bf28}+13.07\%}$
test_redq_speed 19.0623ms 13.5836ms 73.6184 Ops/s 74.4840 Ops/s $\color{#d91a1a}-1.16\%$
test_redq_deprec_speed 13.7290ms 13.2266ms 75.6052 Ops/s 73.3189 Ops/s $\color{#35bf28}+3.12\%$
test_td3_speed 9.2538ms 8.0144ms 124.7750 Ops/s 122.4053 Ops/s $\color{#35bf28}+1.94\%$
test_cql_speed 38.4920ms 36.5863ms 27.3326 Ops/s 27.5102 Ops/s $\color{#d91a1a}-0.65\%$
test_a2c_speed 8.4209ms 7.3485ms 136.0827 Ops/s 131.3937 Ops/s $\color{#35bf28}+3.57\%$
test_ppo_speed 8.9444ms 7.7098ms 129.7044 Ops/s 129.3765 Ops/s $\color{#35bf28}+0.25\%$
test_reinforce_speed 7.7850ms 6.5760ms 152.0678 Ops/s 151.7540 Ops/s $\color{#35bf28}+0.21\%$
test_iql_speed 33.3638ms 32.5011ms 30.7682 Ops/s 30.4200 Ops/s $\color{#35bf28}+1.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3523ms 2.3250ms 430.1071 Ops/s 432.2296 Ops/s $\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2870ms 0.5004ms 1.9984 KOps/s 1.9931 KOps/s $\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6557ms 0.4722ms 2.1179 KOps/s 2.1056 KOps/s $\color{#35bf28}+0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.6121ms 2.3590ms 423.9146 Ops/s 422.4063 Ops/s $\color{#35bf28}+0.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1509ms 0.4920ms 2.0325 KOps/s 2.0225 KOps/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7517ms 0.4664ms 2.1440 KOps/s 2.1062 KOps/s $\color{#35bf28}+1.80\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7693ms 1.2082ms 827.6991 Ops/s 810.4305 Ops/s $\color{#35bf28}+2.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.2937ms 1.1345ms 881.4292 Ops/s 854.7055 Ops/s $\color{#35bf28}+3.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5580ms 2.4575ms 406.9223 Ops/s 396.1537 Ops/s $\color{#35bf28}+2.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0887ms 0.6155ms 1.6247 KOps/s 1.6228 KOps/s $\color{#35bf28}+0.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9094ms 0.5890ms 1.6977 KOps/s 1.6845 KOps/s $\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.5516ms 2.3120ms 432.5209 Ops/s 430.7462 Ops/s $\color{#35bf28}+0.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0262ms 0.4996ms 2.0016 KOps/s 2.0096 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7970ms 0.4783ms 2.0906 KOps/s 2.1054 KOps/s $\color{#d91a1a}-0.70\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5616ms 2.3440ms 426.6131 Ops/s 413.5034 Ops/s $\color{#35bf28}+3.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7535ms 0.4997ms 2.0014 KOps/s 2.0235 KOps/s $\color{#d91a1a}-1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6177ms 0.4718ms 2.1196 KOps/s 2.1114 KOps/s $\color{#35bf28}+0.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5409ms 2.4275ms 411.9518 Ops/s 405.6714 Ops/s $\color{#35bf28}+1.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2065ms 0.6183ms 1.6173 KOps/s 1.6191 KOps/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7719ms 0.5898ms 1.6956 KOps/s 1.6849 KOps/s $\color{#35bf28}+0.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1148s 7.9001ms 126.5805 Ops/s 126.3311 Ops/s $\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.6821ms 12.1989ms 81.9746 Ops/s 83.1145 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6758ms 1.0651ms 938.8933 Ops/s 964.9392 Ops/s $\color{#d91a1a}-2.70\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1053s 5.6296ms 177.6316 Ops/s 174.7251 Ops/s $\color{#35bf28}+1.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.8114ms 12.1431ms 82.3515 Ops/s 71.1065 Ops/s $\textbf{\color{#35bf28}+15.81\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8019ms 1.0708ms 933.8537 Ops/s 980.2970 Ops/s $\color{#d91a1a}-4.74\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1091s 8.1372ms 122.8925 Ops/s 165.1023 Ops/s $\textbf{\color{#d91a1a}-25.57\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.1326ms 12.4697ms 80.1945 Ops/s 80.8068 Ops/s $\color{#d91a1a}-0.76\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8863ms 1.3560ms 737.4407 Ops/s 738.5708 Ops/s $\color{#d91a1a}-0.15\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1030s 0.1009s 9.9084 Ops/s 8.9625 Ops/s $\textbf{\color{#35bf28}+10.55\%}$
test_sync 87.1868ms 86.5954ms 11.5480 Ops/s 11.3158 Ops/s $\color{#35bf28}+2.05\%$
test_async 0.1589s 71.8228ms 13.9232 Ops/s 13.9490 Ops/s $\color{#d91a1a}-0.18\%$
test_single_pixels 0.1120s 0.1110s 9.0094 Ops/s 8.8850 Ops/s $\color{#35bf28}+1.40\%$
test_sync_pixels 67.9500ms 66.3147ms 15.0796 Ops/s 15.0765 Ops/s $\color{#35bf28}+0.02\%$
test_async_pixels 0.1213s 55.4277ms 18.0415 Ops/s 17.8097 Ops/s $\color{#35bf28}+1.30\%$
test_simple 0.6781s 0.6753s 1.4808 Ops/s 1.4398 Ops/s $\color{#35bf28}+2.85\%$
test_transformed 0.8980s 0.8941s 1.1185 Ops/s 1.0859 Ops/s $\color{#35bf28}+3.00\%$
test_serial 2.1371s 2.0805s 0.4806 Ops/s 0.4722 Ops/s $\color{#35bf28}+1.79\%$
test_parallel 1.8457s 1.7924s 0.5579 Ops/s 0.5499 Ops/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-True-True-True-True] 90.7430μs 33.2845μs 30.0440 KOps/s 29.6971 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-True-True-True-False] 68.3520μs 19.7209μs 50.7075 KOps/s 49.6565 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-True-True-False-True] 45.8110μs 18.6599μs 53.5910 KOps/s 52.1286 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-True-True-False-False] 31.3310μs 11.2424μs 88.9491 KOps/s 87.5713 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-False-True-True] 53.1310μs 34.2428μs 29.2032 KOps/s 28.0653 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[True-True-False-True-False] 49.0910μs 21.4899μs 46.5334 KOps/s 45.7315 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-False-False-True] 37.6010μs 20.2166μs 49.4644 KOps/s 48.1322 KOps/s $\color{#35bf28}+2.77\%$
test_step_mdp_speed[True-True-False-False-False] 29.7710μs 13.1683μs 75.9398 KOps/s 75.2843 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-False-True-True-True] 56.6710μs 36.3237μs 27.5302 KOps/s 26.6460 KOps/s $\color{#35bf28}+3.32\%$
test_step_mdp_speed[True-False-True-True-False] 47.8110μs 23.5195μs 42.5179 KOps/s 41.6322 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-False-True-False-True] 39.0010μs 20.2835μs 49.3012 KOps/s 47.6125 KOps/s $\color{#35bf28}+3.55\%$
test_step_mdp_speed[True-False-True-False-False] 38.9610μs 13.0977μs 76.3492 KOps/s 75.4951 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-False-False-True-True] 60.6220μs 38.5284μs 25.9549 KOps/s 25.4265 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-False-False-True-False] 71.7020μs 25.3354μs 39.4705 KOps/s 38.5051 KOps/s $\color{#35bf28}+2.51\%$
test_step_mdp_speed[True-False-False-False-True] 44.5220μs 22.0257μs 45.4014 KOps/s 43.9481 KOps/s $\color{#35bf28}+3.31\%$
test_step_mdp_speed[True-False-False-False-False] 32.1010μs 14.8518μs 67.3318 KOps/s 65.6694 KOps/s $\color{#35bf28}+2.53\%$
test_step_mdp_speed[False-True-True-True-True] 81.1820μs 36.5978μs 27.3240 KOps/s 26.8534 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-True-True-True-False] 58.3920μs 23.8322μs 41.9600 KOps/s 41.5776 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-True-True-False-True] 42.3710μs 23.9557μs 41.7438 KOps/s 39.8196 KOps/s $\color{#35bf28}+4.83\%$
test_step_mdp_speed[False-True-True-False-False] 49.1810μs 15.1391μs 66.0540 KOps/s 65.9982 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-False-True-True] 69.3520μs 38.8503μs 25.7398 KOps/s 25.5610 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-True-False-True-False] 65.4420μs 25.7280μs 38.8681 KOps/s 38.5395 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-True-False-False-True] 45.4810μs 25.6843μs 38.9342 KOps/s 37.5713 KOps/s $\color{#35bf28}+3.63\%$
test_step_mdp_speed[False-True-False-False-False] 44.4010μs 16.8618μs 59.3055 KOps/s 58.6575 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-True-True-True] 89.7220μs 40.7033μs 24.5680 KOps/s 23.9946 KOps/s $\color{#35bf28}+2.39\%$
test_step_mdp_speed[False-False-True-True-False] 50.1310μs 27.2329μs 36.7202 KOps/s 35.9402 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-False-True-False-True] 52.4910μs 26.0664μs 38.3635 KOps/s 37.2706 KOps/s $\color{#35bf28}+2.93\%$
test_step_mdp_speed[False-False-True-False-False] 35.7910μs 16.8110μs 59.4847 KOps/s 58.4811 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[False-False-False-True-True] 71.7420μs 41.5144μs 24.0881 KOps/s 23.5930 KOps/s $\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-False-False-True-False] 53.2220μs 29.2777μs 34.1557 KOps/s 33.8921 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-False-False-False-True] 71.2920μs 27.3501μs 36.5629 KOps/s 35.3016 KOps/s $\color{#35bf28}+3.57\%$
test_step_mdp_speed[False-False-False-False-False] 42.8410μs 18.3154μs 54.5990 KOps/s 52.7026 KOps/s $\color{#35bf28}+3.60\%$
test_values[generalized_advantage_estimate-True-True] 27.0814ms 26.3896ms 37.8938 Ops/s 37.8710 Ops/s $\color{#35bf28}+0.06\%$
test_values[vec_generalized_advantage_estimate-True-True] 90.6871ms 3.3907ms 294.9271 Ops/s 309.2904 Ops/s $\color{#d91a1a}-4.64\%$
test_values[td0_return_estimate-False-False] 95.6920μs 66.0823μs 15.1326 KOps/s 15.0073 KOps/s $\color{#35bf28}+0.83\%$
test_values[td1_return_estimate-False-False] 54.2812ms 53.6894ms 18.6257 Ops/s 18.0651 Ops/s $\color{#35bf28}+3.10\%$
test_values[vec_td1_return_estimate-False-False] 2.0420ms 1.7700ms 564.9621 Ops/s 562.9876 Ops/s $\color{#35bf28}+0.35\%$
test_values[td_lambda_return_estimate-True-False] 91.6013ms 87.2145ms 11.4660 Ops/s 11.3925 Ops/s $\color{#35bf28}+0.65\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.1018ms 1.7693ms 565.1930 Ops/s 563.7891 Ops/s $\color{#35bf28}+0.25\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.6886ms 24.8190ms 40.2917 Ops/s 41.5613 Ops/s $\color{#d91a1a}-3.05\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8922ms 0.7093ms 1.4098 KOps/s 1.3440 KOps/s $\color{#35bf28}+4.90\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7242ms 0.6587ms 1.5182 KOps/s 1.4846 KOps/s $\color{#35bf28}+2.26\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5514ms 1.4589ms 685.4320 Ops/s 683.4635 Ops/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9364ms 0.6810ms 1.4685 KOps/s 1.4674 KOps/s $\color{#35bf28}+0.07\%$
test_dqn_speed 8.1543ms 1.4210ms 703.7427 Ops/s 683.4821 Ops/s $\color{#35bf28}+2.96\%$
test_ddpg_speed 2.9060ms 2.6900ms 371.7414 Ops/s 363.3965 Ops/s $\color{#35bf28}+2.30\%$
test_sac_speed 8.6400ms 8.0545ms 124.1547 Ops/s 121.6329 Ops/s $\color{#35bf28}+2.07\%$
test_redq_speed 10.8842ms 10.1286ms 98.7300 Ops/s 96.9037 Ops/s $\color{#35bf28}+1.88\%$
test_redq_deprec_speed 11.7760ms 11.0279ms 90.6791 Ops/s 88.2183 Ops/s $\color{#35bf28}+2.79\%$
test_td3_speed 16.8297ms 8.0437ms 124.3214 Ops/s 123.2408 Ops/s $\color{#35bf28}+0.88\%$
test_cql_speed 26.7162ms 25.1663ms 39.7356 Ops/s 39.4486 Ops/s $\color{#35bf28}+0.73\%$
test_a2c_speed 5.7271ms 5.4767ms 182.5928 Ops/s 176.8554 Ops/s $\color{#35bf28}+3.24\%$
test_ppo_speed 6.0480ms 5.7806ms 172.9935 Ops/s 167.2854 Ops/s $\color{#35bf28}+3.41\%$
test_reinforce_speed 4.6766ms 4.4460ms 224.9231 Ops/s 216.4773 Ops/s $\color{#35bf28}+3.90\%$
test_iql_speed 19.8212ms 19.2121ms 52.0506 Ops/s 50.6094 Ops/s $\color{#35bf28}+2.85\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1210ms 2.9295ms 341.3531 Ops/s 342.8786 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0872ms 0.5447ms 1.8358 KOps/s 1.8269 KOps/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7066ms 0.5180ms 1.9307 KOps/s 1.9032 KOps/s $\color{#35bf28}+1.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0914ms 2.9358ms 340.6221 Ops/s 341.8828 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.2709ms 0.5344ms 1.8713 KOps/s 1.8436 KOps/s $\color{#35bf28}+1.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6422ms 0.5104ms 1.9591 KOps/s 1.9282 KOps/s $\color{#35bf28}+1.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9843ms 1.4240ms 702.2232 Ops/s 580.7391 Ops/s $\textbf{\color{#35bf28}+20.92\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5368ms 1.3525ms 739.3980 Ops/s 723.8618 Ops/s $\color{#35bf28}+2.15\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1976ms 3.0365ms 329.3216 Ops/s 325.5282 Ops/s $\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7974ms 0.6624ms 1.5098 KOps/s 1.4779 KOps/s $\color{#35bf28}+2.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1198s 0.7429ms 1.3460 KOps/s 1.5366 KOps/s $\textbf{\color{#d91a1a}-12.40\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2572ms 2.9381ms 340.3536 Ops/s 339.4721 Ops/s $\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6657ms 0.5435ms 1.8398 KOps/s 1.8167 KOps/s $\color{#35bf28}+1.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6510ms 0.5180ms 1.9304 KOps/s 1.8812 KOps/s $\color{#35bf28}+2.62\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1230ms 2.9480ms 339.2113 Ops/s 338.3839 Ops/s $\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1216s 0.7078ms 1.4129 KOps/s 1.8654 KOps/s $\textbf{\color{#d91a1a}-24.26\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6693ms 0.5105ms 1.9587 KOps/s 1.9237 KOps/s $\color{#35bf28}+1.82\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1901ms 3.0764ms 325.0546 Ops/s 328.5595 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7790ms 0.6597ms 1.5157 KOps/s 1.4870 KOps/s $\color{#35bf28}+1.93\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.8898ms 0.6494ms 1.5399 KOps/s 1.5301 KOps/s $\color{#35bf28}+0.64\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1162s 7.0897ms 141.0506 Ops/s 134.6525 Ops/s $\color{#35bf28}+4.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1247s 16.7363ms 59.7504 Ops/s 67.9618 Ops/s $\textbf{\color{#d91a1a}-12.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.1063ms 1.0891ms 918.1773 Ops/s 914.7387 Ops/s $\color{#35bf28}+0.38\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1158s 7.1464ms 139.9312 Ops/s 140.4563 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.7713ms 14.4154ms 69.3705 Ops/s 67.8373 Ops/s $\color{#35bf28}+2.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1646ms 1.0708ms 933.9060 Ops/s 926.0721 Ops/s $\color{#35bf28}+0.85\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1151s 7.4339ms 134.5195 Ops/s 102.7381 Ops/s $\textbf{\color{#35bf28}+30.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.2756ms 14.8532ms 67.3258 Ops/s 66.3477 Ops/s $\color{#35bf28}+1.47\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.5813ms 1.4166ms 705.8991 Ops/s 704.5617 Ops/s $\color{#35bf28}+0.19\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Transpose bug in reward2go when the last dim is not 1
2 participants
@vmoens @facebook-github-bot