Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix prefetch in samples without replacement - .sample() compatibility issues #2226

Merged
merged 1 commit into from
Jun 12, 2024
Merged

[BugFix] Fix prefetch in samples without replacement - .sample() compatibility issues #2226

merged 1 commit into from
Jun 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jun 12, 2024

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jun 12, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2226

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 15 Unrelated Failures

As of commit 4472e68 with merge base 0c008db (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 12, 2024
@vmoens vmoens added the bug Something isn't working label Jun 12, 2024
@vmoens vmoens merged commit f613eef into main Jun 12, 2024
18 of 28 checks passed
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1054s 58.4288ms 17.1149 Ops/s 17.8129 Ops/s $\color{#d91a1a}-3.92\%$
test_sync 36.8427ms 30.8508ms 32.4141 Ops/s 32.5147 Ops/s $\color{#d91a1a}-0.31\%$
test_async 73.8574ms 30.5190ms 32.7665 Ops/s 34.9886 Ops/s $\textbf{\color{#d91a1a}-6.35\%}$
test_simple 0.4518s 0.3915s 2.5544 Ops/s 2.6492 Ops/s $\color{#d91a1a}-3.58\%$
test_transformed 0.5281s 0.5259s 1.9015 Ops/s 1.8788 Ops/s $\color{#35bf28}+1.21\%$
test_serial 1.3038s 1.2524s 0.7985 Ops/s 0.7810 Ops/s $\color{#35bf28}+2.24\%$
test_parallel 1.1150s 1.0521s 0.9505 Ops/s 0.9356 Ops/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-True-True-True-True] 0.2469ms 21.0882μs 47.4198 KOps/s 47.9026 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[True-True-True-True-False] 52.2980μs 12.9132μs 77.4402 KOps/s 77.1438 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-True-False-True] 45.4260μs 12.6042μs 79.3389 KOps/s 81.1293 KOps/s $\color{#d91a1a}-2.21\%$
test_step_mdp_speed[True-True-True-False-False] 55.1630μs 7.6116μs 131.3786 KOps/s 133.3409 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[True-True-False-True-True] 71.9750μs 22.6074μs 44.2333 KOps/s 44.4825 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-True-False-True-False] 65.9130μs 13.9758μs 71.5525 KOps/s 71.4742 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-False-False-True] 38.4410μs 13.6433μs 73.2960 KOps/s 73.1042 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-False-False-False] 51.0750μs 8.8118μs 113.4836 KOps/s 115.8091 KOps/s $\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-False-True-True-True] 54.7420μs 23.8729μs 41.8885 KOps/s 42.1533 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-True-True-False] 64.0190μs 15.5419μs 64.3423 KOps/s 65.5036 KOps/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[True-False-True-False-True] 0.1404ms 13.9248μs 71.8143 KOps/s 73.6179 KOps/s $\color{#d91a1a}-2.45\%$
test_step_mdp_speed[True-False-True-False-False] 0.2648ms 9.5162μs 105.0843 KOps/s 115.7755 KOps/s $\textbf{\color{#d91a1a}-9.23\%}$
test_step_mdp_speed[True-False-False-True-True] 61.1040μs 25.0511μs 39.9184 KOps/s 40.0328 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-False-True-False] 43.3710μs 16.6308μs 60.1292 KOps/s 60.1285 KOps/s $+0.00\%$
test_step_mdp_speed[True-False-False-False-True] 56.9170μs 14.8757μs 67.2237 KOps/s 68.0242 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-False-False-False-False] 38.8720μs 9.9183μs 100.8235 KOps/s 102.3502 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[False-True-True-True-True] 71.4640μs 23.5277μs 42.5031 KOps/s 42.3869 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-True-True-False] 68.0880μs 15.4114μs 64.8869 KOps/s 65.5109 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-True-False-True] 53.6510μs 15.9057μs 62.8707 KOps/s 63.5557 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-True-True-False-False] 55.0230μs 9.9772μs 100.2283 KOps/s 102.1222 KOps/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[False-True-False-True-True] 0.2177ms 25.2238μs 39.6451 KOps/s 40.4712 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-True-False-True-False] 61.1440μs 16.5858μs 60.2925 KOps/s 61.0651 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[False-True-False-False-True] 41.9990μs 17.0370μs 58.6959 KOps/s 59.1058 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-False-False-False] 55.6240μs 11.1652μs 89.5642 KOps/s 90.8347 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-False-True-True-True] 61.7750μs 26.2757μs 38.0580 KOps/s 36.6228 KOps/s $\color{#35bf28}+3.92\%$
test_step_mdp_speed[False-False-True-True-False] 55.1230μs 17.9540μs 55.6979 KOps/s 56.3188 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-False-True-False-True] 70.7820μs 17.1040μs 58.4657 KOps/s 59.3791 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-False-True-False-False] 59.3310μs 11.2809μs 88.6456 KOps/s 91.2353 KOps/s $\color{#d91a1a}-2.84\%$
test_step_mdp_speed[False-False-False-True-True] 53.2000μs 28.1627μs 35.5079 KOps/s 36.2279 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-False-False-True-False] 67.2560μs 19.0308μs 52.5465 KOps/s 53.0643 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[False-False-False-False-True] 52.2980μs 18.2036μs 54.9343 KOps/s 56.1053 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[False-False-False-False-False] 47.6900μs 12.1935μs 82.0107 KOps/s 83.6572 KOps/s $\color{#d91a1a}-1.97\%$
test_values[generalized_advantage_estimate-True-True] 10.6455ms 9.5878ms 104.2992 Ops/s 107.6266 Ops/s $\color{#d91a1a}-3.09\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.7917ms 35.4137ms 28.2376 Ops/s 28.2708 Ops/s $\color{#d91a1a}-0.12\%$
test_values[td0_return_estimate-False-False] 0.2087ms 0.1737ms 5.7571 KOps/s 5.9814 KOps/s $\color{#d91a1a}-3.75\%$
test_values[td1_return_estimate-False-False] 26.6199ms 23.6906ms 42.2108 Ops/s 42.8959 Ops/s $\color{#d91a1a}-1.60\%$
test_values[vec_td1_return_estimate-False-False] 54.0216ms 36.2520ms 27.5847 Ops/s 28.1019 Ops/s $\color{#d91a1a}-1.84\%$
test_values[td_lambda_return_estimate-True-False] 37.3634ms 34.1951ms 29.2440 Ops/s 29.6418 Ops/s $\color{#d91a1a}-1.34\%$
test_values[vec_td_lambda_return_estimate-True-False] 51.2042ms 36.1542ms 27.6593 Ops/s 28.2790 Ops/s $\color{#d91a1a}-2.19\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.4019ms 8.3153ms 120.2605 Ops/s 122.2073 Ops/s $\color{#d91a1a}-1.59\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.3432ms 2.0342ms 491.6042 Ops/s 504.4027 Ops/s $\color{#d91a1a}-2.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4213ms 0.3497ms 2.8598 KOps/s 2.8285 KOps/s $\color{#35bf28}+1.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.9795ms 45.8941ms 21.7893 Ops/s 21.5981 Ops/s $\color{#35bf28}+0.89\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.5508ms 3.0341ms 329.5916 Ops/s 329.2844 Ops/s $\color{#35bf28}+0.09\%$
test_dqn_speed 1.4786ms 1.3349ms 749.1281 Ops/s 743.4142 Ops/s $\color{#35bf28}+0.77\%$
test_ddpg_speed 3.5242ms 2.8252ms 353.9605 Ops/s 351.4148 Ops/s $\color{#35bf28}+0.72\%$
test_sac_speed 9.2503ms 8.3618ms 119.5909 Ops/s 119.9786 Ops/s $\color{#d91a1a}-0.32\%$
test_redq_speed 14.5662ms 13.2100ms 75.7001 Ops/s 75.3641 Ops/s $\color{#35bf28}+0.45\%$
test_redq_deprec_speed 14.2454ms 13.5868ms 73.6009 Ops/s 75.3033 Ops/s $\color{#d91a1a}-2.26\%$
test_td3_speed 17.0625ms 8.4824ms 117.8906 Ops/s 118.2089 Ops/s $\color{#d91a1a}-0.27\%$
test_cql_speed 38.9156ms 37.1674ms 26.9053 Ops/s 27.3938 Ops/s $\color{#d91a1a}-1.78\%$
test_a2c_speed 8.4792ms 7.6307ms 131.0501 Ops/s 134.5996 Ops/s $\color{#d91a1a}-2.64\%$
test_ppo_speed 9.1414ms 8.0192ms 124.7005 Ops/s 129.8942 Ops/s $\color{#d91a1a}-4.00\%$
test_reinforce_speed 7.5820ms 6.8648ms 145.6702 Ops/s 151.2162 Ops/s $\color{#d91a1a}-3.67\%$
test_iql_speed 34.8744ms 33.5582ms 29.7990 Ops/s 30.1389 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8847ms 3.6808ms 271.6780 Ops/s 298.0113 Ops/s $\textbf{\color{#d91a1a}-8.84\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0612ms 0.4995ms 2.0022 KOps/s 1.9994 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6695ms 0.4787ms 2.0891 KOps/s 2.0645 KOps/s $\color{#35bf28}+1.19\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.1691ms 3.5908ms 278.4922 Ops/s 288.7587 Ops/s $\color{#d91a1a}-3.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0403ms 0.4858ms 2.0586 KOps/s 2.0401 KOps/s $\color{#35bf28}+0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7708ms 0.4710ms 2.1230 KOps/s 1.9921 KOps/s $\textbf{\color{#35bf28}+6.57\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2805ms 1.6788ms 595.6721 Ops/s 573.3820 Ops/s $\color{#35bf28}+3.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.4462ms 1.5949ms 627.0131 Ops/s 628.5803 Ops/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4473ms 3.7069ms 269.7637 Ops/s 279.5591 Ops/s $\color{#d91a1a}-3.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2124ms 0.6280ms 1.5924 KOps/s 1.6196 KOps/s $\color{#d91a1a}-1.68\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.4821ms 0.5882ms 1.7000 KOps/s 1.6635 KOps/s $\color{#35bf28}+2.20\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.5260ms 3.5823ms 279.1533 Ops/s 275.4522 Ops/s $\color{#35bf28}+1.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5936ms 0.4968ms 2.0130 KOps/s 1.9826 KOps/s $\color{#35bf28}+1.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8154ms 0.5142ms 1.9448 KOps/s 2.0841 KOps/s $\textbf{\color{#d91a1a}-6.69\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.5130ms 3.5907ms 278.4940 Ops/s 273.7596 Ops/s $\color{#35bf28}+1.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1372ms 0.4943ms 2.0232 KOps/s 2.0215 KOps/s $\color{#35bf28}+0.08\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8209ms 0.4733ms 2.1128 KOps/s 2.1483 KOps/s $\color{#d91a1a}-1.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.1729ms 3.6841ms 271.4357 Ops/s 279.3318 Ops/s $\color{#d91a1a}-2.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7025ms 0.6102ms 1.6389 KOps/s 1.6267 KOps/s $\color{#35bf28}+0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.6542ms 0.6068ms 1.6479 KOps/s 1.7095 KOps/s $\color{#d91a1a}-3.60\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1186s 5.9665ms 167.6027 Ops/s 166.7235 Ops/s $\color{#35bf28}+0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1240s 14.5896ms 68.5418 Ops/s 69.9236 Ops/s $\color{#d91a1a}-1.98\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5212ms 1.0368ms 964.5149 Ops/s 949.1350 Ops/s $\color{#35bf28}+1.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1064s 5.7387ms 174.2567 Ops/s 171.7282 Ops/s $\color{#35bf28}+1.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.2639ms 12.6456ms 79.0787 Ops/s 75.3216 Ops/s $\color{#35bf28}+4.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.5579ms 1.0534ms 949.3357 Ops/s 946.8084 Ops/s $\color{#35bf28}+0.27\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1154s 8.2754ms 120.8400 Ops/s 126.3152 Ops/s $\color{#d91a1a}-4.33\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.5491ms 12.6499ms 79.0517 Ops/s 79.9658 Ops/s $\color{#d91a1a}-1.14\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9888ms 1.2044ms 830.3077 Ops/s 834.7574 Ops/s $\color{#d91a1a}-0.53\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1221s 0.1202s 8.3205 Ops/s 8.4016 Ops/s $\color{#d91a1a}-0.96\%$
test_sync 0.1060s 0.1053s 9.5010 Ops/s 9.6062 Ops/s $\color{#d91a1a}-1.10\%$
test_async 0.1991s 97.8119ms 10.2237 Ops/s 12.1182 Ops/s $\textbf{\color{#d91a1a}-15.63\%}$
test_single_pixels 0.1301s 0.1300s 7.6900 Ops/s 7.6893 Ops/s $+0.01\%$
test_sync_pixels 84.9554ms 83.5251ms 11.9724 Ops/s 12.4379 Ops/s $\color{#d91a1a}-3.74\%$
test_async_pixels 0.1595s 67.2996ms 14.8589 Ops/s 14.7496 Ops/s $\color{#35bf28}+0.74\%$
test_simple 0.9014s 0.8399s 1.1907 Ops/s 1.2035 Ops/s $\color{#d91a1a}-1.07\%$
test_transformed 1.1781s 1.1099s 0.9010 Ops/s 0.9191 Ops/s $\color{#d91a1a}-1.97\%$
test_serial 2.6140s 2.5511s 0.3920 Ops/s 0.3996 Ops/s $\color{#d91a1a}-1.91\%$
test_parallel 2.4229s 2.3644s 0.4229 Ops/s 0.4288 Ops/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-True-True-True-True] 75.4610μs 33.5481μs 29.8080 KOps/s 29.6576 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-True-True-False] 36.4510μs 19.9736μs 50.0660 KOps/s 50.1642 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[True-True-True-False-True] 40.0410μs 19.0918μs 52.3785 KOps/s 50.9426 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-True-True-False-False] 28.6910μs 11.3426μs 88.1636 KOps/s 87.6389 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-True-True] 58.2310μs 35.3955μs 28.2522 KOps/s 28.9426 KOps/s $\color{#d91a1a}-2.39\%$
test_step_mdp_speed[True-True-False-True-False] 38.1210μs 21.9269μs 45.6061 KOps/s 46.6822 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[True-True-False-False-True] 44.6810μs 20.8228μs 48.0242 KOps/s 47.9814 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[True-True-False-False-False] 30.5210μs 13.1947μs 75.7879 KOps/s 75.9502 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-True-True] 56.9810μs 37.7407μs 26.4966 KOps/s 26.9402 KOps/s $\color{#d91a1a}-1.65\%$
test_step_mdp_speed[True-False-True-True-False] 42.7810μs 24.0607μs 41.5616 KOps/s 42.3399 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[True-False-True-False-True] 35.5310μs 21.0441μs 47.5193 KOps/s 48.2139 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-False-True-False-False] 30.0800μs 13.2550μs 75.4433 KOps/s 75.5455 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-False-False-True-True] 63.2010μs 39.0526μs 25.6065 KOps/s 25.6326 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-False-False-True-False] 40.2910μs 25.6766μs 38.9460 KOps/s 39.2806 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-False-False-False-True] 39.2110μs 22.7070μs 44.0392 KOps/s 44.0876 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-False-False-False-False] 43.9910μs 14.9810μs 66.7513 KOps/s 66.1717 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-True-True-True-True] 54.1810μs 37.5003μs 26.6665 KOps/s 26.7724 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-True-True-False] 47.6710μs 24.3331μs 41.0963 KOps/s 42.2820 KOps/s $\color{#d91a1a}-2.80\%$
test_step_mdp_speed[False-True-True-False-True] 42.3910μs 25.3802μs 39.4008 KOps/s 39.8475 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[False-True-True-False-False] 34.8610μs 15.2316μs 65.6529 KOps/s 66.0447 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-False-True-True] 62.9010μs 39.3166μs 25.4345 KOps/s 25.6657 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-True-False-True-False] 42.2210μs 26.0637μs 38.3676 KOps/s 39.0777 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-True-False-False-True] 41.4910μs 27.1432μs 36.8416 KOps/s 37.0565 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-False-False-False] 35.2410μs 16.9936μs 58.8456 KOps/s 59.2819 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-True-True-True] 60.7110μs 41.0959μs 24.3333 KOps/s 24.5761 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[False-False-True-True-False] 44.5610μs 27.7200μs 36.0750 KOps/s 36.6095 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[False-False-True-False-True] 42.7710μs 27.4106μs 36.4823 KOps/s 37.1521 KOps/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[False-False-True-False-False] 41.3810μs 17.0282μs 58.7260 KOps/s 58.7387 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[False-False-False-True-True] 56.6610μs 43.4046μs 23.0390 KOps/s 23.0174 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-False-False-True-False] 58.4710μs 29.3698μs 34.0485 KOps/s 33.7899 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-False-False-False-True] 44.8510μs 28.7401μs 34.7946 KOps/s 35.1182 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-False-False-False-False] 44.5810μs 18.8629μs 53.0142 KOps/s 52.9641 KOps/s $\color{#35bf28}+0.09\%$
test_values[generalized_advantage_estimate-True-True] 26.5409ms 25.6621ms 38.9680 Ops/s 39.2593 Ops/s $\color{#d91a1a}-0.74\%$
test_values[vec_generalized_advantage_estimate-True-True] 92.1864ms 2.7563ms 362.8000 Ops/s 372.1684 Ops/s $\color{#d91a1a}-2.52\%$
test_values[td0_return_estimate-False-False] 96.7820μs 67.7320μs 14.7641 KOps/s 14.6163 KOps/s $\color{#35bf28}+1.01\%$
test_values[td1_return_estimate-False-False] 57.3506ms 56.9547ms 17.5578 Ops/s 17.3468 Ops/s $\color{#35bf28}+1.22\%$
test_values[vec_td1_return_estimate-False-False] 1.3842ms 1.1012ms 908.1297 Ops/s 902.1037 Ops/s $\color{#35bf28}+0.67\%$
test_values[td_lambda_return_estimate-True-False] 90.8736ms 90.3437ms 11.0688 Ops/s 11.0965 Ops/s $\color{#d91a1a}-0.25\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4155ms 1.0985ms 910.3370 Ops/s 904.1009 Ops/s $\color{#35bf28}+0.69\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.8583ms 25.5602ms 39.1233 Ops/s 38.3778 Ops/s $\color{#35bf28}+1.94\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9722ms 0.7417ms 1.3482 KOps/s 1.3497 KOps/s $\color{#d91a1a}-0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7684ms 0.6851ms 1.4597 KOps/s 1.4460 KOps/s $\color{#35bf28}+0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5462ms 1.4901ms 671.0834 Ops/s 671.4754 Ops/s $\color{#d91a1a}-0.06\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7586ms 0.7008ms 1.4269 KOps/s 1.4176 KOps/s $\color{#35bf28}+0.65\%$
test_dqn_speed 1.8736ms 1.4965ms 668.2208 Ops/s 691.0004 Ops/s $\color{#d91a1a}-3.30\%$
test_ddpg_speed 3.4156ms 3.0329ms 329.7164 Ops/s 336.8856 Ops/s $\color{#d91a1a}-2.13\%$
test_sac_speed 9.0792ms 8.7080ms 114.8369 Ops/s 116.8112 Ops/s $\color{#d91a1a}-1.69\%$
test_redq_speed 11.4843ms 10.8180ms 92.4389 Ops/s 92.4631 Ops/s $\color{#d91a1a}-0.03\%$
test_redq_deprec_speed 12.5907ms 12.1081ms 82.5891 Ops/s 84.3532 Ops/s $\color{#d91a1a}-2.09\%$
test_td3_speed 9.9069ms 8.9497ms 111.7360 Ops/s 117.2711 Ops/s $\color{#d91a1a}-4.72\%$
test_cql_speed 27.6061ms 26.5700ms 37.6365 Ops/s 37.5719 Ops/s $\color{#35bf28}+0.17\%$
test_a2c_speed 6.3170ms 5.9940ms 166.8339 Ops/s 171.8188 Ops/s $\color{#d91a1a}-2.90\%$
test_ppo_speed 7.1601ms 6.2113ms 160.9964 Ops/s 163.1917 Ops/s $\color{#d91a1a}-1.35\%$
test_reinforce_speed 5.1236ms 4.8604ms 205.7453 Ops/s 210.6341 Ops/s $\color{#d91a1a}-2.32\%$
test_iql_speed 20.9269ms 20.2487ms 49.3860 Ops/s 45.6389 Ops/s $\textbf{\color{#35bf28}+8.21\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1725ms 4.9226ms 203.1458 Ops/s 202.4107 Ops/s $\color{#35bf28}+0.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8214ms 0.6095ms 1.6407 KOps/s 1.6432 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8142ms 0.5827ms 1.7162 KOps/s 1.7242 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1344ms 4.8798ms 204.9251 Ops/s 207.8457 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7256ms 0.6029ms 1.6587 KOps/s 1.6578 KOps/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0204ms 0.5752ms 1.7387 KOps/s 1.7443 KOps/s $\color{#d91a1a}-0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.3643ms 2.1524ms 464.5925 Ops/s 460.7145 Ops/s $\color{#35bf28}+0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2615ms 2.0479ms 488.3167 Ops/s 485.1750 Ops/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1516ms 5.0566ms 197.7606 Ops/s 200.0207 Ops/s $\color{#d91a1a}-1.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3342ms 0.7328ms 1.3647 KOps/s 1.3073 KOps/s $\color{#35bf28}+4.39\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8636ms 0.7113ms 1.4059 KOps/s 1.3885 KOps/s $\color{#35bf28}+1.26\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0659ms 4.9259ms 203.0071 Ops/s 205.6002 Ops/s $\color{#d91a1a}-1.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7425ms 0.6057ms 1.6510 KOps/s 1.6461 KOps/s $\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.5534ms 0.5872ms 1.7029 KOps/s 1.7084 KOps/s $\color{#d91a1a}-0.32\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1127ms 4.8896ms 204.5147 Ops/s 203.0144 Ops/s $\color{#35bf28}+0.74\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7428ms 0.6062ms 1.6496 KOps/s 1.6425 KOps/s $\color{#35bf28}+0.43\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7237ms 0.5777ms 1.7309 KOps/s 1.7199 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.1952ms 5.0490ms 198.0597 Ops/s 199.5497 Ops/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8561ms 0.7371ms 1.3566 KOps/s 1.3637 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.8298ms 0.7203ms 1.3883 KOps/s 1.4086 KOps/s $\color{#d91a1a}-1.44\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1211s 9.5597ms 104.6062 Ops/s 133.0072 Ops/s $\textbf{\color{#d91a1a}-21.35\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.1678ms 16.7380ms 59.7441 Ops/s 58.4366 Ops/s $\color{#35bf28}+2.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.4663ms 1.3507ms 740.3698 Ops/s 751.7219 Ops/s $\color{#d91a1a}-1.51\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1081s 7.2369ms 138.1815 Ops/s 107.9823 Ops/s $\textbf{\color{#35bf28}+27.97\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 19.0260ms 16.7308ms 59.7699 Ops/s 58.6792 Ops/s $\color{#35bf28}+1.86\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4781ms 1.3749ms 727.3379 Ops/s 755.7922 Ops/s $\color{#d91a1a}-3.76\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1095s 9.6106ms 104.0520 Ops/s 134.6567 Ops/s $\textbf{\color{#d91a1a}-22.73\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.2260ms 16.8368ms 59.3938 Ops/s 58.4253 Ops/s $\color{#35bf28}+1.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5989ms 1.5044ms 664.6978 Ops/s 619.2818 Ops/s $\textbf{\color{#35bf28}+7.33\%}$

@vmoens vmoens deleted the fix-replacement-prefetch branch August 7, 2024 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot