Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Deprecate direct usage of memmap tensors #1684

Merged
merged 11 commits into from
Nov 15, 2023
Merged

[Refactor] Deprecate direct usage of memmap tensors #1684

merged 11 commits into from
Nov 15, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 8, 2023 •
edited
Loading

In preparation of pytorch/tensordict#541 or pytorch/tensordict#554 we have to look at our usages of MemmapTensors within torchrl, and move to MemoryMappedTensors instead.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 8, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1684

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 4 Unrelated Failures

As of commit 07cbbfe with merge base 02ff00d (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 8, 2023
@vmoens vmoens added the Refactoring Refactoring of an existing feature label Nov 14, 2023
@vmoens vmoens changed the title [WIP] Deprecate direct usage of memmap tensors [Refactor] Deprecate direct usage of memmap tensors Nov 14, 2023
@vmoens vmoens marked this pull request as ready for review November 14, 2023 21:12
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 14, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}8$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 61.4423ms 60.8901ms 16.4230 Ops/s 15.6049 Ops/s $\textbf{\color{#35bf28}+5.24\%}$
test_sync 34.8984ms 33.2939ms 30.0355 Ops/s 30.4300 Ops/s $\color{#d91a1a}-1.30\%$
test_async 64.3185ms 31.7937ms 31.4528 Ops/s 31.7908 Ops/s $\color{#d91a1a}-1.06\%$
test_simple 0.4799s 0.4321s 2.3145 Ops/s 2.3211 Ops/s $\color{#d91a1a}-0.28\%$
test_transformed 0.6291s 0.5803s 1.7233 Ops/s 1.6668 Ops/s $\color{#35bf28}+3.39\%$
test_serial 1.2740s 1.2250s 0.8163 Ops/s 0.7914 Ops/s $\color{#35bf28}+3.15\%$
test_parallel 1.2583s 1.2322s 0.8115 Ops/s 0.8157 Ops/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[True-True-True-True-True] 0.1565ms 24.8352μs 40.2655 KOps/s 41.1544 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[True-True-True-True-False] 43.8910μs 14.5463μs 68.7459 KOps/s 68.9491 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-True-False-True] 43.2200μs 15.2580μs 65.5394 KOps/s 66.1533 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-True-False-False] 29.9960μs 9.0112μs 110.9731 KOps/s 110.8086 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-True-False-True-True] 63.4280μs 26.6513μs 37.5217 KOps/s 38.2176 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[True-True-False-True-False] 42.6790μs 15.9454μs 62.7141 KOps/s 61.8519 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-False-True] 35.8170μs 17.0536μs 58.6386 KOps/s 59.9710 KOps/s $\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-True-False-False-False] 35.8570μs 10.4053μs 96.1051 KOps/s 95.9270 KOps/s $\color{#35bf28}+0.19\%$
test_step_mdp_speed[True-False-True-True-True] 76.9730μs 28.0499μs 35.6508 KOps/s 36.3332 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-False-True-True-False] 86.1010μs 17.7598μs 56.3068 KOps/s 56.6313 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-False-True-False-True] 59.7010μs 16.7975μs 59.5328 KOps/s 61.2124 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-False-True-False-False] 45.9570μs 10.3257μs 96.8462 KOps/s 96.3894 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-False-False-True-True] 70.5510μs 28.8898μs 34.6143 KOps/s 34.6806 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-False-False-True-False] 43.5700μs 18.7485μs 53.3375 KOps/s 52.5924 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-False-False-False-True] 61.8230μs 18.0023μs 55.5485 KOps/s 56.7167 KOps/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[True-False-False-False-False] 40.7260μs 11.6018μs 86.1935 KOps/s 85.4576 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-True-True-True-True] 75.9910μs 27.7417μs 36.0469 KOps/s 36.4677 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-True-True-True-False] 65.4110μs 17.4902μs 57.1750 KOps/s 56.3530 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-True-True-False-True] 43.0600μs 19.0972μs 52.3638 KOps/s 51.4083 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[False-True-True-False-False] 58.3690μs 11.5402μs 86.6534 KOps/s 85.1033 KOps/s $\color{#35bf28}+1.82\%$
test_step_mdp_speed[False-True-False-True-True] 66.7440μs 29.3645μs 34.0547 KOps/s 34.4339 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-True-False-True-False] 68.9480μs 18.7991μs 53.1940 KOps/s 52.3937 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-True-False-False-True] 54.8420μs 20.4358μs 48.9337 KOps/s 47.6806 KOps/s $\color{#35bf28}+2.63\%$
test_step_mdp_speed[False-True-False-False-False] 42.2080μs 12.8302μs 77.9409 KOps/s 76.0103 KOps/s $\color{#35bf28}+2.54\%$
test_step_mdp_speed[False-False-True-True-True] 76.2220μs 30.4325μs 32.8597 KOps/s 32.8835 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-False-True-True-False] 52.5080μs 19.7916μs 50.5266 KOps/s 49.9108 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-True-False-True] 61.8830μs 20.4491μs 48.9018 KOps/s 48.7514 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-True-False-False] 58.7690μs 13.0681μs 76.5222 KOps/s 77.0499 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[False-False-False-True-True] 72.0440μs 31.2043μs 32.0469 KOps/s 31.8876 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-False-True-False] 67.0150μs 21.3286μs 46.8853 KOps/s 47.1098 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-False-False-True] 55.8940μs 21.4965μs 46.5191 KOps/s 46.6461 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-False-False-False] 53.7200μs 13.9935μs 71.4619 KOps/s 70.3987 KOps/s $\color{#35bf28}+1.51\%$
test_values[generalized_advantage_estimate-True-True] 14.0065ms 11.9182ms 83.9051 Ops/s 83.0290 Ops/s $\color{#35bf28}+1.06\%$
test_values[vec_generalized_advantage_estimate-True-True] 33.9069ms 26.2898ms 38.0375 Ops/s 37.7315 Ops/s $\color{#35bf28}+0.81\%$
test_values[td0_return_estimate-False-False] 0.2497ms 0.1750ms 5.7147 KOps/s 5.6896 KOps/s $\color{#35bf28}+0.44\%$
test_values[td1_return_estimate-False-False] 25.8864ms 25.4420ms 39.3051 Ops/s 38.5243 Ops/s $\color{#35bf28}+2.03\%$
test_values[vec_td1_return_estimate-False-False] 96.0216ms 28.0412ms 35.6619 Ops/s 37.3724 Ops/s $\color{#d91a1a}-4.58\%$
test_values[td_lambda_return_estimate-True-False] 36.8101ms 35.5880ms 28.0994 Ops/s 27.7994 Ops/s $\color{#35bf28}+1.08\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.2555ms 26.4046ms 37.8721 Ops/s 37.6772 Ops/s $\color{#35bf28}+0.52\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1591ms 8.0742ms 123.8511 Ops/s 122.4836 Ops/s $\color{#35bf28}+1.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2250ms 1.9327ms 517.4164 Ops/s 511.2546 Ops/s $\color{#35bf28}+1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.5260ms 0.4326ms 2.3114 KOps/s 2.3107 KOps/s $\color{#35bf28}+0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.2420ms 40.0769ms 24.9520 Ops/s 24.6734 Ops/s $\color{#35bf28}+1.13\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 10.7299ms 2.5097ms 398.4556 Ops/s 399.7401 Ops/s $\color{#d91a1a}-0.32\%$
test_dqn_speed 9.9593ms 1.6710ms 598.4564 Ops/s 556.0820 Ops/s $\textbf{\color{#35bf28}+7.62\%}$
test_ddpg_speed 96.5195ms 3.4240ms 292.0562 Ops/s 328.3068 Ops/s $\textbf{\color{#d91a1a}-11.04\%}$
test_sac_speed 16.4273ms 8.4512ms 118.3260 Ops/s 117.9149 Ops/s $\color{#35bf28}+0.35\%$
test_redq_speed 24.0938ms 15.9629ms 62.6454 Ops/s 61.5753 Ops/s $\color{#35bf28}+1.74\%$
test_redq_deprec_speed 22.3248ms 14.1546ms 70.6483 Ops/s 70.8358 Ops/s $\color{#d91a1a}-0.26\%$
test_td3_speed 9.3595ms 8.5537ms 116.9078 Ops/s 115.6712 Ops/s $\color{#35bf28}+1.07\%$
test_cql_speed 36.9810ms 35.4869ms 28.1794 Ops/s 26.2423 Ops/s $\textbf{\color{#35bf28}+7.38\%}$
test_a2c_speed 16.1632ms 7.8497ms 127.3936 Ops/s 125.9668 Ops/s $\color{#35bf28}+1.13\%$
test_ppo_speed 16.6131ms 8.1371ms 122.8936 Ops/s 121.2225 Ops/s $\color{#35bf28}+1.38\%$
test_reinforce_speed 15.9184ms 6.9709ms 143.4533 Ops/s 142.1131 Ops/s $\color{#35bf28}+0.94\%$
test_iql_speed 40.0271ms 32.1405ms 31.1134 Ops/s 29.7794 Ops/s $\color{#35bf28}+4.48\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2526ms 1.7923ms 557.9408 Ops/s 535.7012 Ops/s $\color{#35bf28}+4.15\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.9748ms 1.8829ms 531.0837 Ops/s 493.3651 Ops/s $\textbf{\color{#35bf28}+7.65\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2.8437ms 1.8946ms 527.8145 Ops/s 503.5275 Ops/s $\color{#35bf28}+4.82\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.1002ms 1.7827ms 560.9469 Ops/s 559.4494 Ops/s $\color{#35bf28}+0.27\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.7335ms 1.8821ms 531.3173 Ops/s 521.7585 Ops/s $\color{#35bf28}+1.83\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2.6817ms 1.8809ms 531.6656 Ops/s 517.7196 Ops/s $\color{#35bf28}+2.69\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3893ms 1.7557ms 569.5641 Ops/s 559.4625 Ops/s $\color{#35bf28}+1.81\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.9238ms 1.8801ms 531.8793 Ops/s 519.2968 Ops/s $\color{#35bf28}+2.42\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1136s 2.1420ms 466.8576 Ops/s 530.3512 Ops/s $\textbf{\color{#d91a1a}-11.97\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3309ms 1.7784ms 562.2878 Ops/s 558.4284 Ops/s $\color{#35bf28}+0.69\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.5833ms 1.8790ms 532.1968 Ops/s 519.0816 Ops/s $\color{#35bf28}+2.53\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2.7187ms 1.8983ms 526.7734 Ops/s 499.1327 Ops/s $\textbf{\color{#35bf28}+5.54\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4870ms 1.7921ms 557.9936 Ops/s 557.7541 Ops/s $\color{#35bf28}+0.04\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1152s 2.2346ms 447.5109 Ops/s 516.2707 Ops/s $\textbf{\color{#d91a1a}-13.32\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.4674ms 1.9806ms 504.8938 Ops/s 463.1316 Ops/s $\textbf{\color{#35bf28}+9.02\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.4226ms 1.8005ms 555.4022 Ops/s 562.4362 Ops/s $\color{#d91a1a}-1.25\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.7515ms 1.8757ms 533.1436 Ops/s 531.1338 Ops/s $\color{#35bf28}+0.38\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1177s 2.1750ms 459.7756 Ops/s 517.9138 Ops/s $\textbf{\color{#d91a1a}-11.23\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2163s 20.2998ms 49.2616 Ops/s 47.0923 Ops/s $\color{#35bf28}+4.61\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1033s 20.0111ms 49.9724 Ops/s 55.1077 Ops/s $\textbf{\color{#d91a1a}-9.32\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1025s 17.6800ms 56.5610 Ops/s 50.7709 Ops/s $\textbf{\color{#35bf28}+11.40\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1258s 20.4510ms 48.8973 Ops/s 55.4498 Ops/s $\textbf{\color{#d91a1a}-11.82\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1234s 18.7088ms 53.4508 Ops/s 50.2588 Ops/s $\textbf{\color{#35bf28}+6.35\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1072s 20.3384ms 49.1680 Ops/s 49.5299 Ops/s $\color{#d91a1a}-0.73\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1230s 19.4953ms 51.2943 Ops/s 50.2846 Ops/s $\color{#35bf28}+2.01\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1200s 21.2914ms 46.9674 Ops/s 45.4954 Ops/s $\color{#35bf28}+3.24\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1074s 18.3404ms 54.5245 Ops/s 52.2311 Ops/s $\color{#35bf28}+4.39\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 14, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.1547ms 58.2984ms 17.1531 Ops/s 16.6733 Ops/s $\color{#35bf28}+2.88\%$
test_sync 37.9541ms 31.3840ms 31.8634 Ops/s 31.9708 Ops/s $\color{#d91a1a}-0.34\%$
test_async 52.5214ms 29.0212ms 34.4575 Ops/s 33.6614 Ops/s $\color{#35bf28}+2.36\%$
test_simple 0.4567s 0.4090s 2.4452 Ops/s 2.4388 Ops/s $\color{#35bf28}+0.26\%$
test_transformed 0.6043s 0.5643s 1.7721 Ops/s 1.7599 Ops/s $\color{#35bf28}+0.70\%$
test_serial 1.2039s 1.1535s 0.8669 Ops/s 0.8420 Ops/s $\color{#35bf28}+2.96\%$
test_parallel 1.1104s 1.0553s 0.9476 Ops/s 0.9336 Ops/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[True-True-True-True-True] 55.4640μs 20.0247μs 49.9383 KOps/s 50.1451 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-True-True-True-False] 33.5120μs 11.9124μs 83.9460 KOps/s 83.4544 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-True-True-False-True] 40.7060μs 12.3669μs 80.8610 KOps/s 81.1355 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-False-False] 33.5120μs 7.4180μs 134.8066 KOps/s 137.1004 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-True-False-True-True] 72.4960μs 20.9822μs 47.6595 KOps/s 47.4338 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[True-True-False-True-False] 35.6370μs 13.2127μs 75.6847 KOps/s 76.3831 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-True-False-False-True] 38.3620μs 13.4732μs 74.2216 KOps/s 74.2495 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-False-False-False] 38.5020μs 8.4152μs 118.8326 KOps/s 116.9067 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-False-True-True-True] 60.5620μs 21.9809μs 45.4940 KOps/s 44.8355 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-True-False] 39.9040μs 14.1961μs 70.4419 KOps/s 69.9237 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-False-True-False-True] 52.0880μs 13.7951μs 72.4895 KOps/s 74.4422 KOps/s $\color{#d91a1a}-2.62\%$
test_step_mdp_speed[True-False-True-False-False] 27.1910μs 8.3918μs 119.1636 KOps/s 118.6421 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-False-False-True-True] 74.0680μs 23.2750μs 42.9645 KOps/s 43.0817 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-False-True-False] 40.6050μs 15.4546μs 64.7058 KOps/s 65.0323 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-False-False-False-True] 46.4170μs 14.5494μs 68.7314 KOps/s 69.2072 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-False-False-False-False] 31.0380μs 9.5215μs 105.0251 KOps/s 104.4897 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-True-True-True-True] 84.4400μs 22.5126μs 44.4196 KOps/s 45.2419 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-True-True-True-False] 49.1810μs 14.2832μs 70.0125 KOps/s 69.9449 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-False-True] 54.4720μs 15.2735μs 65.4727 KOps/s 65.1461 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-True-True-False-False] 41.0170μs 9.5678μs 104.5175 KOps/s 104.6147 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-True-False-True-True] 49.9830μs 23.1688μs 43.1615 KOps/s 43.2511 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-True-False-True-False] 47.3390μs 15.2681μs 65.4962 KOps/s 65.0465 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-True-False-False-True] 40.9370μs 16.3039μs 61.3349 KOps/s 60.5873 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-True-False-False-False] 44.4530μs 10.4957μs 95.2772 KOps/s 94.5540 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-True-True-True] 52.1380μs 24.2009μs 41.3208 KOps/s 40.9883 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-False-True-True-False] 48.3400μs 16.3434μs 61.1867 KOps/s 60.9056 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-False-True-False-True] 39.5030μs 16.2596μs 61.5022 KOps/s 60.1740 KOps/s $\color{#35bf28}+2.21\%$
test_step_mdp_speed[False-False-True-False-False] 46.4240μs 10.4851μs 95.3733 KOps/s 94.2682 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-False-True-True] 50.6540μs 24.9948μs 40.0083 KOps/s 39.6127 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-False-False-True-False] 66.6570μs 17.3711μs 57.5669 KOps/s 57.8085 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-False-False-False-True] 51.6260μs 17.1322μs 58.3695 KOps/s 57.4583 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-False-False-False-False] 50.4540μs 11.6397μs 85.9125 KOps/s 86.7468 KOps/s $\color{#d91a1a}-0.96\%$
test_values[generalized_advantage_estimate-True-True] 12.1967ms 11.9403ms 83.7501 Ops/s 83.3169 Ops/s $\color{#35bf28}+0.52\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.9565ms 26.6483ms 37.5258 Ops/s 37.1551 Ops/s $\color{#35bf28}+1.00\%$
test_values[td0_return_estimate-False-False] 0.2191ms 0.1766ms 5.6625 KOps/s 5.6021 KOps/s $\color{#35bf28}+1.08\%$
test_values[td1_return_estimate-False-False] 27.1533ms 25.4729ms 39.2574 Ops/s 39.4305 Ops/s $\color{#d91a1a}-0.44\%$
test_values[vec_td1_return_estimate-False-False] 36.2425ms 26.4208ms 37.8489 Ops/s 37.7055 Ops/s $\color{#35bf28}+0.38\%$
test_values[td_lambda_return_estimate-True-False] 37.2830ms 35.8392ms 27.9024 Ops/s 27.9041 Ops/s $-0.01\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.6926ms 26.7564ms 37.3742 Ops/s 37.3612 Ops/s $\color{#35bf28}+0.03\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2475ms 7.9398ms 125.9474 Ops/s 127.4484 Ops/s $\color{#d91a1a}-1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 13.0522ms 1.8424ms 542.7701 Ops/s 485.5064 Ops/s $\textbf{\color{#35bf28}+11.79\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 3.2263ms 0.4353ms 2.2970 KOps/s 2.3312 KOps/s $\color{#d91a1a}-1.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9799ms 39.2321ms 25.4893 Ops/s 25.3012 Ops/s $\color{#35bf28}+0.74\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 12.9082ms 2.5237ms 396.2441 Ops/s 395.8426 Ops/s $\color{#35bf28}+0.10\%$
test_dqn_speed 12.3911ms 1.5695ms 637.1603 Ops/s 595.0518 Ops/s $\textbf{\color{#35bf28}+7.08\%}$
test_ddpg_speed 14.0741ms 2.8438ms 351.6479 Ops/s 344.4690 Ops/s $\color{#35bf28}+2.08\%$
test_sac_speed 19.7631ms 8.2339ms 121.4494 Ops/s 120.1923 Ops/s $\color{#35bf28}+1.05\%$
test_redq_speed 26.7635ms 15.0716ms 66.3500 Ops/s 66.0480 Ops/s $\color{#35bf28}+0.46\%$
test_redq_deprec_speed 25.4296ms 13.6802ms 73.0985 Ops/s 73.5400 Ops/s $\color{#d91a1a}-0.60\%$
test_td3_speed 9.8580ms 8.4120ms 118.8776 Ops/s 120.5748 Ops/s $\color{#d91a1a}-1.41\%$
test_cql_speed 48.1138ms 36.3963ms 27.4753 Ops/s 28.3048 Ops/s $\color{#d91a1a}-2.93\%$
test_a2c_speed 19.8972ms 8.2988ms 120.4992 Ops/s 126.3044 Ops/s $\color{#d91a1a}-4.60\%$
test_ppo_speed 20.0182ms 8.6865ms 115.1216 Ops/s 121.8529 Ops/s $\textbf{\color{#d91a1a}-5.52\%}$
test_reinforce_speed 19.1205ms 7.4141ms 134.8779 Ops/s 144.5046 Ops/s $\textbf{\color{#d91a1a}-6.66\%}$
test_iql_speed 44.2221ms 32.7797ms 30.5067 Ops/s 31.6026 Ops/s $\color{#d91a1a}-3.47\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.0491ms 1.6191ms 617.6156 Ops/s 596.9469 Ops/s $\color{#35bf28}+3.46\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.7735ms 1.6703ms 598.6985 Ops/s 582.6280 Ops/s $\color{#35bf28}+2.76\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.1207s 1.9556ms 511.3538 Ops/s 564.1964 Ops/s $\textbf{\color{#d91a1a}-9.37\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.2686ms 1.5710ms 636.5495 Ops/s 616.1058 Ops/s $\color{#35bf28}+3.32\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.4186ms 1.6840ms 593.8186 Ops/s 582.6800 Ops/s $\color{#35bf28}+1.91\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.3276ms 1.6941ms 590.2877 Ops/s 581.2933 Ops/s $\color{#35bf28}+1.55\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.2740ms 1.5802ms 632.8387 Ops/s 623.0877 Ops/s $\color{#35bf28}+1.56\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1182s 1.9271ms 518.9097 Ops/s 570.3954 Ops/s $\textbf{\color{#d91a1a}-9.03\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2.8669ms 1.7289ms 578.3913 Ops/s 569.8950 Ops/s $\color{#35bf28}+1.49\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2969ms 1.5762ms 634.4297 Ops/s 617.7107 Ops/s $\color{#35bf28}+2.71\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.1911ms 1.7737ms 563.8041 Ops/s 568.3604 Ops/s $\color{#d91a1a}-0.80\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2.8806ms 1.7171ms 582.3706 Ops/s 581.7114 Ops/s $\color{#35bf28}+0.11\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.3070ms 1.5766ms 634.2747 Ops/s 612.2196 Ops/s $\color{#35bf28}+3.60\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.8171ms 1.7029ms 587.2417 Ops/s 581.2813 Ops/s $\color{#35bf28}+1.03\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2.6769ms 1.7151ms 583.0669 Ops/s 506.8966 Ops/s $\textbf{\color{#35bf28}+15.03\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.1085ms 1.6090ms 621.5078 Ops/s 613.5311 Ops/s $\color{#35bf28}+1.30\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1189s 1.9226ms 520.1244 Ops/s 579.2743 Ops/s $\textbf{\color{#d91a1a}-10.21\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.3343ms 1.7648ms 566.6392 Ops/s 565.7744 Ops/s $\color{#35bf28}+0.15\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2119s 19.2751ms 51.8805 Ops/s 47.0354 Ops/s $\textbf{\color{#35bf28}+10.30\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1120s 19.2715ms 51.8901 Ops/s 59.4257 Ops/s $\textbf{\color{#d91a1a}-12.68\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1134s 19.2749ms 51.8809 Ops/s 51.0386 Ops/s $\color{#35bf28}+1.65\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1055s 15.3864ms 64.9926 Ops/s 58.1726 Ops/s $\textbf{\color{#35bf28}+11.72\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1161s 19.1802ms 52.1370 Ops/s 52.2509 Ops/s $\color{#d91a1a}-0.22\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1036s 16.9750ms 58.9103 Ops/s 52.3075 Ops/s $\textbf{\color{#35bf28}+12.62\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1098s 19.1806ms 52.1361 Ops/s 55.8729 Ops/s $\textbf{\color{#d91a1a}-6.69\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1131s 17.3753ms 57.5531 Ops/s 51.2304 Ops/s $\textbf{\color{#35bf28}+12.34\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1136s 19.1170ms 52.3094 Ops/s 57.4062 Ops/s $\textbf{\color{#d91a1a}-8.88\%}$

@vmoens vmoens merged commit 0badd6e into main Nov 15, 2023
54 of 61 checks passed
@vmoens vmoens deleted the deprecate_direct_memmap branch November 15, 2023 13:53
@vmoens vmoens restored the deprecate_direct_memmap branch November 15, 2023 13:54
vmoens added a commit that referenced this pull request Nov 15, 2023
@vmoens
Revert "[Refactor] Deprecate direct usage of memmap tensors (#1684)"
584bfa3 
This reverts commit 0badd6e.
This was referenced Nov 15, 2023
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot