Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] RNG for RBs #2379

Merged
merged 4 commits into from
Aug 8, 2024
Merged

[Feature] RNG for RBs #2379

merged 4 commits into from
Aug 8, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 7, 2024 •
edited
Loading

Implements a generator argument in replay buffers that allows to control the local (RB-specific) seed.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 7, 2024
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2379

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 8 Unrelated Failures

As of commit 1fe487e with merge base 342450e (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens added the enhancement New feature or request label Aug 7, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.3034ms 58.9316ms 16.9688 Ops/s 16.9458 Ops/s $\color{#35bf28}+0.14\%$
test_sync 37.7364ms 31.3123ms 31.9363 Ops/s 31.3145 Ops/s $\color{#35bf28}+1.99\%$
test_async 75.6418ms 30.9039ms 32.3583 Ops/s 32.9850 Ops/s $\color{#d91a1a}-1.90\%$
test_simple 0.4964s 0.4200s 2.3807 Ops/s 2.3821 Ops/s $\color{#d91a1a}-0.06\%$
test_transformed 0.6477s 0.5801s 1.7240 Ops/s 1.7351 Ops/s $\color{#d91a1a}-0.64\%$
test_serial 1.3380s 1.2745s 0.7846 Ops/s 0.7911 Ops/s $\color{#d91a1a}-0.81\%$
test_parallel 1.1872s 1.1130s 0.8985 Ops/s 0.9044 Ops/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-True-True-True-True] 0.2175ms 25.3821μs 39.3979 KOps/s 39.7441 KOps/s $\color{#d91a1a}-0.87\%$
test_step_mdp_speed[True-True-True-True-False] 68.7480μs 14.3626μs 69.6252 KOps/s 69.4825 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[True-True-True-False-True] 59.5220μs 14.4114μs 69.3895 KOps/s 69.6290 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-True-False-False] 37.4190μs 8.2551μs 121.1377 KOps/s 120.0972 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-True-False-True-True] 62.8080μs 27.0857μs 36.9199 KOps/s 37.1133 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-True-False-True-False] 64.1500μs 15.8892μs 62.9358 KOps/s 62.4480 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-True-False-False-True] 50.9060μs 15.9298μs 62.7756 KOps/s 62.4238 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[True-True-False-False-False] 86.5000μs 9.7217μs 102.8624 KOps/s 101.6650 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-False-True-True-True] 71.9450μs 28.8137μs 34.7057 KOps/s 34.9953 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-False-True-True-False] 47.4290μs 17.6203μs 56.7526 KOps/s 56.8247 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-False-True-False-True] 47.6600μs 15.9821μs 62.5699 KOps/s 62.6712 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-False-True-False-False] 37.9010μs 9.7711μs 102.3429 KOps/s 102.1421 KOps/s $\color{#35bf28}+0.20\%$
test_step_mdp_speed[True-False-False-True-True] 70.7620μs 30.0300μs 33.3001 KOps/s 32.8924 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-False-False-True-False] 63.3790μs 19.0344μs 52.5365 KOps/s 52.3057 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-False-False-False-True] 63.0180μs 17.3693μs 57.5729 KOps/s 57.2797 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-False-False-False-False] 51.3680μs 11.1801μs 89.4447 KOps/s 87.8665 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-True-True-True-True] 0.1097ms 28.2466μs 35.4024 KOps/s 34.7672 KOps/s $\color{#35bf28}+1.83\%$
test_step_mdp_speed[False-True-True-True-False] 41.9780μs 17.4737μs 57.2289 KOps/s 56.5153 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-True-False-True] 54.7730μs 19.0501μs 52.4932 KOps/s 53.5884 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-True-True-False-False] 40.1750μs 11.0632μs 90.3897 KOps/s 89.4014 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-True-False-True-True] 76.1620μs 29.9500μs 33.3889 KOps/s 32.9471 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-True-False-True-False] 92.7040μs 19.1772μs 52.1453 KOps/s 52.0319 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-False-False-True] 57.9180μs 20.6581μs 48.4071 KOps/s 50.3642 KOps/s $\color{#d91a1a}-3.89\%$
test_step_mdp_speed[False-True-False-False-False] 51.1150μs 12.5314μs 79.7996 KOps/s 78.8286 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-True-True-True] 3.2630ms 32.0992μs 31.1535 KOps/s 31.8472 KOps/s $\color{#d91a1a}-2.18\%$
test_step_mdp_speed[False-False-True-True-False] 46.4170μs 20.5700μs 48.6146 KOps/s 48.2639 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[False-False-True-False-True] 75.0500μs 20.5213μs 48.7299 KOps/s 50.4744 KOps/s $\color{#d91a1a}-3.46\%$
test_step_mdp_speed[False-False-True-False-False] 39.3940μs 12.5528μs 79.6636 KOps/s 79.5609 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-False-True-True] 77.7050μs 32.9510μs 30.3481 KOps/s 30.4497 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-False-False-True-False] 63.8500μs 22.2920μs 44.8590 KOps/s 45.6232 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-False-False-False-True] 53.6710μs 21.9094μs 45.6426 KOps/s 47.5129 KOps/s $\color{#d91a1a}-3.94\%$
test_step_mdp_speed[False-False-False-False-False] 59.0610μs 13.9317μs 71.7785 KOps/s 72.2782 KOps/s $\color{#d91a1a}-0.69\%$
test_values[generalized_advantage_estimate-True-True] 9.9393ms 9.6074ms 104.0864 Ops/s 105.8132 Ops/s $\color{#d91a1a}-1.63\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.7013ms 33.6430ms 29.7239 Ops/s 28.1042 Ops/s $\textbf{\color{#35bf28}+5.76\%}$
test_values[td0_return_estimate-False-False] 0.2114ms 0.1786ms 5.5977 KOps/s 5.6331 KOps/s $\color{#d91a1a}-0.63\%$
test_values[td1_return_estimate-False-False] 23.7716ms 23.4712ms 42.6054 Ops/s 42.4893 Ops/s $\color{#35bf28}+0.27\%$
test_values[vec_td1_return_estimate-False-False] 35.1959ms 33.6491ms 29.7185 Ops/s 27.9691 Ops/s $\textbf{\color{#35bf28}+6.25\%}$
test_values[td_lambda_return_estimate-True-False] 55.8580ms 34.3612ms 29.1026 Ops/s 29.0266 Ops/s $\color{#35bf28}+0.26\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.8079ms 33.8478ms 29.5440 Ops/s 28.0199 Ops/s $\textbf{\color{#35bf28}+5.44\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.2985ms 8.1968ms 121.9982 Ops/s 120.6653 Ops/s $\color{#35bf28}+1.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3009ms 1.9922ms 501.9673 Ops/s 511.4978 Ops/s $\color{#d91a1a}-1.86\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4614ms 0.3592ms 2.7840 KOps/s 2.7531 KOps/s $\color{#35bf28}+1.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.5954ms 48.5190ms 20.6105 Ops/s 20.9149 Ops/s $\color{#d91a1a}-1.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.9180ms 3.0533ms 327.5157 Ops/s 329.0005 Ops/s $\color{#d91a1a}-0.45\%$
test_dqn_speed 6.1731ms 1.3294ms 752.2193 Ops/s 762.8412 Ops/s $\color{#d91a1a}-1.39\%$
test_ddpg_speed 3.4984ms 2.7657ms 361.5757 Ops/s 365.7519 Ops/s $\color{#d91a1a}-1.14\%$
test_sac_speed 10.1674ms 8.1732ms 122.3513 Ops/s 124.5202 Ops/s $\color{#d91a1a}-1.74\%$
test_redq_speed 13.6518ms 13.1293ms 76.1654 Ops/s 77.6021 Ops/s $\color{#d91a1a}-1.85\%$
test_redq_deprec_speed 15.4258ms 13.5972ms 73.5448 Ops/s 77.8071 Ops/s $\textbf{\color{#d91a1a}-5.48\%}$
test_td3_speed 8.4567ms 8.0774ms 123.8020 Ops/s 125.9465 Ops/s $\color{#d91a1a}-1.70\%$
test_cql_speed 37.4179ms 35.8913ms 27.8619 Ops/s 27.8683 Ops/s $\color{#d91a1a}-0.02\%$
test_a2c_speed 8.0793ms 7.4266ms 134.6519 Ops/s 137.0395 Ops/s $\color{#d91a1a}-1.74\%$
test_ppo_speed 8.8836ms 7.7597ms 128.8708 Ops/s 131.8831 Ops/s $\color{#d91a1a}-2.28\%$
test_reinforce_speed 7.4562ms 6.5612ms 152.4109 Ops/s 153.8607 Ops/s $\color{#d91a1a}-0.94\%$
test_iql_speed 33.8885ms 32.0496ms 31.2016 Ops/s 31.2485 Ops/s $\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1435ms 4.9340ms 202.6772 Ops/s 210.6635 Ops/s $\color{#d91a1a}-3.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7702ms 0.4827ms 2.0717 KOps/s 2.0972 KOps/s $\color{#d91a1a}-1.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6526ms 0.4602ms 2.1731 KOps/s 2.1911 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2124ms 4.9486ms 202.0756 Ops/s 205.9971 Ops/s $\color{#d91a1a}-1.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7103ms 0.4844ms 2.0644 KOps/s 2.1233 KOps/s $\color{#d91a1a}-2.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6747ms 0.4648ms 2.1514 KOps/s 2.2285 KOps/s $\color{#d91a1a}-3.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.8280ms 1.6872ms 592.6997 Ops/s 586.0007 Ops/s $\color{#35bf28}+1.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.9466ms 1.6036ms 623.6046 Ops/s 617.6628 Ops/s $\color{#35bf28}+0.96\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.4327ms 5.1159ms 195.4674 Ops/s 204.6041 Ops/s $\color{#d91a1a}-4.47\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.8459ms 0.6273ms 1.5940 KOps/s 1.6270 KOps/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8239ms 0.6087ms 1.6428 KOps/s 1.7046 KOps/s $\color{#d91a1a}-3.63\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5225ms 4.9258ms 203.0111 Ops/s 210.4160 Ops/s $\color{#d91a1a}-3.52\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2435ms 0.4826ms 2.0721 KOps/s 2.0994 KOps/s $\color{#d91a1a}-1.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6787ms 0.4641ms 2.1548 KOps/s 2.1672 KOps/s $\color{#d91a1a}-0.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2088ms 4.8451ms 206.3947 Ops/s 211.6424 Ops/s $\color{#d91a1a}-2.48\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7631ms 0.4751ms 2.1050 KOps/s 2.0957 KOps/s $\color{#35bf28}+0.44\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6892ms 0.4553ms 2.1963 KOps/s 2.2287 KOps/s $\color{#d91a1a}-1.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5340ms 5.0015ms 199.9388 Ops/s 207.6614 Ops/s $\color{#d91a1a}-3.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1236s 0.7840ms 1.2755 KOps/s 1.2786 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8574ms 0.5966ms 1.6762 KOps/s 1.7053 KOps/s $\color{#d91a1a}-1.71\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1140s 5.9835ms 167.1255 Ops/s 167.5959 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.2854ms 13.0531ms 76.6103 Ops/s 78.1658 Ops/s $\color{#d91a1a}-1.99\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.8838ms 1.1493ms 870.0856 Ops/s 932.5615 Ops/s $\textbf{\color{#d91a1a}-6.70\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1113s 5.8977ms 169.5573 Ops/s 172.4658 Ops/s $\color{#d91a1a}-1.69\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.1212ms 13.0074ms 76.8795 Ops/s 78.2919 Ops/s $\color{#d91a1a}-1.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8525ms 1.1292ms 885.6094 Ops/s 824.1795 Ops/s $\textbf{\color{#35bf28}+7.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1129s 6.1192ms 163.4203 Ops/s 120.4316 Ops/s $\textbf{\color{#35bf28}+35.70\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1174s 15.2831ms 65.4319 Ops/s 77.4057 Ops/s $\textbf{\color{#d91a1a}-15.47\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0036ms 1.2842ms 778.6953 Ops/s 823.9426 Ops/s $\textbf{\color{#d91a1a}-5.49\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1034s 0.1026s 9.7482 Ops/s 9.4512 Ops/s $\color{#35bf28}+3.14\%$
test_sync 94.0817ms 92.2628ms 10.8386 Ops/s 10.9821 Ops/s $\color{#d91a1a}-1.31\%$
test_async 0.1771s 84.2826ms 11.8648 Ops/s 11.6874 Ops/s $\color{#35bf28}+1.52\%$
test_single_pixels 0.1139s 0.1138s 8.7870 Ops/s 8.5429 Ops/s $\color{#35bf28}+2.86\%$
test_sync_pixels 76.6489ms 75.9458ms 13.1673 Ops/s 13.1371 Ops/s $\color{#35bf28}+0.23\%$
test_async_pixels 0.1445s 71.4588ms 13.9941 Ops/s 14.2739 Ops/s $\color{#d91a1a}-1.96\%$
test_simple 0.7300s 0.7287s 1.3724 Ops/s 1.3009 Ops/s $\textbf{\color{#35bf28}+5.49\%}$
test_transformed 1.0524s 0.9737s 1.0270 Ops/s 1.0041 Ops/s $\color{#35bf28}+2.27\%$
test_serial 2.1739s 2.1248s 0.4706 Ops/s 0.4633 Ops/s $\color{#35bf28}+1.57\%$
test_parallel 1.8919s 1.8727s 0.5340 Ops/s 0.5311 Ops/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[True-True-True-True-True] 95.3420μs 37.2676μs 26.8330 KOps/s 26.2569 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[True-True-True-True-False] 51.2310μs 20.9416μs 47.7520 KOps/s 46.5800 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[True-True-True-False-True] 46.7610μs 20.6041μs 48.5339 KOps/s 46.0856 KOps/s $\textbf{\color{#35bf28}+5.31\%}$
test_step_mdp_speed[True-True-True-False-False] 43.0500μs 11.8025μs 84.7281 KOps/s 82.0178 KOps/s $\color{#35bf28}+3.30\%$
test_step_mdp_speed[True-True-False-True-True] 91.5520μs 39.0015μs 25.6401 KOps/s 24.3330 KOps/s $\textbf{\color{#35bf28}+5.37\%}$
test_step_mdp_speed[True-True-False-True-False] 51.7710μs 23.0446μs 43.3942 KOps/s 42.0813 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[True-True-False-False-True] 59.8210μs 22.9203μs 43.6295 KOps/s 41.8022 KOps/s $\color{#35bf28}+4.37\%$
test_step_mdp_speed[True-True-False-False-False] 41.8010μs 14.1663μs 70.5901 KOps/s 69.4029 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-False-True-True-True] 80.8220μs 41.3470μs 24.1856 KOps/s 23.3452 KOps/s $\color{#35bf28}+3.60\%$
test_step_mdp_speed[True-False-True-True-False] 54.8010μs 25.3970μs 39.3748 KOps/s 38.0386 KOps/s $\color{#35bf28}+3.51\%$
test_step_mdp_speed[True-False-True-False-True] 56.7310μs 22.8265μs 43.8087 KOps/s 41.2203 KOps/s $\textbf{\color{#35bf28}+6.28\%}$
test_step_mdp_speed[True-False-True-False-False] 40.3310μs 14.1565μs 70.6390 KOps/s 68.6003 KOps/s $\color{#35bf28}+2.97\%$
test_step_mdp_speed[True-False-False-True-True] 84.8520μs 43.6575μs 22.9056 KOps/s 22.1414 KOps/s $\color{#35bf28}+3.45\%$
test_step_mdp_speed[True-False-False-True-False] 54.5910μs 27.2405μs 36.7100 KOps/s 35.2227 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[True-False-False-False-True] 57.7710μs 24.7849μs 40.3472 KOps/s 38.0333 KOps/s $\textbf{\color{#35bf28}+6.08\%}$
test_step_mdp_speed[True-False-False-False-False] 54.1710μs 16.2596μs 61.5022 KOps/s 59.7688 KOps/s $\color{#35bf28}+2.90\%$
test_step_mdp_speed[False-True-True-True-True] 68.6710μs 41.0216μs 24.3774 KOps/s 22.9738 KOps/s $\textbf{\color{#35bf28}+6.11\%}$
test_step_mdp_speed[False-True-True-True-False] 60.2620μs 25.1796μs 39.7147 KOps/s 38.2909 KOps/s $\color{#35bf28}+3.72\%$
test_step_mdp_speed[False-True-True-False-True] 66.4510μs 27.4833μs 36.3858 KOps/s 34.8552 KOps/s $\color{#35bf28}+4.39\%$
test_step_mdp_speed[False-True-True-False-False] 58.4510μs 16.1109μs 62.0696 KOps/s 60.7871 KOps/s $\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-True-False-True-True] 71.7520μs 43.3892μs 23.0472 KOps/s 21.9503 KOps/s $\color{#35bf28}+5.00\%$
test_step_mdp_speed[False-True-False-True-False] 60.5220μs 27.3431μs 36.5723 KOps/s 35.2941 KOps/s $\color{#35bf28}+3.62\%$
test_step_mdp_speed[False-True-False-False-True] 62.0610μs 29.5219μs 33.8731 KOps/s 32.4199 KOps/s $\color{#35bf28}+4.48\%$
test_step_mdp_speed[False-True-False-False-False] 50.4310μs 18.3293μs 54.5576 KOps/s 53.9868 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-True-True-True] 4.2792ms 45.9563μs 21.7598 KOps/s 20.8456 KOps/s $\color{#35bf28}+4.39\%$
test_step_mdp_speed[False-False-True-True-False] 61.3610μs 30.1091μs 33.2126 KOps/s 32.2180 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[False-False-True-False-True] 66.3120μs 29.6515μs 33.7251 KOps/s 32.8250 KOps/s $\color{#35bf28}+2.74\%$
test_step_mdp_speed[False-False-True-False-False] 49.8010μs 18.2874μs 54.6825 KOps/s 53.5642 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-False-False-True-True] 82.5210μs 47.4086μs 21.0932 KOps/s 20.2070 KOps/s $\color{#35bf28}+4.39\%$
test_step_mdp_speed[False-False-False-True-False] 75.7220μs 31.7337μs 31.5122 KOps/s 30.2349 KOps/s $\color{#35bf28}+4.22\%$
test_step_mdp_speed[False-False-False-False-True] 67.0310μs 31.1554μs 32.0972 KOps/s 31.0783 KOps/s $\color{#35bf28}+3.28\%$
test_step_mdp_speed[False-False-False-False-False] 50.7910μs 20.2864μs 49.2942 KOps/s 48.0388 KOps/s $\color{#35bf28}+2.61\%$
test_values[generalized_advantage_estimate-True-True] 23.2132ms 22.8177ms 43.8257 Ops/s 41.6816 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_values[vec_generalized_advantage_estimate-True-True] 98.6810ms 2.8483ms 351.0918 Ops/s 352.3448 Ops/s $\color{#d91a1a}-0.36\%$
test_values[td0_return_estimate-False-False] 89.3620μs 61.2068μs 16.3380 KOps/s 15.7726 KOps/s $\color{#35bf28}+3.58\%$
test_values[td1_return_estimate-False-False] 52.3001ms 50.6176ms 19.7560 Ops/s 18.0101 Ops/s $\textbf{\color{#35bf28}+9.69\%}$
test_values[vec_td1_return_estimate-False-False] 1.3841ms 1.0604ms 943.0054 Ops/s 930.8235 Ops/s $\color{#35bf28}+1.31\%$
test_values[td_lambda_return_estimate-True-False] 81.0567ms 80.3563ms 12.4446 Ops/s 11.3774 Ops/s $\textbf{\color{#35bf28}+9.38\%}$
test_values[vec_td_lambda_return_estimate-True-False] 1.2550ms 1.0480ms 954.2081 Ops/s 929.5218 Ops/s $\color{#35bf28}+2.66\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.8715ms 22.5920ms 44.2635 Ops/s 39.1871 Ops/s $\textbf{\color{#35bf28}+12.95\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9202ms 0.6791ms 1.4725 KOps/s 1.4238 KOps/s $\color{#35bf28}+3.42\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7525ms 0.6346ms 1.5757 KOps/s 1.5272 KOps/s $\color{#35bf28}+3.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4750ms 1.4329ms 697.8648 Ops/s 682.6052 Ops/s $\color{#35bf28}+2.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.6964ms 0.6469ms 1.5459 KOps/s 1.4716 KOps/s $\textbf{\color{#35bf28}+5.04\%}$
test_dqn_speed 7.7010ms 1.3785ms 725.4473 Ops/s 717.9895 Ops/s $\color{#35bf28}+1.04\%$
test_ddpg_speed 3.0221ms 2.7210ms 367.5150 Ops/s 359.2317 Ops/s $\color{#35bf28}+2.31\%$
test_sac_speed 8.1698ms 7.8073ms 128.0855 Ops/s 124.9419 Ops/s $\color{#35bf28}+2.52\%$
test_redq_speed 11.4858ms 10.0721ms 99.2840 Ops/s 94.9240 Ops/s $\color{#35bf28}+4.59\%$
test_redq_deprec_speed 11.6484ms 11.3415ms 88.1715 Ops/s 90.4582 Ops/s $\color{#d91a1a}-2.53\%$
test_td3_speed 8.0332ms 7.7894ms 128.3797 Ops/s 124.7720 Ops/s $\color{#35bf28}+2.89\%$
test_cql_speed 26.9432ms 25.0958ms 39.8473 Ops/s 39.0851 Ops/s $\color{#35bf28}+1.95\%$
test_a2c_speed 6.2273ms 5.6254ms 177.7644 Ops/s 178.0858 Ops/s $\color{#d91a1a}-0.18\%$
test_ppo_speed 6.0871ms 5.9019ms 169.4364 Ops/s 167.7871 Ops/s $\color{#35bf28}+0.98\%$
test_reinforce_speed 5.2205ms 4.5614ms 219.2320 Ops/s 220.6224 Ops/s $\color{#d91a1a}-0.63\%$
test_iql_speed 20.4663ms 19.5785ms 51.0764 Ops/s 51.8916 Ops/s $\color{#d91a1a}-1.57\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7752ms 6.5520ms 152.6261 Ops/s 149.3696 Ops/s $\color{#35bf28}+2.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1117s 0.5941ms 1.6832 KOps/s 1.9343 KOps/s $\textbf{\color{#d91a1a}-12.98\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7254ms 0.4975ms 2.0099 KOps/s 1.9791 KOps/s $\color{#35bf28}+1.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8993ms 6.4701ms 154.5571 Ops/s 150.0567 Ops/s $\color{#35bf28}+3.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5216ms 0.5105ms 1.9590 KOps/s 1.9493 KOps/s $\color{#35bf28}+0.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6697ms 0.4918ms 2.0333 KOps/s 2.0384 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1625ms 1.9376ms 516.0908 Ops/s 514.2788 Ops/s $\color{#35bf28}+0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.9926ms 1.8378ms 544.1431 Ops/s 538.2125 Ops/s $\color{#35bf28}+1.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1143ms 6.7226ms 148.7527 Ops/s 147.0204 Ops/s $\color{#35bf28}+1.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0294ms 0.6701ms 1.4924 KOps/s 1.5065 KOps/s $\color{#d91a1a}-0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8462ms 0.6575ms 1.5210 KOps/s 1.5559 KOps/s $\color{#d91a1a}-2.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8348ms 6.5925ms 151.6876 Ops/s 150.0560 Ops/s $\color{#35bf28}+1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1523ms 0.5175ms 1.9325 KOps/s 1.9298 KOps/s $\color{#35bf28}+0.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7575ms 0.4932ms 2.0277 KOps/s 2.0037 KOps/s $\color{#35bf28}+1.20\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7100ms 6.4729ms 154.4914 Ops/s 151.3915 Ops/s $\color{#35bf28}+2.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1626s 0.7831ms 1.2770 KOps/s 1.9573 KOps/s $\textbf{\color{#d91a1a}-34.76\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8287ms 0.4936ms 2.0258 KOps/s 2.0044 KOps/s $\color{#35bf28}+1.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9367ms 6.6971ms 149.3184 Ops/s 145.7780 Ops/s $\color{#35bf28}+2.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7557ms 0.6676ms 1.4978 KOps/s 1.4955 KOps/s $\color{#35bf28}+0.15\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.5709ms 0.6507ms 1.5369 KOps/s 1.2426 KOps/s $\textbf{\color{#35bf28}+23.68\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1358s 7.7288ms 129.3856 Ops/s 131.5199 Ops/s $\color{#d91a1a}-1.62\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.9202ms 15.8031ms 63.2785 Ops/s 61.6027 Ops/s $\color{#35bf28}+2.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2554ms 1.3761ms 726.6731 Ops/s 799.0067 Ops/s $\textbf{\color{#d91a1a}-9.05\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1263s 7.5823ms 131.8856 Ops/s 131.3191 Ops/s $\color{#35bf28}+0.43\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1381s 18.3100ms 54.6149 Ops/s 61.6174 Ops/s $\textbf{\color{#d91a1a}-11.36\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.1351ms 1.3295ms 752.1441 Ops/s 791.0841 Ops/s $\color{#d91a1a}-4.92\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1262s 7.7277ms 129.4049 Ops/s 128.5139 Ops/s $\color{#35bf28}+0.69\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.4073ms 16.0266ms 62.3962 Ops/s 61.0341 Ops/s $\color{#35bf28}+2.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6075ms 1.5071ms 663.5098 Ops/s 621.7604 Ops/s $\textbf{\color{#35bf28}+6.71\%}$

@vmoens vmoens merged commit 918bfe6 into main Aug 8, 2024
61 of 68 checks passed
@vmoens vmoens deleted the rng-samplers branch August 8, 2024 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot