Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Auto-resetting envs #2073

Merged
merged 18 commits into from
Apr 16, 2024
Merged

[Feature] Auto-resetting envs #2073

merged 18 commits into from
Apr 16, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 9, 2024 •
edited
Loading

Allows the creation of auto-resetting envs through env = SomeEnvClass(..., auto_reset=True)

This should be doable with all envs without requiring users to put auto_reset as an arg in the constructor.

cc @luisenp

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 9, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2073

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures, 4 Unrelated Failures

As of commit be647c4 with merge base bedd2b7 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 9, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 16, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.7407ms 53.1847ms 18.8024 Ops/s 18.0006 Ops/s $\color{#35bf28}+4.45\%$
test_sync 42.5357ms 34.7314ms 28.7924 Ops/s 34.7485 Ops/s $\textbf{\color{#d91a1a}-17.14\%}$
test_async 52.6196ms 28.1506ms 35.5232 Ops/s 37.8710 Ops/s $\textbf{\color{#d91a1a}-6.20\%}$
test_simple 0.3970s 0.3463s 2.8873 Ops/s 3.0196 Ops/s $\color{#d91a1a}-4.38\%$
test_transformed 0.4804s 0.4790s 2.0876 Ops/s 2.0462 Ops/s $\color{#35bf28}+2.02\%$
test_serial 1.2493s 1.2037s 0.8307 Ops/s 0.8410 Ops/s $\color{#d91a1a}-1.21\%$
test_parallel 1.0550s 1.0231s 0.9774 Ops/s 1.0011 Ops/s $\color{#d91a1a}-2.36\%$
test_step_mdp_speed[True-True-True-True-True] 0.1429ms 20.9766μs 47.6721 KOps/s 47.0775 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-True-True-True-False] 38.7520μs 12.9599μs 77.1610 KOps/s 76.8417 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-True-True-False-True] 42.1590μs 12.3753μs 80.8061 KOps/s 75.3362 KOps/s $\textbf{\color{#35bf28}+7.26\%}$
test_step_mdp_speed[True-True-True-False-False] 45.9890μs 7.6235μs 131.1725 KOps/s 131.6320 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-False-True-True] 64.3900μs 22.4640μs 44.5156 KOps/s 43.9075 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-False-True-False] 42.9200μs 14.2561μs 70.1454 KOps/s 69.3436 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-False-False-True] 36.4180μs 13.6971μs 73.0083 KOps/s 73.2915 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[True-True-False-False-False] 48.4710μs 8.9112μs 112.2180 KOps/s 113.9181 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[True-False-True-True-True] 61.6450μs 23.7611μs 42.0856 KOps/s 41.5729 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-False-True-True-False] 53.8910μs 15.4634μs 64.6688 KOps/s 64.0112 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-False-True-False-True] 38.4320μs 13.6410μs 73.3085 KOps/s 72.8582 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-False-True-False-False] 33.6130μs 8.8390μs 113.1346 KOps/s 111.5982 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-False-False-True-True] 51.3460μs 24.8535μs 40.2358 KOps/s 39.6846 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-False-False-True-False] 43.1710μs 16.6093μs 60.2072 KOps/s 59.0589 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-False-False-False-True] 46.7470μs 14.7994μs 67.5701 KOps/s 67.3923 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-False-False-False-False] 48.0400μs 9.9567μs 100.4344 KOps/s 100.7576 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-True-True] 59.8410μs 23.9185μs 41.8086 KOps/s 41.4511 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-True-True-True-False] 42.9600μs 15.4767μs 64.6132 KOps/s 64.1063 KOps/s $\color{#35bf28}+0.79\%$
test_step_mdp_speed[False-True-True-False-True] 43.2010μs 15.8823μs 62.9630 KOps/s 62.7992 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-True-True-False-False] 35.9980μs 10.0595μs 99.4089 KOps/s 99.6064 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-False-True-True] 52.2770μs 25.5758μs 39.0994 KOps/s 38.8609 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-True-False-True-False] 45.0340μs 16.6162μs 60.1823 KOps/s 59.6474 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-False-False-True] 56.8160μs 17.0239μs 58.7410 KOps/s 58.4952 KOps/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[False-True-False-False-False] 60.9040μs 11.0799μs 90.2534 KOps/s 89.1896 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-False-True-True-True] 62.2360μs 26.0600μs 38.3730 KOps/s 37.9467 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-False-True-True-False] 44.4430μs 17.8387μs 56.0578 KOps/s 55.4816 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-True-False-True] 40.9660μs 16.9652μs 58.9443 KOps/s 58.3873 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[False-False-True-False-False] 36.0470μs 11.1508μs 89.6794 KOps/s 89.0391 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-False-False-True-True] 67.1450μs 27.2178μs 36.7407 KOps/s 36.6421 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-False-True-False] 53.7600μs 18.8505μs 53.0491 KOps/s 52.3747 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-False-False-False-True] 48.2910μs 17.9706μs 55.6465 KOps/s 55.2792 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-False-False-False-False] 41.9180μs 12.2930μs 81.3470 KOps/s 81.3558 KOps/s $\color{#d91a1a}-0.01\%$
test_values[generalized_advantage_estimate-True-True] 11.2768ms 9.4844ms 105.4363 Ops/s 103.6104 Ops/s $\color{#35bf28}+1.76\%$
test_values[vec_generalized_advantage_estimate-True-True] 50.6002ms 35.5355ms 28.1409 Ops/s 30.2285 Ops/s $\textbf{\color{#d91a1a}-6.91\%}$
test_values[td0_return_estimate-False-False] 0.2248ms 0.1710ms 5.8469 KOps/s 5.9521 KOps/s $\color{#d91a1a}-1.77\%$
test_values[td1_return_estimate-False-False] 23.5050ms 23.2180ms 43.0701 Ops/s 41.5452 Ops/s $\color{#35bf28}+3.67\%$
test_values[vec_td1_return_estimate-False-False] 36.4009ms 35.0645ms 28.5189 Ops/s 30.3438 Ops/s $\textbf{\color{#d91a1a}-6.01\%}$
test_values[td_lambda_return_estimate-True-False] 35.1123ms 33.4808ms 29.8679 Ops/s 28.9672 Ops/s $\color{#35bf28}+3.11\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.3233ms 35.1087ms 28.4830 Ops/s 30.2702 Ops/s $\textbf{\color{#d91a1a}-5.90\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.3511ms 8.2310ms 121.4913 Ops/s 117.1416 Ops/s $\color{#35bf28}+3.71\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2875ms 1.9475ms 513.4770 Ops/s 491.5913 Ops/s $\color{#35bf28}+4.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5528ms 0.3511ms 2.8482 KOps/s 2.8132 KOps/s $\color{#35bf28}+1.24\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.2674ms 46.4165ms 21.5441 Ops/s 24.5540 Ops/s $\textbf{\color{#d91a1a}-12.26\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8096ms 3.0249ms 330.5935 Ops/s 330.7871 Ops/s $\color{#d91a1a}-0.06\%$
test_dqn_speed 6.6652ms 1.3547ms 738.1625 Ops/s 745.4581 Ops/s $\color{#d91a1a}-0.98\%$
test_ddpg_speed 3.4356ms 2.6891ms 371.8672 Ops/s 370.9949 Ops/s $\color{#35bf28}+0.24\%$
test_sac_speed 9.5252ms 8.2377ms 121.3930 Ops/s 121.4127 Ops/s $\color{#d91a1a}-0.02\%$
test_redq_speed 14.2698ms 13.0605ms 76.5665 Ops/s 76.0188 Ops/s $\color{#35bf28}+0.72\%$
test_redq_deprec_speed 14.3253ms 13.2361ms 75.5508 Ops/s 76.5081 Ops/s $\color{#d91a1a}-1.25\%$
test_td3_speed 8.6907ms 8.2090ms 121.8172 Ops/s 122.7623 Ops/s $\color{#d91a1a}-0.77\%$
test_cql_speed 38.5273ms 36.9653ms 27.0524 Ops/s 27.0363 Ops/s $\color{#35bf28}+0.06\%$
test_a2c_speed 8.4277ms 7.3765ms 135.5662 Ops/s 134.9798 Ops/s $\color{#35bf28}+0.43\%$
test_ppo_speed 9.2646ms 7.5807ms 131.9145 Ops/s 131.2655 Ops/s $\color{#35bf28}+0.49\%$
test_reinforce_speed 7.2472ms 6.5519ms 152.6279 Ops/s 152.1733 Ops/s $\color{#35bf28}+0.30\%$
test_iql_speed 33.4734ms 32.2944ms 30.9651 Ops/s 30.9651 Ops/s $+0.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3876ms 2.1994ms 454.6730 Ops/s 455.7123 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4843ms 0.4932ms 2.0274 KOps/s 2.0335 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7978ms 0.4695ms 2.1300 KOps/s 2.1429 KOps/s $\color{#d91a1a}-0.60\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5219ms 2.1935ms 455.8895 Ops/s 448.9688 Ops/s $\color{#35bf28}+1.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7909ms 0.4873ms 2.0522 KOps/s 2.0720 KOps/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.6057ms 0.4648ms 2.1514 KOps/s 2.0114 KOps/s $\textbf{\color{#35bf28}+6.96\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.3591ms 1.2223ms 818.1035 Ops/s 810.7058 Ops/s $\color{#35bf28}+0.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 4.6826ms 1.1653ms 858.1802 Ops/s 856.4612 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6461ms 2.3047ms 433.8920 Ops/s 432.1041 Ops/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2400ms 0.6058ms 1.6507 KOps/s 1.6347 KOps/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8885ms 0.5815ms 1.7198 KOps/s 1.7000 KOps/s $\color{#35bf28}+1.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3106ms 2.2043ms 453.6636 Ops/s 455.9705 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1259ms 0.4926ms 2.0300 KOps/s 2.0105 KOps/s $\color{#35bf28}+0.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8054ms 0.4721ms 2.1181 KOps/s 2.1118 KOps/s $\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1911ms 2.2812ms 438.3707 Ops/s 453.1326 Ops/s $\color{#d91a1a}-3.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8370ms 0.4898ms 2.0415 KOps/s 2.0631 KOps/s $\color{#d91a1a}-1.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.2343ms 0.4764ms 2.0989 KOps/s 2.0924 KOps/s $\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.4167ms 2.3274ms 429.6705 Ops/s 407.4629 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8706ms 0.6100ms 1.6393 KOps/s 1.6427 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8469ms 0.6226ms 1.6061 KOps/s 1.6984 KOps/s $\textbf{\color{#d91a1a}-5.43\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1010s 7.5026ms 133.2877 Ops/s 132.7582 Ops/s $\color{#35bf28}+0.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 14.1836ms 11.9908ms 83.3975 Ops/s 81.8697 Ops/s $\color{#35bf28}+1.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0579ms 1.0206ms 979.8177 Ops/s 907.7224 Ops/s $\textbf{\color{#35bf28}+7.94\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 95.9778ms 5.4890ms 182.1833 Ops/s 183.0429 Ops/s $\color{#d91a1a}-0.47\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.7151ms 12.1375ms 82.3894 Ops/s 72.2232 Ops/s $\textbf{\color{#35bf28}+14.08\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.0618ms 1.0181ms 982.2023 Ops/s 867.2477 Ops/s $\textbf{\color{#35bf28}+13.26\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1019s 7.7703ms 128.6953 Ops/s 168.4547 Ops/s $\textbf{\color{#d91a1a}-23.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.6142ms 12.3802ms 80.7738 Ops/s 81.7887 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9248ms 1.4036ms 712.4447 Ops/s 690.8267 Ops/s $\color{#35bf28}+3.13\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 16, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 99.3626ms 99.1240ms 10.0884 Ops/s 9.5199 Ops/s $\textbf{\color{#35bf28}+5.97\%}$
test_sync 88.3838ms 87.4342ms 11.4372 Ops/s 11.5074 Ops/s $\color{#d91a1a}-0.61\%$
test_async 0.1612s 70.2739ms 14.2300 Ops/s 14.3380 Ops/s $\color{#d91a1a}-0.75\%$
test_single_pixels 0.1103s 0.1099s 9.0957 Ops/s 9.0783 Ops/s $\color{#35bf28}+0.19\%$
test_sync_pixels 69.2687ms 66.9625ms 14.9337 Ops/s 14.9815 Ops/s $\color{#d91a1a}-0.32\%$
test_async_pixels 0.1229s 55.3903ms 18.0537 Ops/s 18.0892 Ops/s $\color{#d91a1a}-0.20\%$
test_simple 0.7387s 0.6796s 1.4716 Ops/s 1.4370 Ops/s $\color{#35bf28}+2.41\%$
test_transformed 0.9516s 0.8964s 1.1156 Ops/s 1.1052 Ops/s $\color{#35bf28}+0.94\%$
test_serial 2.1464s 2.0866s 0.4792 Ops/s 0.4671 Ops/s $\color{#35bf28}+2.61\%$
test_parallel 1.8450s 1.7808s 0.5615 Ops/s 0.5658 Ops/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-True-True-True-True] 80.9320μs 32.8690μs 30.4238 KOps/s 30.3915 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-True-True-False] 45.0910μs 19.5147μs 51.2434 KOps/s 51.9541 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-True-True-False-True] 42.1310μs 18.6186μs 53.7098 KOps/s 52.9130 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-True-False-False] 30.6710μs 11.2318μs 89.0328 KOps/s 90.8468 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-False-True-True] 58.3010μs 34.6111μs 28.8925 KOps/s 28.7152 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-False-True-False] 41.6710μs 21.7451μs 45.9874 KOps/s 46.9932 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[True-True-False-False-True] 0.1081ms 20.5043μs 48.7702 KOps/s 48.9433 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-False-False-False] 38.1520μs 13.1033μs 76.3166 KOps/s 77.9793 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[True-False-True-True-True] 60.1510μs 36.9685μs 27.0500 KOps/s 27.0999 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-True-True-False] 54.1510μs 23.7706μs 42.0688 KOps/s 42.9271 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-False-True-False-True] 41.5410μs 20.5623μs 48.6327 KOps/s 48.4567 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-False-False] 33.2510μs 13.1722μs 75.9173 KOps/s 77.8706 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[True-False-False-True-True] 67.1610μs 38.2893μs 26.1170 KOps/s 26.1370 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[True-False-False-True-False] 44.7100μs 25.3597μs 39.4326 KOps/s 39.8143 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-False-False-True] 47.7110μs 22.5186μs 44.4078 KOps/s 46.0662 KOps/s $\color{#d91a1a}-3.60\%$
test_step_mdp_speed[True-False-False-False-False] 32.0910μs 15.0482μs 66.4532 KOps/s 67.9214 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-True-True-True-True] 55.7210μs 37.1004μs 26.9539 KOps/s 26.9955 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-True-True-False] 46.9810μs 23.6818μs 42.2265 KOps/s 42.7245 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-True-True-False-True] 45.1610μs 24.7136μs 40.4636 KOps/s 40.3113 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[False-True-True-False-False] 31.8310μs 14.9791μs 66.7596 KOps/s 68.1544 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[False-True-False-True-True] 62.4810μs 38.7566μs 25.8020 KOps/s 26.0313 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-True-False-True-False] 48.3310μs 25.4758μs 39.2529 KOps/s 39.8642 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[False-True-False-False-True] 45.7810μs 26.7073μs 37.4429 KOps/s 37.5389 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[False-True-False-False-False] 43.0010μs 16.9150μs 59.1193 KOps/s 60.1492 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[False-False-True-True-True] 67.1810μs 40.6718μs 24.5871 KOps/s 24.6393 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-True-True-False] 47.6310μs 27.7009μs 36.0999 KOps/s 36.8408 KOps/s $\color{#d91a1a}-2.01\%$
test_step_mdp_speed[False-False-True-False-True] 44.2510μs 26.6472μs 37.5273 KOps/s 37.5306 KOps/s $-0.01\%$
test_step_mdp_speed[False-False-True-False-False] 30.8300μs 16.8603μs 59.3108 KOps/s 60.6061 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-False-True-True] 61.2410μs 42.1891μs 23.7028 KOps/s 23.7995 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-False-False-True-False] 54.4910μs 29.3761μs 34.0412 KOps/s 34.9513 KOps/s $\color{#d91a1a}-2.60\%$
test_step_mdp_speed[False-False-False-False-True] 50.4010μs 28.1091μs 35.5756 KOps/s 35.7851 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-False-False] 38.5710μs 18.5815μs 53.8169 KOps/s 55.0214 KOps/s $\color{#d91a1a}-2.19\%$
test_values[generalized_advantage_estimate-True-True] 24.8736ms 24.4457ms 40.9070 Ops/s 39.7367 Ops/s $\color{#35bf28}+2.95\%$
test_values[vec_generalized_advantage_estimate-True-True] 83.5574ms 3.2333ms 309.2819 Ops/s 308.9481 Ops/s $\color{#35bf28}+0.11\%$
test_values[td0_return_estimate-False-False] 90.2520μs 63.7675μs 15.6820 KOps/s 15.5783 KOps/s $\color{#35bf28}+0.67\%$
test_values[td1_return_estimate-False-False] 52.8607ms 52.3765ms 19.0925 Ops/s 17.9462 Ops/s $\textbf{\color{#35bf28}+6.39\%}$
test_values[vec_td1_return_estimate-False-False] 1.9817ms 1.7523ms 570.6785 Ops/s 568.0373 Ops/s $\color{#35bf28}+0.46\%$
test_values[td_lambda_return_estimate-True-False] 83.6870ms 83.1797ms 12.0222 Ops/s 11.2809 Ops/s $\textbf{\color{#35bf28}+6.57\%}$
test_values[vec_td_lambda_return_estimate-True-False] 2.0531ms 1.7495ms 571.5885 Ops/s 569.4475 Ops/s $\color{#35bf28}+0.38\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.2758ms 23.1515ms 43.1937 Ops/s 40.6276 Ops/s $\textbf{\color{#35bf28}+6.32\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8865ms 0.6925ms 1.4440 KOps/s 1.4233 KOps/s $\color{#35bf28}+1.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6986ms 0.6429ms 1.5554 KOps/s 1.4975 KOps/s $\color{#35bf28}+3.87\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4994ms 1.4465ms 691.3107 Ops/s 685.7154 Ops/s $\color{#35bf28}+0.82\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9529ms 0.6631ms 1.5081 KOps/s 1.4921 KOps/s $\color{#35bf28}+1.07\%$
test_dqn_speed 1.8620ms 1.4298ms 699.4013 Ops/s 697.1172 Ops/s $\color{#35bf28}+0.33\%$
test_ddpg_speed 2.9445ms 2.7109ms 368.8875 Ops/s 370.3293 Ops/s $\color{#d91a1a}-0.39\%$
test_sac_speed 9.2639ms 8.0311ms 124.5156 Ops/s 124.8478 Ops/s $\color{#d91a1a}-0.27\%$
test_redq_speed 11.0869ms 10.2265ms 97.7850 Ops/s 99.4026 Ops/s $\color{#d91a1a}-1.63\%$
test_redq_deprec_speed 11.3953ms 10.8880ms 91.8438 Ops/s 92.8448 Ops/s $\color{#d91a1a}-1.08\%$
test_td3_speed 8.1605ms 7.9795ms 125.3209 Ops/s 126.2719 Ops/s $\color{#d91a1a}-0.75\%$
test_cql_speed 26.4161ms 25.1944ms 39.6913 Ops/s 40.6884 Ops/s $\color{#d91a1a}-2.45\%$
test_a2c_speed 6.1867ms 5.4918ms 182.0896 Ops/s 184.7108 Ops/s $\color{#d91a1a}-1.42\%$
test_ppo_speed 6.0060ms 5.7896ms 172.7243 Ops/s 173.9769 Ops/s $\color{#d91a1a}-0.72\%$
test_reinforce_speed 4.6926ms 4.4948ms 222.4817 Ops/s 227.5138 Ops/s $\color{#d91a1a}-2.21\%$
test_iql_speed 19.7233ms 19.2430ms 51.9670 Ops/s 52.7239 Ops/s $\color{#d91a1a}-1.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0530ms 2.9317ms 341.0990 Ops/s 348.3301 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2639ms 0.5366ms 1.8636 KOps/s 1.8819 KOps/s $\color{#d91a1a}-0.97\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7321ms 0.5115ms 1.9551 KOps/s 1.9635 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1094ms 2.9377ms 340.4037 Ops/s 346.3589 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7158ms 0.5284ms 1.8924 KOps/s 1.9035 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6995ms 0.5094ms 1.9630 KOps/s 1.9887 KOps/s $\color{#d91a1a}-1.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5443ms 1.4288ms 699.8982 Ops/s 704.3099 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5450ms 1.3518ms 739.7288 Ops/s 739.3278 Ops/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1974ms 3.0910ms 323.5153 Ops/s 333.0112 Ops/s $\color{#d91a1a}-2.85\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2120ms 0.6626ms 1.5091 KOps/s 1.5252 KOps/s $\color{#d91a1a}-1.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8773ms 0.6412ms 1.5597 KOps/s 1.5900 KOps/s $\color{#d91a1a}-1.90\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1189ms 2.9335ms 340.8903 Ops/s 348.1071 Ops/s $\color{#d91a1a}-2.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6552ms 0.5368ms 1.8629 KOps/s 1.8769 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4659ms 0.5172ms 1.9333 KOps/s 1.9347 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1393ms 2.9568ms 338.2019 Ops/s 345.7186 Ops/s $\color{#d91a1a}-2.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6703ms 0.5283ms 1.8929 KOps/s 1.8978 KOps/s $\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6984ms 0.5087ms 1.9659 KOps/s 1.9753 KOps/s $\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1714ms 3.0649ms 326.2770 Ops/s 332.9905 Ops/s $\color{#d91a1a}-2.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7840ms 0.6654ms 1.5028 KOps/s 1.5277 KOps/s $\color{#d91a1a}-1.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6059ms 0.6471ms 1.5455 KOps/s 1.5953 KOps/s $\color{#d91a1a}-3.12\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1351s 7.4426ms 134.3620 Ops/s 105.8501 Ops/s $\textbf{\color{#35bf28}+26.94\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.2408ms 14.9392ms 66.9381 Ops/s 68.8692 Ops/s $\color{#d91a1a}-2.80\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3037ms 1.0869ms 920.0484 Ops/s 895.9240 Ops/s $\color{#35bf28}+2.69\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1172s 7.0806ms 141.2318 Ops/s 142.1682 Ops/s $\color{#d91a1a}-0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.1462ms 14.7356ms 67.8628 Ops/s 68.4600 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1599ms 1.1094ms 901.4150 Ops/s 906.0862 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1177s 7.4531ms 134.1715 Ops/s 103.5256 Ops/s $\textbf{\color{#35bf28}+29.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1278s 17.4937ms 57.1633 Ops/s 66.9247 Ops/s $\textbf{\color{#d91a1a}-14.59\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4415ms 1.4169ms 705.7726 Ops/s 711.1946 Ops/s $\color{#d91a1a}-0.76\%$

@vmoens vmoens changed the title [WIP] Auto-resetting envs [Feature] Auto-resetting envs Apr 16, 2024
vmoens
vmoens commented Apr 16, 2024
View reviewed changes
torchrl/envs/batched_envs.py Outdated Show resolved Hide resolved
vmoens
vmoens commented Apr 16, 2024
View reviewed changes
torchrl/envs/common.py Outdated Show resolved Hide resolved
vmoens
vmoens commented Apr 16, 2024
View reviewed changes
torchrl/envs/libs/gym.py Outdated Show resolved Hide resolved
@vmoens vmoens added the enhancement New feature or request label Apr 16, 2024
@vmoens vmoens merged commit 8570bd3 into main Apr 16, 2024
58 of 67 checks passed
@vmoens vmoens deleted the autorest-envs branch April 16, 2024 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot