Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] GenDGRL #1773

Merged
merged 14 commits into from
Jan 16, 2024
Merged

[Feature] GenDGRL #1773

merged 14 commits into from
Jan 16, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 6, 2024 •
edited
Loading

Gen-DGRL dataset.

To comply with other datasets, we use a dataset_id that contains the procgen env as well as the category, eg bigfish-1M_E.

Tests just process the first 100 samples of a bunch of datasets.

Cc @ishitamed19 @SharathRaparthy @braj29

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 6, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1773

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 5 Unrelated Failures

As of commit 600d9f4 with merge base 3d7e49c (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 6, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 64.3982ms 63.0923ms 15.8498 Ops/s 16.0397 Ops/s $\color{#d91a1a}-1.18\%$
test_sync 35.3886ms 33.6948ms 29.6782 Ops/s 29.0419 Ops/s $\color{#35bf28}+2.19\%$
test_async 65.7607ms 32.9627ms 30.3373 Ops/s 29.4935 Ops/s $\color{#35bf28}+2.86\%$
test_simple 0.4921s 0.4350s 2.2987 Ops/s 2.2811 Ops/s $\color{#35bf28}+0.77\%$
test_transformed 0.6589s 0.6053s 1.6520 Ops/s 1.6521 Ops/s $-0.01\%$
test_serial 1.4084s 1.3507s 0.7403 Ops/s 0.7405 Ops/s $\color{#d91a1a}-0.03\%$
test_parallel 1.3801s 1.3270s 0.7536 Ops/s 0.7634 Ops/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-True-True-True-True] 0.1532ms 21.8158μs 45.8383 KOps/s 46.5245 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[True-True-True-True-False] 46.4660μs 13.2035μs 75.7378 KOps/s 76.1827 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-True-False-True] 41.6570μs 12.8687μs 77.7081 KOps/s 78.3621 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-True-False-False] 36.9990μs 7.7615μs 128.8412 KOps/s 129.2130 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-False-True-True] 75.9810μs 23.0996μs 43.2909 KOps/s 43.5881 KOps/s $\color{#d91a1a}-0.68\%$
test_step_mdp_speed[True-True-False-True-False] 35.1560μs 14.5479μs 68.7387 KOps/s 69.1161 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-True-False-False-True] 80.2900μs 14.2900μs 69.9791 KOps/s 71.6531 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[True-True-False-False-False] 28.0120μs 9.0386μs 110.6366 KOps/s 111.6250 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[True-False-True-True-True] 49.6320μs 24.9181μs 40.1315 KOps/s 41.6168 KOps/s $\color{#d91a1a}-3.57\%$
test_step_mdp_speed[True-False-True-True-False] 57.2570μs 16.2987μs 61.3548 KOps/s 63.7841 KOps/s $\color{#d91a1a}-3.81\%$
test_step_mdp_speed[True-False-True-False-True] 90.6840μs 13.7850μs 72.5426 KOps/s 71.8798 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-True-False-False] 37.1390μs 8.9024μs 112.3287 KOps/s 111.3090 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-False-True-True] 0.1070ms 25.4799μs 39.2467 KOps/s 39.2693 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-False-False-True-False] 49.6330μs 17.1575μs 58.2836 KOps/s 59.1709 KOps/s $\color{#d91a1a}-1.50\%$
test_step_mdp_speed[True-False-False-False-True] 39.5040μs 15.0128μs 66.6100 KOps/s 65.2109 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-False-False-False-False] 34.0840μs 10.1896μs 98.1397 KOps/s 98.4101 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-True-True-True-True] 58.4390μs 24.4836μs 40.8437 KOps/s 41.1862 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-True-True-True-False] 46.9270μs 16.0041μs 62.4838 KOps/s 62.6072 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-True-True-False-True] 50.1140μs 16.2768μs 61.4371 KOps/s 61.5146 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-True-False-False] 47.6090μs 10.4372μs 95.8115 KOps/s 97.0410 KOps/s $\color{#d91a1a}-1.27\%$
test_step_mdp_speed[False-True-False-True-True] 62.4060μs 25.6835μs 38.9355 KOps/s 39.4266 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-True-False-True-False] 44.5130μs 17.2890μs 57.8403 KOps/s 58.9975 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[False-True-False-False-True] 45.0040μs 17.3972μs 57.4805 KOps/s 57.7117 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-False-False] 34.0030μs 11.5682μs 86.4440 KOps/s 88.5312 KOps/s $\color{#d91a1a}-2.36\%$
test_step_mdp_speed[False-False-True-True-True] 65.3920μs 27.3126μs 36.6131 KOps/s 37.2274 KOps/s $\color{#d91a1a}-1.65\%$
test_step_mdp_speed[False-False-True-True-False] 48.4710μs 18.6610μs 53.5876 KOps/s 54.7619 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-False-True-False-True] 46.1060μs 17.6075μs 56.7938 KOps/s 57.2180 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[False-False-True-False-False] 48.2390μs 11.5028μs 86.9350 KOps/s 87.2600 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-False-False-True-True] 56.6660μs 28.0719μs 35.6228 KOps/s 36.3380 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-False-False-True-False] 52.5880μs 19.7013μs 50.7581 KOps/s 51.7247 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-False-False-False-True] 54.2510μs 18.6334μs 53.6670 KOps/s 54.8700 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[False-False-False-False-False] 34.5950μs 12.7321μs 78.5417 KOps/s 79.7917 KOps/s $\color{#d91a1a}-1.57\%$
test_values[generalized_advantage_estimate-True-True] 12.1658ms 11.8957ms 84.0641 Ops/s 83.1660 Ops/s $\color{#35bf28}+1.08\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.0678ms 29.0625ms 34.4086 Ops/s 37.8551 Ops/s $\textbf{\color{#d91a1a}-9.10\%}$
test_values[td0_return_estimate-False-False] 0.2445ms 0.1741ms 5.7442 KOps/s 5.6072 KOps/s $\color{#35bf28}+2.44\%$
test_values[td1_return_estimate-False-False] 27.9578ms 24.9391ms 40.0978 Ops/s 39.4035 Ops/s $\color{#35bf28}+1.76\%$
test_values[vec_td1_return_estimate-False-False] 36.4593ms 28.4175ms 35.1896 Ops/s 36.6217 Ops/s $\color{#d91a1a}-3.91\%$
test_values[td_lambda_return_estimate-True-False] 35.2956ms 34.6059ms 28.8968 Ops/s 28.1692 Ops/s $\color{#35bf28}+2.58\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.9186ms 28.3286ms 35.3001 Ops/s 36.8704 Ops/s $\color{#d91a1a}-4.26\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.0838ms 8.0035ms 124.9453 Ops/s 122.0721 Ops/s $\color{#35bf28}+2.35\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 13.5646ms 1.9049ms 524.9675 Ops/s 511.7044 Ops/s $\color{#35bf28}+2.59\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5402ms 0.4211ms 2.3746 KOps/s 2.2588 KOps/s $\textbf{\color{#35bf28}+5.12\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.2122ms 40.7299ms 24.5520 Ops/s 27.7766 Ops/s $\textbf{\color{#d91a1a}-11.61\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 10.3593ms 2.6380ms 379.0755 Ops/s 382.8700 Ops/s $\color{#d91a1a}-0.99\%$
test_dqn_speed 18.4502ms 7.7984ms 128.2313 Ops/s 123.0566 Ops/s $\color{#35bf28}+4.21\%$
test_ddpg_speed 20.0090ms 14.7587ms 67.7565 Ops/s 69.1788 Ops/s $\color{#d91a1a}-2.06\%$
test_sac_speed 32.0065ms 29.8535ms 33.4969 Ops/s 34.0225 Ops/s $\color{#d91a1a}-1.54\%$
test_redq_speed 48.7728ms 46.9291ms 21.3087 Ops/s 21.2693 Ops/s $\color{#35bf28}+0.19\%$
test_redq_deprec_speed 35.4912ms 26.3598ms 37.9365 Ops/s 37.5708 Ops/s $\color{#35bf28}+0.97\%$
test_td3_speed 30.6523ms 20.6936ms 48.3241 Ops/s 49.3108 Ops/s $\color{#d91a1a}-2.00\%$
test_cql_speed 97.2442ms 89.9892ms 11.1124 Ops/s 11.1119 Ops/s $+0.00\%$
test_a2c_speed 37.5564ms 27.6301ms 36.1925 Ops/s 36.6203 Ops/s $\color{#d91a1a}-1.17\%$
test_ppo_speed 30.6967ms 27.9675ms 35.7558 Ops/s 34.0373 Ops/s $\textbf{\color{#35bf28}+5.05\%}$
test_reinforce_speed 34.9278ms 26.7033ms 37.4485 Ops/s 39.1965 Ops/s $\color{#d91a1a}-4.46\%$
test_iql_speed 78.3364ms 66.2556ms 15.0931 Ops/s 15.5585 Ops/s $\color{#d91a1a}-2.99\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 1.7592ms 1.4267ms 700.9001 Ops/s 711.7238 Ops/s $\color{#d91a1a}-1.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 9.5276ms 0.5234ms 1.9107 KOps/s 1.9503 KOps/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 8.7712ms 0.5083ms 1.9672 KOps/s 1.9627 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.1980ms 1.4053ms 711.5708 Ops/s 733.4806 Ops/s $\color{#d91a1a}-2.99\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 12.9744ms 0.5214ms 1.9178 KOps/s 1.9181 KOps/s $\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6234ms 0.4987ms 2.0054 KOps/s 1.6648 KOps/s $\textbf{\color{#35bf28}+20.45\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3348ms 1.5941ms 627.2979 Ops/s 620.1941 Ops/s $\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 8.9762ms 0.6678ms 1.4975 KOps/s 1.5082 KOps/s $\color{#d91a1a}-0.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.9505ms 0.6433ms 1.5544 KOps/s 1.5029 KOps/s $\color{#35bf28}+3.43\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7932ms 1.4917ms 670.3646 Ops/s 707.7431 Ops/s $\textbf{\color{#d91a1a}-5.28\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5010ms 0.5241ms 1.9079 KOps/s 1.5063 KOps/s $\textbf{\color{#35bf28}+26.66\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 9.1534ms 0.5177ms 1.9316 KOps/s 1.9859 KOps/s $\color{#d91a1a}-2.74\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.1591ms 1.4294ms 699.5753 Ops/s 722.3218 Ops/s $\color{#d91a1a}-3.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6393ms 0.5176ms 1.9321 KOps/s 1.8964 KOps/s $\color{#35bf28}+1.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.7623ms 0.5283ms 1.8929 KOps/s 2.0000 KOps/s $\textbf{\color{#d91a1a}-5.35\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5705ms 1.6473ms 607.0645 Ops/s 617.2323 Ops/s $\color{#d91a1a}-1.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0094ms 0.6507ms 1.5369 KOps/s 1.5042 KOps/s $\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.3958ms 0.6498ms 1.5389 KOps/s 1.5597 KOps/s $\color{#d91a1a}-1.34\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1355s 18.4463ms 54.2115 Ops/s 56.3885 Ops/s $\color{#d91a1a}-3.86\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.8285ms 13.6982ms 73.0022 Ops/s 73.2047 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 13.3022ms 3.4930ms 286.2863 Ops/s 303.4100 Ops/s $\textbf{\color{#d91a1a}-5.64\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1608s 19.7537ms 50.6234 Ops/s 54.8324 Ops/s $\textbf{\color{#d91a1a}-7.68\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.5036ms 13.8228ms 72.3443 Ops/s 71.8428 Ops/s $\color{#35bf28}+0.70\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.7247ms 3.3734ms 296.4401 Ops/s 304.1896 Ops/s $\color{#d91a1a}-2.55\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1444s 18.6412ms 53.6445 Ops/s 52.3990 Ops/s $\color{#35bf28}+2.38\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.9998ms 13.9688ms 71.5880 Ops/s 71.6622 Ops/s $\color{#d91a1a}-0.10\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.8525ms 3.4985ms 285.8406 Ops/s 284.1081 Ops/s $\color{#35bf28}+0.61\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 6, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1180s 0.1163s 8.5983 Ops/s 8.6566 Ops/s $\color{#d91a1a}-0.67\%$
test_sync 0.1707s 0.1025s 9.7579 Ops/s 9.7083 Ops/s $\color{#35bf28}+0.51\%$
test_async 0.1828s 92.5202ms 10.8084 Ops/s 10.9077 Ops/s $\color{#d91a1a}-0.91\%$
test_single_pixels 0.1272s 0.1270s 7.8736 Ops/s 7.1839 Ops/s $\textbf{\color{#35bf28}+9.60\%}$
test_sync_pixels 82.4421ms 76.4929ms 13.0731 Ops/s 12.9769 Ops/s $\color{#35bf28}+0.74\%$
test_async_pixels 0.1384s 71.5614ms 13.9740 Ops/s 13.7051 Ops/s $\color{#35bf28}+1.96\%$
test_simple 0.9357s 0.8711s 1.1480 Ops/s 1.2338 Ops/s $\textbf{\color{#d91a1a}-6.96\%}$
test_transformed 1.1706s 1.1132s 0.8983 Ops/s 0.9378 Ops/s $\color{#d91a1a}-4.20\%$
test_serial 2.5366s 2.4700s 0.4049 Ops/s 0.4327 Ops/s $\textbf{\color{#d91a1a}-6.43\%}$
test_parallel 2.1129s 1.9607s 0.5100 Ops/s 0.5179 Ops/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-True-True-True-True] 97.4410μs 33.1529μs 30.1633 KOps/s 29.5265 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-True-False] 43.2500μs 19.4519μs 51.4090 KOps/s 49.9520 KOps/s $\color{#35bf28}+2.92\%$
test_step_mdp_speed[True-True-True-False-True] 44.8210μs 18.7512μs 53.3298 KOps/s 52.1431 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-True-False-False] 42.0410μs 11.2114μs 89.1947 KOps/s 88.4036 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-True-False-True-True] 65.2500μs 35.0127μs 28.5611 KOps/s 28.6595 KOps/s $\color{#d91a1a}-0.34\%$
test_step_mdp_speed[True-True-False-True-False] 47.3200μs 21.4084μs 46.7107 KOps/s 46.2183 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-False-False-True] 47.1500μs 20.8689μs 47.9182 KOps/s 47.9276 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-True-False-False-False] 34.0600μs 13.0836μs 76.4316 KOps/s 75.6348 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-False-True-True-True] 97.0310μs 36.6878μs 27.2570 KOps/s 26.8641 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-False-True-True-False] 46.9700μs 23.2711μs 42.9718 KOps/s 42.1830 KOps/s $\color{#35bf28}+1.87\%$
test_step_mdp_speed[True-False-True-False-True] 46.2110μs 20.8012μs 48.0742 KOps/s 48.0986 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-True-False-False] 35.1700μs 13.0557μs 76.5952 KOps/s 75.1271 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-False-False-True-True] 70.4710μs 38.5962μs 25.9093 KOps/s 25.9411 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[True-False-False-True-False] 49.3410μs 24.7565μs 40.3935 KOps/s 39.6422 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[True-False-False-False-True] 67.4500μs 22.7117μs 44.0302 KOps/s 44.3505 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-False-False-False-False] 36.8410μs 14.7274μs 67.9004 KOps/s 66.2828 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-True-True-True-True] 66.6010μs 37.1658μs 26.9064 KOps/s 26.8382 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-True-True-True-False] 48.4810μs 23.2063μs 43.0918 KOps/s 42.5374 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-True-True-False-True] 50.7210μs 25.1729μs 39.7253 KOps/s 39.9804 KOps/s $\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-True-False-False] 69.0000μs 14.9500μs 66.8896 KOps/s 65.1934 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[False-True-False-True-True] 63.6800μs 38.5033μs 25.9718 KOps/s 25.7398 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-True-False-True-False] 45.9900μs 25.2147μs 39.6594 KOps/s 39.2212 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[False-True-False-False-True] 53.0900μs 27.1366μs 36.8506 KOps/s 36.9484 KOps/s $\color{#d91a1a}-0.26\%$
test_step_mdp_speed[False-True-False-False-False] 41.5700μs 16.7246μs 59.7922 KOps/s 57.7951 KOps/s $\color{#35bf28}+3.46\%$
test_step_mdp_speed[False-False-True-True-True] 67.4310μs 40.3750μs 24.7678 KOps/s 24.6099 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-True-True-False] 79.3810μs 27.0249μs 37.0029 KOps/s 36.6472 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-True-False-True] 51.7610μs 26.9979μs 37.0400 KOps/s 37.7594 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-True-False-False] 58.4700μs 16.8548μs 59.3302 KOps/s 58.9870 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-False-True-True] 71.9600μs 42.3049μs 23.6379 KOps/s 23.8884 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[False-False-False-True-False] 55.6600μs 29.0300μs 34.4471 KOps/s 34.6849 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-False-False-True] 87.4310μs 28.2246μs 35.4301 KOps/s 35.8385 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[False-False-False-False-False] 42.1800μs 18.4862μs 54.0943 KOps/s 53.6123 KOps/s $\color{#35bf28}+0.90\%$
test_values[generalized_advantage_estimate-True-True] 23.8599ms 23.3013ms 42.9161 Ops/s 41.1320 Ops/s $\color{#35bf28}+4.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 81.9624ms 3.1875ms 313.7295 Ops/s 314.4460 Ops/s $\color{#d91a1a}-0.23\%$
test_values[td0_return_estimate-False-False] 96.5310μs 60.5785μs 16.5075 KOps/s 16.9019 KOps/s $\color{#d91a1a}-2.33\%$
test_values[td1_return_estimate-False-False] 53.7033ms 53.2164ms 18.7912 Ops/s 19.4191 Ops/s $\color{#d91a1a}-3.23\%$
test_values[vec_td1_return_estimate-False-False] 2.0628ms 1.7618ms 567.5973 Ops/s 568.0024 Ops/s $\color{#d91a1a}-0.07\%$
test_values[td_lambda_return_estimate-True-False] 85.6953ms 83.6381ms 11.9563 Ops/s 11.9349 Ops/s $\color{#35bf28}+0.18\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0686ms 1.7372ms 575.6399 Ops/s 570.1089 Ops/s $\color{#35bf28}+0.97\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.0952ms 21.9281ms 45.6037 Ops/s 43.2364 Ops/s $\textbf{\color{#35bf28}+5.48\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8389ms 0.6728ms 1.4863 KOps/s 1.5036 KOps/s $\color{#d91a1a}-1.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6808ms 0.6307ms 1.5856 KOps/s 1.6103 KOps/s $\color{#d91a1a}-1.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4668ms 1.4344ms 697.1544 Ops/s 701.3668 Ops/s $\color{#d91a1a}-0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9130ms 0.6510ms 1.5362 KOps/s 1.5540 KOps/s $\color{#d91a1a}-1.15\%$
test_dqn_speed 13.8737ms 7.5524ms 132.4089 Ops/s 136.8126 Ops/s $\color{#d91a1a}-3.22\%$
test_ddpg_speed 15.6787ms 14.7932ms 67.5986 Ops/s 69.5704 Ops/s $\color{#d91a1a}-2.83\%$
test_sac_speed 30.4610ms 29.7790ms 33.5807 Ops/s 34.2468 Ops/s $\color{#d91a1a}-1.95\%$
test_redq_speed 50.3164ms 49.3132ms 20.2786 Ops/s 20.4503 Ops/s $\color{#d91a1a}-0.84\%$
test_redq_deprec_speed 25.2238ms 24.3805ms 41.0164 Ops/s 41.6525 Ops/s $\color{#d91a1a}-1.53\%$
test_td3_speed 29.4037ms 20.0989ms 49.7541 Ops/s 51.1073 Ops/s $\color{#d91a1a}-2.65\%$
test_cql_speed 85.5575ms 84.8492ms 11.7856 Ops/s 12.1103 Ops/s $\color{#d91a1a}-2.68\%$
test_a2c_speed 28.1659ms 27.3829ms 36.5191 Ops/s 37.8689 Ops/s $\color{#d91a1a}-3.56\%$
test_ppo_speed 28.9146ms 27.6972ms 36.1048 Ops/s 37.4419 Ops/s $\color{#d91a1a}-3.57\%$
test_reinforce_speed 27.7108ms 26.4726ms 37.7749 Ops/s 38.9366 Ops/s $\color{#d91a1a}-2.98\%$
test_iql_speed 59.3399ms 58.3266ms 17.1448 Ops/s 17.5683 Ops/s $\color{#d91a1a}-2.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3731ms 1.8158ms 550.7268 Ops/s 552.3598 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0401ms 0.8361ms 1.1960 KOps/s 1.1970 KOps/s $\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.0515ms 0.8249ms 1.2123 KOps/s 1.2128 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5011ms 1.7979ms 556.2092 Ops/s 558.3688 Ops/s $\color{#d91a1a}-0.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9384ms 0.8257ms 1.2111 KOps/s 1.2140 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9366ms 0.8155ms 1.2263 KOps/s 1.2280 KOps/s $\color{#d91a1a}-0.14\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8458ms 2.0959ms 477.1268 Ops/s 481.2379 Ops/s $\color{#d91a1a}-0.85\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2505ms 0.9526ms 1.0497 KOps/s 1.0497 KOps/s $-0.00\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1476ms 0.9464ms 1.0567 KOps/s 1.0609 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.5225ms 1.8250ms 547.9576 Ops/s 549.5763 Ops/s $\color{#d91a1a}-0.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0553ms 0.8366ms 1.1953 KOps/s 1.1982 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.0315ms 0.8262ms 1.2103 KOps/s 1.2132 KOps/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4720ms 1.7979ms 556.2050 Ops/s 561.2843 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0579ms 0.8272ms 1.2089 KOps/s 1.2129 KOps/s $\color{#d91a1a}-0.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1109s 0.9664ms 1.0348 KOps/s 1.2268 KOps/s $\textbf{\color{#d91a1a}-15.65\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9427ms 2.0955ms 477.2028 Ops/s 479.9653 Ops/s $\color{#d91a1a}-0.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0744ms 0.9553ms 1.0468 KOps/s 1.0501 KOps/s $\color{#d91a1a}-0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0551ms 0.9456ms 1.0575 KOps/s 1.0572 KOps/s $\color{#35bf28}+0.03\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1208s 17.9108ms 55.8323 Ops/s 53.5208 Ops/s $\color{#35bf28}+4.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.2452ms 13.8202ms 72.3576 Ops/s 72.4416 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.7889ms 3.3547ms 298.0860 Ops/s 303.8702 Ops/s $\color{#d91a1a}-1.90\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1223s 18.0329ms 55.4541 Ops/s 55.0792 Ops/s $\color{#35bf28}+0.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.9697ms 13.9133ms 71.8738 Ops/s 72.2867 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.8169ms 3.3520ms 298.3271 Ops/s 304.6752 Ops/s $\color{#d91a1a}-2.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1217s 18.2166ms 54.8950 Ops/s 55.0148 Ops/s $\color{#d91a1a}-0.22\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.4705ms 13.9412ms 71.7298 Ops/s 71.2527 Ops/s $\color{#35bf28}+0.67\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.9465ms 3.5276ms 283.4783 Ops/s 287.2603 Ops/s $\color{#d91a1a}-1.32\%$

@vmoens vmoens added the Data Data-related PR, will launch data-related jobs label Jan 7, 2024
@vmoens vmoens marked this pull request as ready for review January 7, 2024 07:10
YhgzXxfz
YhgzXxfz reviewed Jan 16, 2024
View reviewed changes
torchrl/data/datasets/gen_dgrl.py

@staticmethod
def _convert_category_name(category_name: str) -> str:
if category_name == "1M_E":
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good for now. In the future if we have more datasets, we would want to modify the category_name to reflect the parameters, like "1M_200_E", "100k_40_S", etc, so that we can easily map them separately and combine together for paths:

  • "1M" -> "1M", "100k" -> "100k", ...
  • "200" -> "level_200", "40" -> "level_40", ...
  • "E" -> "expert", "S" -> "suboptimal"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! If you add more of these just ping us and we'll do the fixes

@vmoens vmoens merged commit 1460b16 into main Jan 16, 2024
58 of 63 checks passed
@vmoens vmoens deleted the gendgrl branch January 16, 2024 11:09
@vmoens vmoens mentioned this pull request Mar 5, 2024
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@YhgzXxfz YhgzXxfz YhgzXxfz left review comments

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Data Data-related PR, will launch data-related jobs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @YhgzXxfz @facebook-github-bot