Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix run_type_checks #1570

Merged
merged 9 commits into from
Sep 24, 2023
Merged

[BugFix] Fix run_type_checks #1570

merged 9 commits into from
Sep 24, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 23, 2023 •
edited
Loading

Solves a bug when run_type_checks=True where a composite spec is queried with a non-existent get method.
Also adds tests for env._run_type_checks.

Note that this test may fail if the env is substituted with a GymEnv as GymEnv (and others) do the casting of the obs/reward/done internally during the step computation. This trades off security (ie, we can make sure that the dtype is always appropriate) to consistency between gym and torchrl (the dtype in torchrl can differ wrt gym at the expense of loosing information).

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 23, 2023
@vmoens vmoens linked an issue Sep 23, 2023 that may be closed by this pull request
[BUG] check_env_specs() fails when run_type_checks=True #1567
Closed
3 tasks
@vmoens vmoens added the bug Something isn't working label Sep 23, 2023
@github-actions
Copy link

github-actions bot commented Sep 23, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 67.2749ms 66.7840ms 14.9736 Ops/s 14.9604 Ops/s $\color{#35bf28}+0.09\%$
test_sync 0.1086s 38.1290ms 26.2268 Ops/s 26.3662 Ops/s $\color{#d91a1a}-0.53\%$
test_async 55.5401ms 34.4500ms 29.0276 Ops/s 29.3572 Ops/s $\color{#d91a1a}-1.12\%$
test_simple 0.6004s 0.5440s 1.8383 Ops/s 1.8226 Ops/s $\color{#35bf28}+0.86\%$
test_transformed 0.7472s 0.7066s 1.4153 Ops/s 1.3943 Ops/s $\color{#35bf28}+1.51\%$
test_serial 1.5802s 1.5397s 0.6495 Ops/s 0.6409 Ops/s $\color{#35bf28}+1.34\%$
test_parallel 1.4667s 1.4099s 0.7093 Ops/s 0.7164 Ops/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[True-True-True-True-True] 0.2298ms 43.1134μs 23.1947 KOps/s 23.9878 KOps/s $\color{#d91a1a}-3.31\%$
test_step_mdp_speed[True-True-True-True-False] 72.1000μs 24.7606μs 40.3867 KOps/s 41.6530 KOps/s $\color{#d91a1a}-3.04\%$
test_step_mdp_speed[True-True-True-False-True] 0.1745ms 29.6441μs 33.7336 KOps/s 33.9924 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-True-False-False] 48.4000μs 16.8461μs 59.3609 KOps/s 60.5167 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[True-True-False-True-True] 67.0000μs 44.1693μs 22.6401 KOps/s 23.2489 KOps/s $\color{#d91a1a}-2.62\%$
test_step_mdp_speed[True-True-False-True-False] 73.2000μs 26.5001μs 37.7357 KOps/s 38.9089 KOps/s $\color{#d91a1a}-3.02\%$
test_step_mdp_speed[True-True-False-False-True] 70.3000μs 31.9438μs 31.3050 KOps/s 31.8791 KOps/s $\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-True-False-False-False] 42.3000μs 18.7167μs 53.4281 KOps/s 54.3008 KOps/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[True-False-True-True-True] 83.7000μs 46.7662μs 21.3830 KOps/s 21.9466 KOps/s $\color{#d91a1a}-2.57\%$
test_step_mdp_speed[True-False-True-True-False] 48.2000μs 28.4861μs 35.1048 KOps/s 35.9023 KOps/s $\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-False-True-False-True] 60.1000μs 31.7989μs 31.4476 KOps/s 31.6760 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-False-True-False-False] 68.6000μs 18.8021μs 53.1857 KOps/s 54.6302 KOps/s $\color{#d91a1a}-2.64\%$
test_step_mdp_speed[True-False-False-True-True] 73.5000μs 47.9299μs 20.8638 KOps/s 21.1707 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-False-True-False] 52.7000μs 30.0347μs 33.2948 KOps/s 34.2075 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[True-False-False-False-True] 85.5000μs 33.5102μs 29.8417 KOps/s 30.3922 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[True-False-False-False-False] 50.1000μs 20.6989μs 48.3117 KOps/s 49.7867 KOps/s $\color{#d91a1a}-2.96\%$
test_step_mdp_speed[False-True-True-True-True] 78.3000μs 46.6013μs 21.4586 KOps/s 22.0814 KOps/s $\color{#d91a1a}-2.82\%$
test_step_mdp_speed[False-True-True-True-False] 66.4000μs 28.8602μs 34.6497 KOps/s 36.3033 KOps/s $\color{#d91a1a}-4.55\%$
test_step_mdp_speed[False-True-True-False-True] 0.1454ms 34.8150μs 28.7233 KOps/s 29.2793 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-True-True-False-False] 59.2000μs 20.7393μs 48.2176 KOps/s 49.9084 KOps/s $\color{#d91a1a}-3.39\%$
test_step_mdp_speed[False-True-False-True-True] 0.1234ms 47.9860μs 20.8394 KOps/s 21.1761 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-True-False-True-False] 86.6000μs 30.8098μs 32.4572 KOps/s 33.9284 KOps/s $\color{#d91a1a}-4.34\%$
test_step_mdp_speed[False-True-False-False-True] 81.1000μs 36.4673μs 27.4219 KOps/s 28.1031 KOps/s $\color{#d91a1a}-2.42\%$
test_step_mdp_speed[False-True-False-False-False] 76.5000μs 22.2032μs 45.0386 KOps/s 45.9031 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[False-False-True-True-True] 73.2000μs 49.7970μs 20.0815 KOps/s 20.5252 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-False-True-True-False] 73.9000μs 32.4208μs 30.8444 KOps/s 32.0194 KOps/s $\color{#d91a1a}-3.67\%$
test_step_mdp_speed[False-False-True-False-True] 65.0000μs 36.4980μs 27.3988 KOps/s 28.2218 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[False-False-True-False-False] 40.9000μs 22.4416μs 44.5600 KOps/s 46.4444 KOps/s $\color{#d91a1a}-4.06\%$
test_step_mdp_speed[False-False-False-True-True] 0.1604ms 51.2034μs 19.5299 KOps/s 19.9421 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-False-False-True-False] 61.0000μs 34.2201μs 29.2226 KOps/s 30.7985 KOps/s $\textbf{\color{#d91a1a}-5.12\%}$
test_step_mdp_speed[False-False-False-False-True] 88.0000μs 37.9736μs 26.3341 KOps/s 27.2273 KOps/s $\color{#d91a1a}-3.28\%$
test_step_mdp_speed[False-False-False-False-False] 46.4000μs 23.8528μs 41.9237 KOps/s 43.3872 KOps/s $\color{#d91a1a}-3.37\%$
test_values[generalized_advantage_estimate-True-True] 15.6621ms 14.2212ms 70.3177 Ops/s 68.8977 Ops/s $\color{#35bf28}+2.06\%$
test_values[vec_generalized_advantage_estimate-True-True] 64.7149ms 52.8529ms 18.9204 Ops/s 18.6756 Ops/s $\color{#35bf28}+1.31\%$
test_values[td0_return_estimate-False-False] 0.5455ms 0.4180ms 2.3921 KOps/s 2.1018 KOps/s $\textbf{\color{#35bf28}+13.81\%}$
test_values[td1_return_estimate-False-False] 13.9275ms 13.6147ms 73.4500 Ops/s 71.4698 Ops/s $\color{#35bf28}+2.77\%$
test_values[vec_td1_return_estimate-False-False] 63.9003ms 52.9094ms 18.9002 Ops/s 18.4934 Ops/s $\color{#35bf28}+2.20\%$
test_values[td_lambda_return_estimate-True-False] 32.8872ms 32.3621ms 30.9004 Ops/s 30.2452 Ops/s $\color{#35bf28}+2.17\%$
test_values[vec_td_lambda_return_estimate-True-False] 63.0657ms 52.2010ms 19.1567 Ops/s 18.9094 Ops/s $\color{#35bf28}+1.31\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.2852ms 12.0887ms 82.7221 Ops/s 79.9039 Ops/s $\color{#35bf28}+3.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 5.0242ms 2.9972ms 333.6408 Ops/s 340.1599 Ops/s $\color{#d91a1a}-1.92\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6812ms 0.5033ms 1.9869 KOps/s 1.9625 KOps/s $\color{#35bf28}+1.24\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 63.1798ms 53.4115ms 18.7226 Ops/s 18.9364 Ops/s $\color{#d91a1a}-1.13\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 8.6748ms 4.1948ms 238.3928 Ops/s 238.1013 Ops/s $\color{#35bf28}+0.12\%$
test_dqn_speed 6.3072ms 2.3463ms 426.2107 Ops/s 422.2699 Ops/s $\color{#35bf28}+0.93\%$
test_ddpg_speed 6.2492ms 3.9685ms 251.9862 Ops/s 249.0357 Ops/s $\color{#35bf28}+1.18\%$
test_sac_speed 16.1785ms 11.3337ms 88.2325 Ops/s 88.8980 Ops/s $\color{#d91a1a}-0.75\%$
test_redq_speed 23.6399ms 18.0439ms 55.4205 Ops/s 55.1076 Ops/s $\color{#35bf28}+0.57\%$
test_redq_deprec_speed 24.7756ms 16.6753ms 59.9689 Ops/s 57.8635 Ops/s $\color{#35bf28}+3.64\%$
test_td3_speed 12.1186ms 11.4336ms 87.4616 Ops/s 86.0549 Ops/s $\color{#35bf28}+1.63\%$
test_cql_speed 44.4684ms 36.5384ms 27.3684 Ops/s 25.8604 Ops/s $\textbf{\color{#35bf28}+5.83\%}$
test_a2c_speed 12.6755ms 6.9748ms 143.3732 Ops/s 142.9122 Ops/s $\color{#35bf28}+0.32\%$
test_ppo_speed 16.8623ms 8.0942ms 123.5452 Ops/s 122.2650 Ops/s $\color{#35bf28}+1.05\%$
test_reinforce_speed 11.7564ms 5.8493ms 170.9614 Ops/s 173.6205 Ops/s $\color{#d91a1a}-1.53\%$
test_iql_speed 36.4015ms 28.8624ms 34.6471 Ops/s 36.3661 Ops/s $\color{#d91a1a}-4.73\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1555ms 2.5527ms 391.7461 Ops/s 383.2384 Ops/s $\color{#35bf28}+2.22\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.1392ms 2.7613ms 362.1467 Ops/s 374.0793 Ops/s $\color{#d91a1a}-3.19\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.3527ms 2.6550ms 376.6449 Ops/s 369.4388 Ops/s $\color{#35bf28}+1.95\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3059ms 2.5251ms 396.0303 Ops/s 387.7076 Ops/s $\color{#35bf28}+2.15\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.8799ms 2.6983ms 370.6081 Ops/s 368.1424 Ops/s $\color{#35bf28}+0.67\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.0024ms 2.6594ms 376.0310 Ops/s 368.9799 Ops/s $\color{#35bf28}+1.91\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8313ms 2.5322ms 394.9163 Ops/s 388.6969 Ops/s $\color{#35bf28}+1.60\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.3075ms 2.6645ms 375.3062 Ops/s 366.4292 Ops/s $\color{#35bf28}+2.42\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.2295ms 2.6928ms 371.3575 Ops/s 368.4983 Ops/s $\color{#35bf28}+0.78\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2635ms 2.5101ms 398.3891 Ops/s 386.4732 Ops/s $\color{#35bf28}+3.08\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.5657ms 2.6712ms 374.3573 Ops/s 359.8060 Ops/s $\color{#35bf28}+4.04\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.9305ms 2.6140ms 382.5514 Ops/s 372.0970 Ops/s $\color{#35bf28}+2.81\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1484ms 2.5534ms 391.6374 Ops/s 391.7409 Ops/s $\color{#d91a1a}-0.03\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.9455ms 2.6542ms 376.7586 Ops/s 367.7610 Ops/s $\color{#35bf28}+2.45\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3586ms 2.6833ms 372.6745 Ops/s 372.2679 Ops/s $\color{#35bf28}+0.11\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2790ms 2.4935ms 401.0409 Ops/s 379.7005 Ops/s $\textbf{\color{#35bf28}+5.62\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.5731ms 2.6848ms 372.4632 Ops/s 364.1451 Ops/s $\color{#35bf28}+2.28\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.4688ms 2.7336ms 365.8156 Ops/s 361.9664 Ops/s $\color{#35bf28}+1.06\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2157s 26.6987ms 37.4550 Ops/s 38.3507 Ops/s $\color{#d91a1a}-2.34\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1190s 26.3362ms 37.9706 Ops/s 40.5162 Ops/s $\textbf{\color{#d91a1a}-6.28\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1163s 24.4313ms 40.9312 Ops/s 37.6372 Ops/s $\textbf{\color{#35bf28}+8.75\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1137s 24.2348ms 41.2630 Ops/s 38.2513 Ops/s $\textbf{\color{#35bf28}+7.87\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1131s 22.2687ms 44.9061 Ops/s 41.2998 Ops/s $\textbf{\color{#35bf28}+8.73\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1133s 25.8747ms 38.6478 Ops/s 37.7156 Ops/s $\color{#35bf28}+2.47\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1105s 23.9444ms 41.7635 Ops/s 40.5049 Ops/s $\color{#35bf28}+3.11\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1125s 25.7636ms 38.8144 Ops/s 37.6937 Ops/s $\color{#35bf28}+2.97\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1132s 24.0572ms 41.5676 Ops/s 40.2308 Ops/s $\color{#35bf28}+3.32\%$

@vmoens vmoens marked this pull request as ready for review September 24, 2023 05:55
matteobettini
matteobettini approved these changes Sep 24, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vmoens vmoens merged commit 8503378 into main Sep 24, 2023
vmoens added a commit to hyerra/rl that referenced this pull request Oct 10, 2023
@vmoens
[BugFix] Fix run_type_checks (pytorch#1570)
f5b886e
@vmoens vmoens deleted the fix_1567 branch February 27, 2024 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] check_env_specs() fails when run_type_checks=True
3 participants
@vmoens @matteobettini @facebook-github-bot