Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] ClipTransform + rename min/maximum -> low/high #1500

Merged
merged 4 commits into from
Sep 7, 2023
Merged

[Feature] ClipTransform + rename min/maximum -> low/high #1500

merged 4 commits into from
Sep 7, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 6, 2023 •
edited
Loading

Clips observations, actions and rewrards.

  • tests to be implemented
  • docstrings

cc @albertbou92

We also rename maximum and minimum in the specs to low and high and raise deprecation warnings when necessary.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 6, 2023
@vmoens vmoens added the enhancement New feature or request label Sep 6, 2023
@github-actions
Copy link

github-actions bot commented Sep 6, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}27$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1574s 0.1567s 6.3797 Ops/s 6.2951 Ops/s $\color{#35bf28}+1.34\%$
test_sync 0.1773s 89.7749ms 11.1390 Ops/s 10.5790 Ops/s $\textbf{\color{#35bf28}+5.29\%}$
test_async 0.2221s 84.2675ms 11.8670 Ops/s 11.7256 Ops/s $\color{#35bf28}+1.21\%$
test_simple 0.9236s 0.8406s 1.1896 Ops/s 1.1734 Ops/s $\color{#35bf28}+1.38\%$
test_transformed 1.8884s 1.8131s 0.5515 Ops/s 0.5407 Ops/s $\color{#35bf28}+2.01\%$
test_serial 2.3426s 2.2790s 0.4388 Ops/s 0.4323 Ops/s $\color{#35bf28}+1.51\%$
test_parallel 1.7768s 1.7693s 0.5652 Ops/s 0.5311 Ops/s $\textbf{\color{#35bf28}+6.43\%}$
test_step_mdp_speed[True-True-True-True-True] 0.2974ms 44.4949μs 22.4745 KOps/s 21.9998 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-True-False] 0.7830ms 25.5545μs 39.1320 KOps/s 38.9144 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[True-True-True-False-True] 0.4320ms 31.0642μs 32.1914 KOps/s 31.0836 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[True-True-True-False-False] 70.4020μs 17.6996μs 56.4986 KOps/s 56.0605 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-True-False-True-True] 90.6020μs 46.0919μs 21.6958 KOps/s 20.9687 KOps/s $\color{#35bf28}+3.47\%$
test_step_mdp_speed[True-True-False-True-False] 87.3020μs 27.1060μs 36.8922 KOps/s 35.6317 KOps/s $\color{#35bf28}+3.54\%$
test_step_mdp_speed[True-True-False-False-True] 93.8020μs 33.5085μs 29.8432 KOps/s 29.2637 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-True-False-False-False] 45.7010μs 19.5018μs 51.2774 KOps/s 49.4438 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[True-False-True-True-True] 90.7020μs 47.9343μs 20.8619 KOps/s 20.3007 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[True-False-True-True-False] 58.9020μs 28.9160μs 34.5830 KOps/s 33.6292 KOps/s $\color{#35bf28}+2.84\%$
test_step_mdp_speed[True-False-True-False-True] 60.4010μs 33.2014μs 30.1192 KOps/s 29.2241 KOps/s $\color{#35bf28}+3.06\%$
test_step_mdp_speed[True-False-True-False-False] 0.1117ms 19.4409μs 51.4380 KOps/s 49.7357 KOps/s $\color{#35bf28}+3.42\%$
test_step_mdp_speed[True-False-False-True-True] 86.4010μs 49.8016μs 20.0797 KOps/s 19.7449 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-False-False-True-False] 85.9020μs 30.8808μs 32.3826 KOps/s 31.8688 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-False-False-False-True] 67.0020μs 34.9359μs 28.6238 KOps/s 27.8814 KOps/s $\color{#35bf28}+2.66\%$
test_step_mdp_speed[True-False-False-False-False] 0.1072ms 21.4017μs 46.7252 KOps/s 46.0595 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-True-True-True] 73.5020μs 47.9102μs 20.8724 KOps/s 20.2902 KOps/s $\color{#35bf28}+2.87\%$
test_step_mdp_speed[False-True-True-True-False] 70.9020μs 29.3660μs 34.0530 KOps/s 32.6832 KOps/s $\color{#35bf28}+4.19\%$
test_step_mdp_speed[False-True-True-False-True] 62.1020μs 36.7046μs 27.2445 KOps/s 25.8512 KOps/s $\textbf{\color{#35bf28}+5.39\%}$
test_step_mdp_speed[False-True-True-False-False] 4.5349ms 22.1249μs 45.1980 KOps/s 44.4911 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[False-True-False-True-True] 0.1411ms 49.5490μs 20.1821 KOps/s 19.5972 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-True-False-True-False] 67.3010μs 30.9539μs 32.3061 KOps/s 31.9029 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-False-False-True] 86.9020μs 38.8672μs 25.7286 KOps/s 24.8839 KOps/s $\color{#35bf28}+3.39\%$
test_step_mdp_speed[False-True-False-False-False] 84.9020μs 23.6695μs 42.2484 KOps/s 41.7181 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-False-True-True-True] 0.1047ms 51.1839μs 19.5374 KOps/s 18.8626 KOps/s $\color{#35bf28}+3.58\%$
test_step_mdp_speed[False-False-True-True-False] 58.7020μs 32.7827μs 30.5039 KOps/s 30.0883 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[False-False-True-False-True] 88.7020μs 38.6991μs 25.8404 KOps/s 25.3622 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-True-False-False] 48.5010μs 23.3128μs 42.8950 KOps/s 42.1756 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-False-False-True-True] 0.1403ms 52.9026μs 18.9026 KOps/s 18.6659 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-False-False-True-False] 61.6010μs 34.0225μs 29.3923 KOps/s 28.6745 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[False-False-False-False-True] 0.1013ms 40.3963μs 24.7547 KOps/s 24.4918 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-False-False-False] 52.5010μs 24.8959μs 40.1672 KOps/s 38.8684 KOps/s $\color{#35bf28}+3.34\%$
test_values[generalized_advantage_estimate-True-True] 18.5707ms 13.7244ms 72.8629 Ops/s 72.0861 Ops/s $\color{#35bf28}+1.08\%$
test_values[vec_generalized_advantage_estimate-True-True] 50.8812ms 43.1109ms 23.1960 Ops/s 22.7071 Ops/s $\color{#35bf28}+2.15\%$
test_values[td0_return_estimate-False-False] 0.4315ms 0.2165ms 4.6193 KOps/s 4.3016 KOps/s $\textbf{\color{#35bf28}+7.39\%}$
test_values[td1_return_estimate-False-False] 13.8977ms 13.2122ms 75.6879 Ops/s 74.7225 Ops/s $\color{#35bf28}+1.29\%$
test_values[vec_td1_return_estimate-False-False] 49.2086ms 43.0279ms 23.2407 Ops/s 20.9118 Ops/s $\textbf{\color{#35bf28}+11.14\%}$
test_values[td_lambda_return_estimate-True-False] 33.4241ms 31.7925ms 31.4540 Ops/s 31.2713 Ops/s $\color{#35bf28}+0.58\%$
test_values[vec_td_lambda_return_estimate-True-False] 64.3502ms 43.7509ms 22.8567 Ops/s 22.5262 Ops/s $\color{#35bf28}+1.47\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 17.6941ms 11.8263ms 84.5571 Ops/s 86.3924 Ops/s $\color{#d91a1a}-2.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 4.9304ms 3.4580ms 289.1851 Ops/s 286.6732 Ops/s $\color{#35bf28}+0.88\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 6.7233ms 0.4696ms 2.1296 KOps/s 2.1117 KOps/s $\color{#35bf28}+0.85\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 65.1610ms 60.7183ms 16.4695 Ops/s 17.1279 Ops/s $\color{#d91a1a}-3.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 12.5013ms 2.8576ms 349.9412 Ops/s 331.2926 Ops/s $\textbf{\color{#35bf28}+5.63\%}$
test_dqn_speed 6.6036ms 1.7787ms 562.2233 Ops/s 544.3759 Ops/s $\color{#35bf28}+3.28\%$
test_ddpg_speed 3.4632ms 2.7113ms 368.8235 Ops/s 364.2796 Ops/s $\color{#35bf28}+1.25\%$
test_sac_speed 8.3469ms 7.9735ms 125.4152 Ops/s 121.1423 Ops/s $\color{#35bf28}+3.53\%$
test_redq_speed 23.8363ms 17.0002ms 58.8228 Ops/s 57.5009 Ops/s $\color{#35bf28}+2.30\%$
test_redq_deprec_speed 19.0612ms 12.5850ms 79.4599 Ops/s 76.2074 Ops/s $\color{#35bf28}+4.27\%$
test_td3_speed 10.8445ms 9.8347ms 101.6804 Ops/s 98.2324 Ops/s $\color{#35bf28}+3.51\%$
test_cql_speed 42.0087ms 37.5755ms 26.6131 Ops/s 35.6591 Ops/s $\textbf{\color{#d91a1a}-25.37\%}$
test_a2c_speed 11.6288ms 5.1661ms 193.5689 Ops/s 184.8808 Ops/s $\color{#35bf28}+4.70\%$
test_ppo_speed 12.2331ms 5.5491ms 180.2106 Ops/s 177.2235 Ops/s $\color{#35bf28}+1.69\%$
test_reinforce_speed 10.8768ms 4.1154ms 242.9908 Ops/s 246.1912 Ops/s $\color{#d91a1a}-1.30\%$
test_iql_speed 30.8454ms 23.0698ms 43.3467 Ops/s 45.0882 Ops/s $\color{#d91a1a}-3.86\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2678ms 2.5772ms 388.0235 Ops/s 347.4489 Ops/s $\textbf{\color{#35bf28}+11.68\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.6639ms 2.8023ms 356.8456 Ops/s 350.5366 Ops/s $\color{#35bf28}+1.80\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.2820ms 2.7939ms 357.9257 Ops/s 345.5580 Ops/s $\color{#35bf28}+3.58\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.8511ms 2.5285ms 395.4934 Ops/s 364.0760 Ops/s $\textbf{\color{#35bf28}+8.63\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.7344ms 2.7745ms 360.4297 Ops/s 340.6829 Ops/s $\textbf{\color{#35bf28}+5.80\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.0499ms 2.7750ms 360.3587 Ops/s 341.7213 Ops/s $\textbf{\color{#35bf28}+5.45\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.2534s 3.4249ms 291.9767 Ops/s 373.7147 Ops/s $\textbf{\color{#d91a1a}-21.87\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.9190ms 2.7365ms 365.4346 Ops/s 336.6105 Ops/s $\textbf{\color{#35bf28}+8.56\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0920ms 2.7587ms 362.4939 Ops/s 339.3665 Ops/s $\textbf{\color{#35bf28}+6.81\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.4223ms 2.5840ms 386.9953 Ops/s 357.8473 Ops/s $\textbf{\color{#35bf28}+8.15\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.8720ms 2.7714ms 360.8336 Ops/s 332.4541 Ops/s $\textbf{\color{#35bf28}+8.54\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4916ms 2.8212ms 354.4583 Ops/s 327.7781 Ops/s $\textbf{\color{#35bf28}+8.14\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.6677ms 2.6120ms 382.8458 Ops/s 357.2563 Ops/s $\textbf{\color{#35bf28}+7.16\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.9443ms 2.7869ms 358.8183 Ops/s 330.4180 Ops/s $\textbf{\color{#35bf28}+8.60\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.3823ms 2.8044ms 356.5879 Ops/s 334.6579 Ops/s $\textbf{\color{#35bf28}+6.55\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1098ms 2.5810ms 387.4452 Ops/s 356.8154 Ops/s $\textbf{\color{#35bf28}+8.58\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.5445ms 2.7936ms 357.9568 Ops/s 325.1723 Ops/s $\textbf{\color{#35bf28}+10.08\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.0486ms 2.8130ms 355.4951 Ops/s 319.7392 Ops/s $\textbf{\color{#35bf28}+11.18\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2843s 33.2504ms 30.0749 Ops/s 27.7060 Ops/s $\textbf{\color{#35bf28}+8.55\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1543s 27.6347ms 36.1864 Ops/s 33.2840 Ops/s $\textbf{\color{#35bf28}+8.72\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1562s 30.5360ms 32.7482 Ops/s 30.1672 Ops/s $\textbf{\color{#35bf28}+8.56\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1633s 28.3442ms 35.2806 Ops/s 33.6218 Ops/s $\color{#35bf28}+4.93\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1612s 30.9651ms 32.2945 Ops/s 30.5937 Ops/s $\textbf{\color{#35bf28}+5.56\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1640s 28.4532ms 35.1454 Ops/s 33.8263 Ops/s $\color{#35bf28}+3.90\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1624s 30.8322ms 32.4336 Ops/s 30.7208 Ops/s $\textbf{\color{#35bf28}+5.58\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1581s 27.8946ms 35.8492 Ops/s 30.3895 Ops/s $\textbf{\color{#35bf28}+17.97\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1580s 30.4057ms 32.8886 Ops/s 33.6312 Ops/s $\color{#d91a1a}-2.21\%$

@vmoens vmoens marked this pull request as ready for review September 7, 2023 12:53
@vmoens vmoens changed the title [Feature] ClipTransform [Feature] ClipTransform + rename min/maximum -> low/high Sep 7, 2023
@albertbou92
Copy link
Contributor

LGTM!

@vmoens vmoens merged commit bca6adf into main Sep 7, 2023
@vmoens vmoens deleted the clip_transform branch September 7, 2023 14:16
vmoens added a commit to hyerra/rl that referenced this pull request Oct 10, 2023
@vmoens
[Feature] ClipTransform + rename min/maximum -> low/high (pytorch…
e28ef5c 
…#1500)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@albertbou92 albertbou92 albertbou92 approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @albertbou92 @facebook-github-bot