Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Minor] More efficient SAC v1 #1507

Merged
merged 1 commit into from
Sep 7, 2023
Merged

[Minor] More efficient SAC v1 #1507

merged 1 commit into from
Sep 7, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 7, 2023

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 7, 2023
@vmoens vmoens marked this pull request as ready for review September 7, 2023 14:59
matteobettini
matteobettini approved these changes Sep 7, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link

github-actions bot commented Sep 7, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1563s 0.1558s 6.4202 Ops/s 6.3849 Ops/s $\color{#35bf28}+0.55\%$
test_sync 0.1615s 90.6232ms 11.0347 Ops/s 11.4043 Ops/s $\color{#d91a1a}-3.24\%$
test_async 0.2222s 81.8100ms 12.2234 Ops/s 12.1981 Ops/s $\color{#35bf28}+0.21\%$
test_simple 0.9057s 0.8336s 1.1996 Ops/s 1.1830 Ops/s $\color{#35bf28}+1.40\%$
test_transformed 1.8773s 1.8290s 0.5467 Ops/s 0.5465 Ops/s $\color{#35bf28}+0.05\%$
test_serial 2.3448s 2.2835s 0.4379 Ops/s 0.4300 Ops/s $\color{#35bf28}+1.85\%$
test_parallel 1.8629s 1.8047s 0.5541 Ops/s 0.5558 Ops/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-True-True-True-True] 0.1805ms 44.4890μs 22.4774 KOps/s 22.6773 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-True-True-False] 44.8020μs 25.5116μs 39.1979 KOps/s 39.9098 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-True-True-False-True] 0.1071ms 31.1501μs 32.1026 KOps/s 32.0448 KOps/s $\color{#35bf28}+0.18\%$
test_step_mdp_speed[True-True-True-False-False] 45.6010μs 17.5539μs 56.9674 KOps/s 56.6045 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-True-False-True-True] 0.1282ms 46.2312μs 21.6304 KOps/s 21.8802 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[True-True-False-True-False] 62.7020μs 27.1871μs 36.7822 KOps/s 37.1961 KOps/s $\color{#d91a1a}-1.11\%$
test_step_mdp_speed[True-True-False-False-True] 0.1007ms 33.1352μs 30.1793 KOps/s 29.6881 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-False-False-False] 0.1032ms 19.7617μs 50.6031 KOps/s 51.7614 KOps/s $\color{#d91a1a}-2.24\%$
test_step_mdp_speed[True-False-True-True-True] 73.9020μs 48.1632μs 20.7627 KOps/s 20.9488 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[True-False-True-True-False] 0.1061ms 29.3885μs 34.0269 KOps/s 34.9010 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[True-False-True-False-True] 60.7020μs 33.0884μs 30.2221 KOps/s 30.2072 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-False-False] 87.4030μs 19.9819μs 50.0452 KOps/s 51.2112 KOps/s $\color{#d91a1a}-2.28\%$
test_step_mdp_speed[True-False-False-True-True] 70.0030μs 50.0473μs 19.9811 KOps/s 20.3382 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-False-False-True-False] 1.9627ms 31.1922μs 32.0593 KOps/s 32.7047 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[True-False-False-False-True] 0.1071ms 35.4264μs 28.2275 KOps/s 28.7510 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[True-False-False-False-False] 46.8020μs 21.0489μs 47.5085 KOps/s 47.2128 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-True-True-True] 0.1438ms 48.3630μs 20.6770 KOps/s 20.9393 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-True-True-True-False] 53.4010μs 29.1470μs 34.3089 KOps/s 34.5080 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-True-False-True] 0.1228ms 37.1797μs 26.8964 KOps/s 27.0726 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-True-True-False-False] 3.3159ms 21.7673μs 45.9405 KOps/s 46.0845 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[False-True-False-True-True] 0.1269ms 50.4512μs 19.8211 KOps/s 20.3192 KOps/s $\color{#d91a1a}-2.45\%$
test_step_mdp_speed[False-True-False-True-False] 0.1124ms 30.8876μs 32.3754 KOps/s 33.0329 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-True-False-False-True] 88.5030μs 39.0120μs 25.6332 KOps/s 25.9351 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-True-False-False-False] 0.1276ms 23.3426μs 42.8400 KOps/s 43.4306 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-False-True-True-True] 0.1307ms 52.1230μs 19.1854 KOps/s 19.6406 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[False-False-True-True-False] 0.1843ms 32.7926μs 30.4946 KOps/s 30.9942 KOps/s $\color{#d91a1a}-1.61\%$
test_step_mdp_speed[False-False-True-False-True] 85.4030μs 39.3576μs 25.4080 KOps/s 25.7423 KOps/s $\color{#d91a1a}-1.30\%$
test_step_mdp_speed[False-False-True-False-False] 54.2020μs 23.2911μs 42.9349 KOps/s 43.1512 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-False-False-True-True] 0.1430ms 53.4053μs 18.7247 KOps/s 18.9630 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-False-False-True-False] 0.1110ms 34.1343μs 29.2960 KOps/s 29.6037 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-False-False-False-True] 66.1020μs 40.2939μs 24.8176 KOps/s 24.9940 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-False-False-False] 0.1212ms 24.6587μs 40.5536 KOps/s 40.9492 KOps/s $\color{#d91a1a}-0.97\%$
test_values[generalized_advantage_estimate-True-True] 13.9559ms 13.4912ms 74.1226 Ops/s 72.7913 Ops/s $\color{#35bf28}+1.83\%$
test_values[vec_generalized_advantage_estimate-True-True] 52.5578ms 42.1893ms 23.7027 Ops/s 23.5069 Ops/s $\color{#35bf28}+0.83\%$
test_values[td0_return_estimate-False-False] 0.3656ms 0.1948ms 5.1345 KOps/s 4.8878 KOps/s $\textbf{\color{#35bf28}+5.05\%}$
test_values[td1_return_estimate-False-False] 13.3677ms 13.1325ms 76.1470 Ops/s 74.0449 Ops/s $\color{#35bf28}+2.84\%$
test_values[vec_td1_return_estimate-False-False] 49.5840ms 41.9429ms 23.8419 Ops/s 23.7379 Ops/s $\color{#35bf28}+0.44\%$
test_values[td_lambda_return_estimate-True-False] 31.8076ms 31.4941ms 31.7519 Ops/s 31.3750 Ops/s $\color{#35bf28}+1.20\%$
test_values[vec_td_lambda_return_estimate-True-False] 48.7969ms 41.5881ms 24.0453 Ops/s 23.8433 Ops/s $\color{#35bf28}+0.85\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.9717ms 11.8190ms 84.6098 Ops/s 83.2745 Ops/s $\color{#35bf28}+1.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.5012ms 3.4482ms 290.0091 Ops/s 289.6939 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8291ms 0.4729ms 2.1146 KOps/s 2.0778 KOps/s $\color{#35bf28}+1.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 57.8112ms 52.3946ms 19.0860 Ops/s 18.3638 Ops/s $\color{#35bf28}+3.93\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 9.5675ms 2.8632ms 349.2545 Ops/s 348.7739 Ops/s $\color{#35bf28}+0.14\%$
test_dqn_speed 9.0510ms 1.8828ms 531.1176 Ops/s 532.2268 Ops/s $\color{#d91a1a}-0.21\%$
test_ddpg_speed 81.1439ms 3.4861ms 286.8564 Ops/s 366.2073 Ops/s $\textbf{\color{#d91a1a}-21.67\%}$
test_sac_speed 15.1065ms 8.2544ms 121.1472 Ops/s 120.6465 Ops/s $\color{#35bf28}+0.41\%$
test_redq_speed 24.6094ms 16.2295ms 61.6160 Ops/s 62.4264 Ops/s $\color{#d91a1a}-1.30\%$
test_redq_deprec_speed 20.0165ms 13.6027ms 73.5147 Ops/s 77.8986 Ops/s $\textbf{\color{#d91a1a}-5.63\%}$
test_td3_speed 11.6805ms 10.2371ms 97.6843 Ops/s 94.9488 Ops/s $\color{#35bf28}+2.88\%$
test_cql_speed 35.1918ms 27.9375ms 35.7942 Ops/s 37.8576 Ops/s $\textbf{\color{#d91a1a}-5.45\%}$
test_a2c_speed 17.3251ms 5.4592ms 183.1758 Ops/s 188.6495 Ops/s $\color{#d91a1a}-2.90\%$
test_ppo_speed 21.0929ms 5.9178ms 168.9816 Ops/s 175.9751 Ops/s $\color{#d91a1a}-3.97\%$
test_reinforce_speed 10.5486ms 4.2190ms 237.0237 Ops/s 242.5743 Ops/s $\color{#d91a1a}-2.29\%$
test_iql_speed 29.1973ms 21.9943ms 45.4664 Ops/s 46.6205 Ops/s $\color{#d91a1a}-2.48\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3891ms 2.6883ms 371.9801 Ops/s 369.5195 Ops/s $\color{#35bf28}+0.67\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.2440ms 2.8424ms 351.8118 Ops/s 350.7029 Ops/s $\color{#35bf28}+0.32\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.6782ms 2.8618ms 349.4301 Ops/s 345.1800 Ops/s $\color{#35bf28}+1.23\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3647ms 2.6852ms 372.4151 Ops/s 358.7119 Ops/s $\color{#35bf28}+3.82\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.0529ms 2.8644ms 349.1164 Ops/s 345.4410 Ops/s $\color{#35bf28}+1.06\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.5952ms 2.8635ms 349.2185 Ops/s 341.5981 Ops/s $\color{#35bf28}+2.23\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5334ms 2.7176ms 367.9666 Ops/s 362.0555 Ops/s $\color{#35bf28}+1.63\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.2909ms 2.8776ms 347.5087 Ops/s 343.6296 Ops/s $\color{#35bf28}+1.13\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.2269ms 2.8934ms 345.6189 Ops/s 351.7327 Ops/s $\color{#d91a1a}-1.74\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.5501ms 2.7038ms 369.8563 Ops/s 301.0244 Ops/s $\textbf{\color{#35bf28}+22.87\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 4.6483ms 2.8526ms 350.5630 Ops/s 347.2405 Ops/s $\color{#35bf28}+0.96\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.4867ms 2.8714ms 348.2663 Ops/s 348.9504 Ops/s $\color{#d91a1a}-0.20\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8401ms 2.7253ms 366.9265 Ops/s 363.2250 Ops/s $\color{#35bf28}+1.02\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.0060ms 2.8992ms 344.9284 Ops/s 350.4767 Ops/s $\color{#d91a1a}-1.58\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.5688ms 2.8759ms 347.7196 Ops/s 347.8314 Ops/s $\color{#d91a1a}-0.03\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2105ms 2.7787ms 359.8826 Ops/s 369.0634 Ops/s $\color{#d91a1a}-2.49\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.9633ms 2.9032ms 344.4453 Ops/s 347.0231 Ops/s $\color{#d91a1a}-0.74\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.2774ms 2.8694ms 348.5070 Ops/s 345.8308 Ops/s $\color{#35bf28}+0.77\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2729s 29.8489ms 33.5020 Ops/s 31.9227 Ops/s $\color{#35bf28}+4.95\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1487s 29.6773ms 33.6958 Ops/s 34.0286 Ops/s $\color{#d91a1a}-0.98\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1404s 26.6852ms 37.4740 Ops/s 37.4237 Ops/s $\color{#35bf28}+0.13\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1409s 29.1858ms 34.2632 Ops/s 37.0732 Ops/s $\textbf{\color{#d91a1a}-7.58\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1444s 26.9540ms 37.1003 Ops/s 40.9951 Ops/s $\textbf{\color{#d91a1a}-9.50\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1400s 29.1528ms 34.3020 Ops/s 34.3936 Ops/s $\color{#d91a1a}-0.27\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1426s 26.8372ms 37.2617 Ops/s 37.0653 Ops/s $\color{#35bf28}+0.53\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1452s 29.4202ms 33.9903 Ops/s 34.1455 Ops/s $\color{#d91a1a}-0.45\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1443s 29.3547ms 34.0661 Ops/s 37.3065 Ops/s $\textbf{\color{#d91a1a}-8.69\%}$

@vmoens vmoens added the Refactoring Refactoring of an existing feature label Sep 7, 2023
@vmoens vmoens merged commit 435584e into main Sep 7, 2023
@vmoens vmoens deleted the minor_sac branch September 7, 2023 15:12
vmoens added a commit to hyerra/rl that referenced this pull request Oct 10, 2023
@vmoens
[Minor] More efficient SAC v1 (pytorch#1507)
97d19f3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot