Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Remove select() in favor of empty() #1811

Merged
merged 4 commits into from
Jan 17, 2024
Merged

[BugFix] Remove select() in favor of empty() #1811

merged 4 commits into from
Jan 17, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 17, 2024 •
edited
Loading

Select now returns a locked TD if the original TD was memmaped or shared. We now use empty which does not have that feature.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 17, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1811

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 25 Unrelated Failures

As of commit fcf8df0 with merge base baea10b (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 17, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 17, 2024 •
edited
Loading

$\color{#35bf28}\textsf{\Large✔\kern{0.2cm}\normalsize OK}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}0$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1132s 0.1128s 8.8631 Ops/s 8.9569 Ops/s $\color{#d91a1a}-1.05\%$
test_sync 0.1748s 0.1030s 9.7054 Ops/s 9.7547 Ops/s $\color{#d91a1a}-0.51\%$
test_async 0.2478s 91.4115ms 10.9395 Ops/s 10.8789 Ops/s $\color{#35bf28}+0.56\%$
test_single_pixels 0.1372s 0.1365s 7.3260 Ops/s 7.3264 Ops/s $-0.01\%$
test_sync_pixels 77.2460ms 74.7223ms 13.3829 Ops/s 13.0261 Ops/s $\color{#35bf28}+2.74\%$
test_async_pixels 0.2061s 70.3341ms 14.2179 Ops/s 14.2403 Ops/s $\color{#d91a1a}-0.16\%$
test_simple 0.8864s 0.8160s 1.2255 Ops/s 1.2438 Ops/s $\color{#d91a1a}-1.48\%$
test_transformed 1.0365s 1.0360s 0.9652 Ops/s 0.9663 Ops/s $\color{#d91a1a}-0.11\%$
test_serial 2.3183s 2.2553s 0.4434 Ops/s 0.4453 Ops/s $\color{#d91a1a}-0.44\%$
test_parallel 2.1041s 1.8584s 0.5381 Ops/s 0.5386 Ops/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-True-True-True-True] 98.7310μs 32.4256μs 30.8398 KOps/s 30.9279 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-True-True-False] 40.1510μs 18.8937μs 52.9278 KOps/s 51.5913 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-True-True-False-True] 37.9100μs 18.0511μs 55.3984 KOps/s 54.8031 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-True-True-False-False] 35.3510μs 10.8679μs 92.0141 KOps/s 90.6387 KOps/s $\color{#35bf28}+1.52\%$
test_step_mdp_speed[True-True-False-True-True] 60.6710μs 33.7392μs 29.6391 KOps/s 29.3684 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-True-False] 43.6810μs 21.0525μs 47.5002 KOps/s 46.9249 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[True-True-False-False-True] 47.8800μs 19.8687μs 50.3305 KOps/s 49.1011 KOps/s $\color{#35bf28}+2.50\%$
test_step_mdp_speed[True-True-False-False-False] 32.1810μs 12.8496μs 77.8233 KOps/s 77.6417 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-True-True-True] 62.2910μs 35.2734μs 28.3500 KOps/s 27.4311 KOps/s $\color{#35bf28}+3.35\%$
test_step_mdp_speed[True-False-True-True-False] 47.0400μs 22.6685μs 44.1141 KOps/s 42.7751 KOps/s $\color{#35bf28}+3.13\%$
test_step_mdp_speed[True-False-True-False-True] 40.3100μs 19.6336μs 50.9332 KOps/s 48.9997 KOps/s $\color{#35bf28}+3.95\%$
test_step_mdp_speed[True-False-True-False-False] 35.6610μs 12.5199μs 79.8730 KOps/s 76.7013 KOps/s $\color{#35bf28}+4.14\%$
test_step_mdp_speed[True-False-False-True-True] 62.9910μs 37.4761μs 26.6837 KOps/s 26.4434 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-False-False-True-False] 42.8210μs 24.8887μs 40.1789 KOps/s 39.5131 KOps/s $\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-False-False-True] 44.3210μs 21.4474μs 46.6258 KOps/s 45.4767 KOps/s $\color{#35bf28}+2.53\%$
test_step_mdp_speed[True-False-False-False-False] 38.0200μs 14.3034μs 69.9134 KOps/s 67.1064 KOps/s $\color{#35bf28}+4.18\%$
test_step_mdp_speed[False-True-True-True-True] 62.7210μs 35.1661μs 28.4365 KOps/s 28.0901 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-True-True-True-False] 48.7400μs 23.0668μs 43.3523 KOps/s 43.0735 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-True-False-True] 40.4800μs 23.9215μs 41.8033 KOps/s 40.6649 KOps/s $\color{#35bf28}+2.80\%$
test_step_mdp_speed[False-True-True-False-False] 34.1010μs 14.4958μs 68.9854 KOps/s 66.8949 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[False-True-False-True-True] 65.6100μs 37.5411μs 26.6374 KOps/s 26.3260 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-True-False-True-False] 46.2400μs 25.0603μs 39.9037 KOps/s 39.9248 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-True-False-False-True] 51.3800μs 25.3071μs 39.5146 KOps/s 38.3961 KOps/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-True-False-False-False] 40.7200μs 16.3678μs 61.0955 KOps/s 60.7777 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-True-True-True] 68.6210μs 39.2505μs 25.4774 KOps/s 25.1853 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-True-True-False] 50.4800μs 26.3168μs 37.9985 KOps/s 36.5096 KOps/s $\color{#35bf28}+4.08\%$
test_step_mdp_speed[False-False-True-False-True] 50.5400μs 24.8314μs 40.2716 KOps/s 38.0764 KOps/s $\textbf{\color{#35bf28}+5.77\%}$
test_step_mdp_speed[False-False-True-False-False] 35.1600μs 16.3761μs 61.0646 KOps/s 60.6401 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[False-False-False-True-True] 63.5010μs 40.6324μs 24.6109 KOps/s 24.3275 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-False-True-False] 63.3810μs 28.6803μs 34.8671 KOps/s 34.5125 KOps/s $\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-False-False-False-True] 49.1310μs 26.7587μs 37.3710 KOps/s 36.3665 KOps/s $\color{#35bf28}+2.76\%$
test_step_mdp_speed[False-False-False-False-False] 42.6300μs 18.0682μs 55.3457 KOps/s 54.3578 KOps/s $\color{#35bf28}+1.82\%$
test_values[generalized_advantage_estimate-True-True] 24.2480ms 23.8202ms 41.9812 Ops/s 40.6240 Ops/s $\color{#35bf28}+3.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 85.7838ms 3.2717ms 305.6486 Ops/s 301.0659 Ops/s $\color{#35bf28}+1.52\%$
test_values[td0_return_estimate-False-False] 91.6310μs 60.3797μs 16.5618 KOps/s 16.5028 KOps/s $\color{#35bf28}+0.36\%$
test_values[td1_return_estimate-False-False] 51.9705ms 51.2075ms 19.5284 Ops/s 18.7665 Ops/s $\color{#35bf28}+4.06\%$
test_values[vec_td1_return_estimate-False-False] 2.1350ms 1.7493ms 571.6637 Ops/s 568.4503 Ops/s $\color{#35bf28}+0.57\%$
test_values[td_lambda_return_estimate-True-False] 84.9051ms 82.2170ms 12.1629 Ops/s 11.7336 Ops/s $\color{#35bf28}+3.66\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0742ms 1.7458ms 572.8171 Ops/s 570.7419 Ops/s $\color{#35bf28}+0.36\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 22.6475ms 22.5161ms 44.4127 Ops/s 43.1337 Ops/s $\color{#35bf28}+2.97\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8319ms 0.6815ms 1.4674 KOps/s 1.4662 KOps/s $\color{#35bf28}+0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7001ms 0.6407ms 1.5607 KOps/s 1.5602 KOps/s $\color{#35bf28}+0.03\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5057ms 1.4382ms 695.2947 Ops/s 690.0656 Ops/s $\color{#35bf28}+0.76\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9277ms 0.6624ms 1.5096 KOps/s 1.5122 KOps/s $\color{#d91a1a}-0.17\%$
test_dqn_speed 7.5457ms 7.1601ms 139.6628 Ops/s 123.5552 Ops/s $\textbf{\color{#35bf28}+13.04\%}$
test_ddpg_speed 98.3593ms 15.2530ms 65.5610 Ops/s 68.6617 Ops/s $\color{#d91a1a}-4.52\%$
test_sac_speed 28.6504ms 28.2345ms 35.4177 Ops/s 34.5608 Ops/s $\color{#35bf28}+2.48\%$
test_redq_speed 47.5095ms 46.4147ms 21.5449 Ops/s 21.4593 Ops/s $\color{#35bf28}+0.40\%$
test_redq_deprec_speed 23.8956ms 23.1996ms 43.1042 Ops/s 42.2809 Ops/s $\color{#35bf28}+1.95\%$
test_td3_speed 28.9948ms 19.2039ms 52.0727 Ops/s 50.6956 Ops/s $\color{#35bf28}+2.72\%$
test_cql_speed 81.9959ms 80.8356ms 12.3708 Ops/s 12.3375 Ops/s $\color{#35bf28}+0.27\%$
test_a2c_speed 26.8866ms 25.9197ms 38.5806 Ops/s 38.8828 Ops/s $\color{#d91a1a}-0.78\%$
test_ppo_speed 27.1251ms 26.2199ms 38.1390 Ops/s 38.3758 Ops/s $\color{#d91a1a}-0.62\%$
test_reinforce_speed 25.5350ms 24.9623ms 40.0604 Ops/s 40.0957 Ops/s $\color{#d91a1a}-0.09\%$
test_iql_speed 56.6226ms 55.6602ms 17.9662 Ops/s 17.7348 Ops/s $\color{#35bf28}+1.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2102ms 1.8290ms 546.7573 Ops/s 550.7338 Ops/s $\color{#d91a1a}-0.72\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9493ms 0.8323ms 1.2015 KOps/s 1.1994 KOps/s $\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9231ms 0.8151ms 1.2269 KOps/s 1.2259 KOps/s $\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4622ms 1.7913ms 558.2664 Ops/s 564.1625 Ops/s $\color{#d91a1a}-1.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9288ms 0.8207ms 1.2184 KOps/s 1.2184 KOps/s $-0.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9204ms 0.8037ms 1.2443 KOps/s 1.2425 KOps/s $\color{#35bf28}+0.15\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0966ms 2.0641ms 484.4641 Ops/s 483.0575 Ops/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0561ms 0.9484ms 1.0544 KOps/s 1.0592 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0537ms 0.9309ms 1.0742 KOps/s 1.0774 KOps/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2199ms 1.8316ms 545.9642 Ops/s 550.2394 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9361ms 0.8329ms 1.2006 KOps/s 1.2016 KOps/s $\color{#d91a1a}-0.08\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9281ms 0.8163ms 1.2251 KOps/s 1.2250 KOps/s $+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.4675ms 1.8037ms 554.4149 Ops/s 558.9589 Ops/s $\color{#d91a1a}-0.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9408ms 0.8223ms 1.2161 KOps/s 1.2152 KOps/s $\color{#35bf28}+0.08\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9221ms 0.8054ms 1.2416 KOps/s 1.2411 KOps/s $\color{#35bf28}+0.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1712ms 2.0811ms 480.5221 Ops/s 482.6044 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0534ms 0.9496ms 1.0530 KOps/s 1.0551 KOps/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0227ms 0.9337ms 1.0711 KOps/s 1.0749 KOps/s $\color{#d91a1a}-0.36\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1329s 17.1823ms 58.1993 Ops/s 58.0067 Ops/s $\color{#35bf28}+0.33\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1195s 15.7992ms 63.2943 Ops/s 63.1116 Ops/s $\color{#35bf28}+0.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.2913ms 3.2933ms 303.6424 Ops/s 295.5559 Ops/s $\color{#35bf28}+2.74\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1195s 16.8369ms 59.3934 Ops/s 58.7945 Ops/s $\color{#35bf28}+1.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.1710ms 13.6325ms 73.3542 Ops/s 73.3743 Ops/s $\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.2991ms 3.2917ms 303.7981 Ops/s 296.1647 Ops/s $\color{#35bf28}+2.58\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1215s 17.0841ms 58.5339 Ops/s 57.8432 Ops/s $\color{#35bf28}+1.19\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.4121ms 13.7764ms 72.5881 Ops/s 70.7272 Ops/s $\color{#35bf28}+2.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.5687ms 3.4825ms 287.1479 Ops/s 283.9632 Ops/s $\color{#35bf28}+1.12\%$

@vmoens vmoens added the bug Something isn't working label Jan 17, 2024
@vmoens vmoens merged commit 93748e9 into main Jan 17, 2024
23 of 46 checks passed
@vmoens vmoens deleted the fix-td-update branch January 17, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot