Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refacto] Migration due to tensordict 473 and 474 #1354

Merged
merged 8 commits into from
Jul 4, 2023
Merged

[Refacto] Migration due to tensordict 473 and 474 #1354

merged 8 commits into from
Jul 4, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 4, 2023 •
edited
Loading

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 4, 2023
@github-actions
Copy link

github-actions bot commented Jul 4, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1668s 0.1659s 6.0283 Ops/s 6.0486 Ops/s $\color{#d91a1a}-0.34\%$
test_sync 88.2755ms 87.0632ms 11.4859 Ops/s 11.4866 Ops/s $-0.01\%$
test_async 0.1412s 85.5286ms 11.6920 Ops/s 11.5746 Ops/s $\color{#35bf28}+1.01\%$
test_simple 0.7192s 0.6369s 1.5701 Ops/s 1.5785 Ops/s $\color{#d91a1a}-0.53\%$
test_transformed 1.7587s 1.7076s 0.5856 Ops/s 0.5914 Ops/s $\color{#d91a1a}-0.98\%$
test_serial 2.0130s 1.9532s 0.5120 Ops/s 0.5081 Ops/s $\color{#35bf28}+0.77\%$
test_parallel 1.6405s 1.5202s 0.6578 Ops/s 0.6587 Ops/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-True-True-True] 0.1628ms 38.8925μs 25.7119 KOps/s 22.5037 KOps/s $\textbf{\color{#35bf28}+14.26\%}$
test_step_mdp_speed[True-True-True-True-False] 54.1000μs 21.4122μs 46.7023 KOps/s 40.6500 KOps/s $\textbf{\color{#35bf28}+14.89\%}$
test_step_mdp_speed[True-True-True-False-True] 92.2990μs 29.5377μs 33.8551 KOps/s 33.2603 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-True-False-False] 38.6010μs 16.3513μs 61.1573 KOps/s 59.9862 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-True-False-True-True] 64.4010μs 39.6203μs 25.2396 KOps/s 24.7956 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-False-True-False] 51.6010μs 22.6356μs 44.1782 KOps/s 43.4514 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-True-False-False-True] 54.4010μs 30.7936μs 32.4743 KOps/s 31.6536 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-True-False-False-False] 61.2020μs 17.4576μs 57.2815 KOps/s 55.6791 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[True-False-True-True-True] 74.2020μs 41.1288μs 24.3138 KOps/s 23.9040 KOps/s $\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-False-True-True-False] 48.8010μs 23.8706μs 41.8925 KOps/s 41.0857 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[True-False-True-False-True] 91.8900μs 30.7786μs 32.4901 KOps/s 32.1198 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-False-True-False-False] 41.5960μs 17.4243μs 57.3910 KOps/s 56.0746 KOps/s $\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-False-False-True-True] 90.1890μs 41.9294μs 23.8496 KOps/s 23.2644 KOps/s $\color{#35bf28}+2.52\%$
test_step_mdp_speed[True-False-False-True-False] 47.4010μs 25.0956μs 39.8476 KOps/s 39.2858 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-False-False-True] 57.0010μs 31.7940μs 31.4525 KOps/s 30.7484 KOps/s $\color{#35bf28}+2.29\%$
test_step_mdp_speed[True-False-False-False-False] 44.3000μs 18.4696μs 54.1431 KOps/s 52.5129 KOps/s $\color{#35bf28}+3.10\%$
test_step_mdp_speed[False-True-True-True-True] 67.9010μs 40.8546μs 24.4770 KOps/s 23.6981 KOps/s $\color{#35bf28}+3.29\%$
test_step_mdp_speed[False-True-True-True-False] 51.3000μs 23.7923μs 42.0304 KOps/s 41.3494 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-True-True-False-True] 91.2010μs 35.6089μs 28.0829 KOps/s 28.0666 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-True-False-False] 90.0010μs 19.4390μs 51.4429 KOps/s 51.1283 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-False-True-True] 69.9020μs 41.9303μs 23.8491 KOps/s 22.7572 KOps/s $\color{#35bf28}+4.80\%$
test_step_mdp_speed[False-True-False-True-False] 87.7020μs 24.9482μs 40.0830 KOps/s 39.1082 KOps/s $\color{#35bf28}+2.49\%$
test_step_mdp_speed[False-True-False-False-True] 63.7010μs 35.9916μs 27.7842 KOps/s 27.0225 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[False-True-False-False-False] 0.6195ms 20.6838μs 48.3469 KOps/s 48.1673 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-False-True-True-True] 0.4502ms 43.0147μs 23.2479 KOps/s 22.5667 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-False-True-True-False] 53.1000μs 26.1978μs 38.1711 KOps/s 37.9794 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-True-False-True] 52.3000μs 36.0179μs 27.7640 KOps/s 27.0519 KOps/s $\color{#35bf28}+2.63\%$
test_step_mdp_speed[False-False-True-False-False] 84.0010μs 20.2841μs 49.2997 KOps/s 48.9173 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[False-False-False-True-True] 73.2000μs 43.5644μs 22.9545 KOps/s 22.3053 KOps/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-False-False-True-False] 49.2000μs 26.9802μs 37.0643 KOps/s 36.0365 KOps/s $\color{#35bf28}+2.85\%$
test_step_mdp_speed[False-False-False-False-True] 60.9000μs 36.6872μs 27.2575 KOps/s 26.6163 KOps/s $\color{#35bf28}+2.41\%$
test_step_mdp_speed[False-False-False-False-False] 0.1034ms 21.5393μs 46.4267 KOps/s 46.1031 KOps/s $\color{#35bf28}+0.70\%$
test_values[generalized_advantage_estimate-True-True] 15.4387ms 13.9703ms 71.5807 Ops/s 71.8937 Ops/s $\color{#d91a1a}-0.44\%$
test_values[vec_generalized_advantage_estimate-True-True] 44.2883ms 40.8973ms 24.4515 Ops/s 24.2666 Ops/s $\color{#35bf28}+0.76\%$
test_values[td0_return_estimate-False-False] 0.3326ms 0.1913ms 5.2286 KOps/s 5.0580 KOps/s $\color{#35bf28}+3.37\%$
test_values[td1_return_estimate-False-False] 13.6519ms 13.3959ms 74.6497 Ops/s 73.9576 Ops/s $\color{#35bf28}+0.94\%$
test_values[vec_td1_return_estimate-False-False] 41.3886ms 40.6562ms 24.5965 Ops/s 24.1242 Ops/s $\color{#35bf28}+1.96\%$
test_values[td_lambda_return_estimate-True-False] 32.9384ms 32.4894ms 30.7793 Ops/s 30.7109 Ops/s $\color{#35bf28}+0.22\%$
test_values[vec_td_lambda_return_estimate-True-False] 50.4543ms 41.2338ms 24.2520 Ops/s 24.4994 Ops/s $\color{#d91a1a}-1.01\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.4276ms 12.3357ms 81.0658 Ops/s 81.8417 Ops/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.8906ms 3.3460ms 298.8667 Ops/s 297.6328 Ops/s $\color{#35bf28}+0.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7638ms 0.4545ms 2.2000 KOps/s 2.2571 KOps/s $\color{#d91a1a}-2.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 58.0558ms 55.7663ms 17.9320 Ops/s 18.2010 Ops/s $\color{#d91a1a}-1.48\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.2181ms 2.7847ms 359.1100 Ops/s 356.3237 Ops/s $\color{#35bf28}+0.78\%$
test_dqn_speed 17.1138ms 1.6448ms 607.9850 Ops/s 609.1286 Ops/s $\color{#d91a1a}-0.19\%$
test_ddpg_speed 3.2976ms 2.6614ms 375.7353 Ops/s 365.0549 Ops/s $\color{#35bf28}+2.93\%$
test_sac_speed 86.9860ms 8.5917ms 116.3917 Ops/s 126.4078 Ops/s $\textbf{\color{#d91a1a}-7.92\%}$
test_redq_speed 15.8201ms 14.3770ms 69.5555 Ops/s 71.6207 Ops/s $\color{#d91a1a}-2.88\%$
test_redq_deprec_speed 16.8809ms 11.7210ms 85.3172 Ops/s 86.1830 Ops/s $\color{#d91a1a}-1.00\%$
test_td3_speed 11.3529ms 10.4615ms 95.5884 Ops/s 95.1097 Ops/s $\color{#35bf28}+0.50\%$
test_cql_speed 30.7344ms 25.2522ms 39.6005 Ops/s 39.5916 Ops/s $\color{#35bf28}+0.02\%$
test_a2c_speed 6.0593ms 5.1524ms 194.0825 Ops/s 195.1064 Ops/s $\color{#d91a1a}-0.52\%$
test_ppo_speed 6.6558ms 5.6129ms 178.1625 Ops/s 182.8532 Ops/s $\color{#d91a1a}-2.57\%$
test_reinforce_speed 5.1209ms 4.0881ms 244.6150 Ops/s 250.6191 Ops/s $\color{#d91a1a}-2.40\%$
test_iql_speed 22.7695ms 20.5701ms 48.6142 Ops/s 50.0931 Ops/s $\color{#d91a1a}-2.95\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.6203ms 4.2171ms 237.1297 Ops/s 237.7336 Ops/s $\color{#d91a1a}-0.25\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 7.0352ms 4.3739ms 228.6285 Ops/s 226.0915 Ops/s $\color{#35bf28}+1.12\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.2575ms 4.3353ms 230.6629 Ops/s 196.2265 Ops/s $\textbf{\color{#35bf28}+17.55\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.1289s 4.6741ms 213.9439 Ops/s 233.6863 Ops/s $\textbf{\color{#d91a1a}-8.45\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 16.7752ms 4.4539ms 224.5204 Ops/s 225.2671 Ops/s $\color{#d91a1a}-0.33\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 7.3815ms 4.3211ms 231.4240 Ops/s 230.7932 Ops/s $\color{#35bf28}+0.27\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.7871ms 4.1526ms 240.8109 Ops/s 211.4914 Ops/s $\textbf{\color{#35bf28}+13.86\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 7.1744ms 4.3096ms 232.0421 Ops/s 225.7914 Ops/s $\color{#35bf28}+2.77\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.8447ms 4.4090ms 226.8090 Ops/s 195.3873 Ops/s $\textbf{\color{#35bf28}+16.08\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1259s 4.7211ms 211.8154 Ops/s 234.4206 Ops/s $\textbf{\color{#d91a1a}-9.64\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 9.7631ms 4.3546ms 229.6428 Ops/s 229.0877 Ops/s $\color{#35bf28}+0.24\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 17.3231ms 4.4718ms 223.6256 Ops/s 227.2338 Ops/s $\color{#d91a1a}-1.59\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.2710s 5.2869ms 189.1478 Ops/s 211.2700 Ops/s $\textbf{\color{#d91a1a}-10.47\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.6145ms 4.3431ms 230.2529 Ops/s 228.4942 Ops/s $\color{#35bf28}+0.77\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1614s 5.0380ms 198.4895 Ops/s 193.0276 Ops/s $\color{#35bf28}+2.83\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.5561ms 4.1372ms 241.7099 Ops/s 240.3636 Ops/s $\color{#35bf28}+0.56\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 16.0111ms 4.4419ms 225.1265 Ops/s 226.8657 Ops/s $\color{#d91a1a}-0.77\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 15.0041ms 4.4282ms 225.8270 Ops/s 224.6940 Ops/s $\color{#35bf28}+0.50\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2700s 37.6343ms 26.5715 Ops/s 25.6648 Ops/s $\color{#35bf28}+3.53\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1580s 32.6549ms 30.6233 Ops/s 30.0505 Ops/s $\color{#35bf28}+1.91\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1575s 32.8361ms 30.4543 Ops/s 30.4999 Ops/s $\color{#d91a1a}-0.15\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1570s 32.3776ms 30.8855 Ops/s 30.4267 Ops/s $\color{#35bf28}+1.51\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1557s 32.3647ms 30.8978 Ops/s 30.7974 Ops/s $\color{#35bf28}+0.33\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1634s 36.1664ms 27.6499 Ops/s 28.2396 Ops/s $\color{#d91a1a}-2.09\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1573s 32.4726ms 30.7952 Ops/s 30.4263 Ops/s $\color{#35bf28}+1.21\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1612s 33.1865ms 30.1327 Ops/s 30.5243 Ops/s $\color{#d91a1a}-1.28\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1611s 33.2682ms 30.0587 Ops/s 30.2212 Ops/s $\color{#d91a1a}-0.54\%$

@github-actions
Copy link

github-actions bot commented Jul 4, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}27$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.2574s 0.2418s 4.1363 Ops/s 4.2511 Ops/s $\color{#d91a1a}-2.70\%$
test_sync 0.1416s 0.1287s 7.7725 Ops/s 7.7766 Ops/s $\color{#d91a1a}-0.05\%$
test_async 0.2291s 0.1268s 7.8871 Ops/s 8.2764 Ops/s $\color{#d91a1a}-4.70\%$
test_simple 1.0707s 0.9935s 1.0066 Ops/s 1.0019 Ops/s $\color{#35bf28}+0.46\%$
test_transformed 2.5256s 2.4118s 0.4146 Ops/s 0.4182 Ops/s $\color{#d91a1a}-0.85\%$
test_serial 3.0142s 2.9875s 0.3347 Ops/s 0.3349 Ops/s $\color{#d91a1a}-0.05\%$
test_parallel 2.4527s 2.3372s 0.4279 Ops/s 0.4273 Ops/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-True-True-True] 1.2787ms 37.5679μs 26.6184 KOps/s 23.6119 KOps/s $\textbf{\color{#35bf28}+12.73\%}$
test_step_mdp_speed[True-True-True-True-False] 1.7387ms 22.1067μs 45.2352 KOps/s 42.5916 KOps/s $\textbf{\color{#35bf28}+6.21\%}$
test_step_mdp_speed[True-True-True-False-True] 1.5890ms 29.1568μs 34.2973 KOps/s 30.8483 KOps/s $\textbf{\color{#35bf28}+11.18\%}$
test_step_mdp_speed[True-True-True-False-False] 2.0724ms 17.2510μs 57.9675 KOps/s 52.4259 KOps/s $\textbf{\color{#35bf28}+10.57\%}$
test_step_mdp_speed[True-True-False-True-True] 7.4534ms 42.8451μs 23.3399 KOps/s 23.0696 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-True-False-True-False] 0.7707ms 22.9785μs 43.5189 KOps/s 38.4176 KOps/s $\textbf{\color{#35bf28}+13.28\%}$
test_step_mdp_speed[True-True-False-False-True] 6.5987ms 32.2855μs 30.9737 KOps/s 28.9448 KOps/s $\textbf{\color{#35bf28}+7.01\%}$
test_step_mdp_speed[True-True-False-False-False] 2.1499ms 19.4055μs 51.5317 KOps/s 50.0316 KOps/s $\color{#35bf28}+3.00\%$
test_step_mdp_speed[True-False-True-True-True] 4.0227ms 44.9717μs 22.2362 KOps/s 22.4706 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-False-True-True-False] 1.8238ms 26.2503μs 38.0949 KOps/s 36.3305 KOps/s $\color{#35bf28}+4.86\%$
test_step_mdp_speed[True-False-True-False-True] 3.2417ms 31.7013μs 31.5444 KOps/s 28.4351 KOps/s $\textbf{\color{#35bf28}+10.93\%}$
test_step_mdp_speed[True-False-True-False-False] 1.1938ms 18.3900μs 54.3775 KOps/s 49.7590 KOps/s $\textbf{\color{#35bf28}+9.28\%}$
test_step_mdp_speed[True-False-False-True-True] 2.4650ms 42.8942μs 23.3132 KOps/s 22.0286 KOps/s $\textbf{\color{#35bf28}+5.83\%}$
test_step_mdp_speed[True-False-False-True-False] 1.1762ms 24.7449μs 40.4124 KOps/s 36.0898 KOps/s $\textbf{\color{#35bf28}+11.98\%}$
test_step_mdp_speed[True-False-False-False-True] 8.8926ms 34.8289μs 28.7118 KOps/s 26.7937 KOps/s $\textbf{\color{#35bf28}+7.16\%}$
test_step_mdp_speed[True-False-False-False-False] 1.4035ms 21.1128μs 47.3646 KOps/s 48.7653 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-True-True-True-True] 1.6543ms 42.9474μs 23.2843 KOps/s 22.2884 KOps/s $\color{#35bf28}+4.47\%$
test_step_mdp_speed[False-True-True-True-False] 3.3098ms 25.4025μs 39.3663 KOps/s 38.9221 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-True-True-False-True] 1.8347ms 36.2555μs 27.5820 KOps/s 26.9820 KOps/s $\color{#35bf28}+2.22\%$
test_step_mdp_speed[False-True-True-False-False] 0.4921ms 19.9727μs 50.0683 KOps/s 44.6756 KOps/s $\textbf{\color{#35bf28}+12.07\%}$
test_step_mdp_speed[False-True-False-True-True] 7.3824ms 42.8305μs 23.3479 KOps/s 21.4320 KOps/s $\textbf{\color{#35bf28}+8.94\%}$
test_step_mdp_speed[False-True-False-True-False] 1.7771ms 26.0608μs 38.3719 KOps/s 36.6220 KOps/s $\color{#35bf28}+4.78\%$
test_step_mdp_speed[False-True-False-False-True] 0.7585ms 35.8174μs 27.9194 KOps/s 25.6523 KOps/s $\textbf{\color{#35bf28}+8.84\%}$
test_step_mdp_speed[False-True-False-False-False] 1.6033ms 21.7963μs 45.8793 KOps/s 42.9931 KOps/s $\textbf{\color{#35bf28}+6.71\%}$
test_step_mdp_speed[False-False-True-True-True] 0.6899ms 41.8793μs 23.8781 KOps/s 22.5317 KOps/s $\textbf{\color{#35bf28}+5.98\%}$
test_step_mdp_speed[False-False-True-True-False] 1.7172ms 27.3772μs 36.5267 KOps/s 34.1640 KOps/s $\textbf{\color{#35bf28}+6.92\%}$
test_step_mdp_speed[False-False-True-False-True] 1.2094ms 42.5114μs 23.5231 KOps/s 18.7036 KOps/s $\textbf{\color{#35bf28}+25.77\%}$
test_step_mdp_speed[False-False-True-False-False] 2.7413ms 23.0661μs 43.3536 KOps/s 44.0020 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[False-False-False-True-True] 5.3307ms 44.2309μs 22.6086 KOps/s 20.8094 KOps/s $\textbf{\color{#35bf28}+8.65\%}$
test_step_mdp_speed[False-False-False-True-False] 3.0570ms 28.0673μs 35.6287 KOps/s 34.0122 KOps/s $\color{#35bf28}+4.75\%$
test_step_mdp_speed[False-False-False-False-True] 2.0459ms 38.9451μs 25.6771 KOps/s 25.2710 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-False-False-False-False] 1.8219ms 22.8443μs 43.7745 KOps/s 40.5297 KOps/s $\textbf{\color{#35bf28}+8.01\%}$
test_values[generalized_advantage_estimate-True-True] 22.4669ms 18.9895ms 52.6607 Ops/s 53.6554 Ops/s $\color{#d91a1a}-1.85\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1005s 78.8129ms 12.6883 Ops/s 12.6437 Ops/s $\color{#35bf28}+0.35\%$
test_values[td0_return_estimate-False-False] 1.2956ms 0.3841ms 2.6038 KOps/s 2.5097 KOps/s $\color{#35bf28}+3.75\%$
test_values[td1_return_estimate-False-False] 24.0595ms 18.7820ms 53.2424 Ops/s 55.2837 Ops/s $\color{#d91a1a}-3.69\%$
test_values[vec_td1_return_estimate-False-False] 90.8920ms 78.7026ms 12.7061 Ops/s 12.9002 Ops/s $\color{#d91a1a}-1.51\%$
test_values[td_lambda_return_estimate-True-False] 62.1688ms 55.4289ms 18.0411 Ops/s 19.6880 Ops/s $\textbf{\color{#d91a1a}-8.36\%}$
test_values[vec_td_lambda_return_estimate-True-False] 91.1054ms 77.9640ms 12.8264 Ops/s 12.7810 Ops/s $\color{#35bf28}+0.36\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 19.9264ms 14.8577ms 67.3051 Ops/s 58.9591 Ops/s $\textbf{\color{#35bf28}+14.16\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 8.3420ms 4.6481ms 215.1410 Ops/s 190.4201 Ops/s $\textbf{\color{#35bf28}+12.98\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 5.2853ms 0.8385ms 1.1926 KOps/s 1.1579 KOps/s $\color{#35bf28}+2.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 92.4895ms 80.5018ms 12.4221 Ops/s 12.5407 Ops/s $\color{#d91a1a}-0.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 11.6574ms 6.5826ms 151.9157 Ops/s 157.1323 Ops/s $\color{#d91a1a}-3.32\%$
test_dqn_speed 7.1751ms 2.6797ms 373.1693 Ops/s 369.9668 Ops/s $\color{#35bf28}+0.87\%$
test_ddpg_speed 9.7314ms 4.9914ms 200.3441 Ops/s 172.0087 Ops/s $\textbf{\color{#35bf28}+16.47\%}$
test_sac_speed 20.3599ms 14.4277ms 69.3113 Ops/s 71.7549 Ops/s $\color{#d91a1a}-3.41\%$
test_redq_speed 0.1388s 29.0757ms 34.3929 Ops/s 37.7906 Ops/s $\textbf{\color{#d91a1a}-8.99\%}$
test_redq_deprec_speed 40.9450ms 24.2959ms 41.1592 Ops/s 41.5771 Ops/s $\color{#d91a1a}-1.01\%$
test_td3_speed 30.4345ms 19.2608ms 51.9188 Ops/s 54.3460 Ops/s $\color{#d91a1a}-4.47\%$
test_cql_speed 80.5293ms 61.4717ms 16.2676 Ops/s 16.8533 Ops/s $\color{#d91a1a}-3.48\%$
test_a2c_speed 15.7372ms 12.1942ms 82.0062 Ops/s 79.8955 Ops/s $\color{#35bf28}+2.64\%$
test_ppo_speed 17.2664ms 12.7979ms 78.1376 Ops/s 77.1821 Ops/s $\color{#35bf28}+1.24\%$
test_reinforce_speed 16.8957ms 10.1296ms 98.7203 Ops/s 93.8147 Ops/s $\textbf{\color{#35bf28}+5.23\%}$
test_iql_speed 63.7583ms 54.2423ms 18.4358 Ops/s 19.4647 Ops/s $\textbf{\color{#d91a1a}-5.29\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 10.7335ms 5.0215ms 199.1439 Ops/s 203.1637 Ops/s $\color{#d91a1a}-1.98\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 10.3406ms 5.2424ms 190.7521 Ops/s 189.2220 Ops/s $\color{#35bf28}+0.81\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 9.1143ms 5.2162ms 191.7120 Ops/s 160.1934 Ops/s $\textbf{\color{#35bf28}+19.68\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 9.0999ms 5.0457ms 198.1884 Ops/s 205.1416 Ops/s $\color{#d91a1a}-3.39\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 10.0712ms 5.1007ms 196.0530 Ops/s 195.1896 Ops/s $\color{#35bf28}+0.44\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 9.8145ms 5.3039ms 188.5391 Ops/s 202.0702 Ops/s $\textbf{\color{#d91a1a}-6.70\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 11.4766ms 5.6547ms 176.8446 Ops/s 195.7298 Ops/s $\textbf{\color{#d91a1a}-9.65\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 8.2354ms 4.9199ms 203.2544 Ops/s 195.3919 Ops/s $\color{#35bf28}+4.02\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.5940ms 5.1262ms 195.0759 Ops/s 192.0081 Ops/s $\color{#35bf28}+1.60\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 10.6782ms 4.8058ms 208.0816 Ops/s 197.9029 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 9.7784ms 5.0883ms 196.5290 Ops/s 204.1183 Ops/s $\color{#d91a1a}-3.72\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 13.2635ms 5.0450ms 198.2171 Ops/s 198.6307 Ops/s $\color{#d91a1a}-0.21\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 8.1067ms 4.8685ms 205.4013 Ops/s 216.6378 Ops/s $\textbf{\color{#d91a1a}-5.19\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.2443s 6.4630ms 154.7278 Ops/s 206.7602 Ops/s $\textbf{\color{#d91a1a}-25.17\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.4429ms 5.0334ms 198.6738 Ops/s 210.9425 Ops/s $\textbf{\color{#d91a1a}-5.82\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.3923s 6.6909ms 149.4563 Ops/s 191.6920 Ops/s $\textbf{\color{#d91a1a}-22.03\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 8.2566ms 5.3385ms 187.3202 Ops/s 212.6914 Ops/s $\textbf{\color{#d91a1a}-11.93\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 11.2134ms 5.4604ms 183.1371 Ops/s 199.1402 Ops/s $\textbf{\color{#d91a1a}-8.04\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3789s 50.8679ms 19.6588 Ops/s 21.3071 Ops/s $\textbf{\color{#d91a1a}-7.74\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.2192s 42.9390ms 23.2888 Ops/s 24.1395 Ops/s $\color{#d91a1a}-3.52\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.2178s 42.3720ms 23.6005 Ops/s 23.9136 Ops/s $\color{#d91a1a}-1.31\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2045s 41.1300ms 24.3132 Ops/s 24.0296 Ops/s $\color{#35bf28}+1.18\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.2073s 41.6829ms 23.9906 Ops/s 23.4449 Ops/s $\color{#35bf28}+2.33\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.2025s 44.8280ms 22.3075 Ops/s 21.0939 Ops/s $\textbf{\color{#35bf28}+5.75\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1959s 40.1481ms 24.9078 Ops/s 23.9755 Ops/s $\color{#35bf28}+3.89\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.2149s 41.6360ms 24.0177 Ops/s 23.2035 Ops/s $\color{#35bf28}+3.51\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.2104s 41.8427ms 23.8990 Ops/s 23.6621 Ops/s $\color{#35bf28}+1.00\%$

@vmoens vmoens added the Refactoring Refactoring of an existing feature label Jul 4, 2023
matteobettini
matteobettini approved these changes Jul 4, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

torchrl/modules/tensordict_module/common.py Outdated
# unravel_key_list(self.out_keys) can be removed once 473 is merged in tensordict
spec_keys = sorted(unravel_key_list(list(spec.keys(True, True))), key=str)
out_keys = sorted(unravel_key_list(self.out_keys), key=str)
if set(spec_keys) != set(out_keys):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can anticipate the call to set() in line 229 and 230, like u do in probabilistic.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean exactly? Trying to solve it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in probabilistic.py (linked below), there is the same code but you do set(unravel_key_list(list(spec.keys(True, True)))). I would uniform them

matteobettini
matteobettini reviewed Jul 4, 2023
View reviewed changes
torchrl/modules/tensordict_module/probabilistic.py

if set(spec.keys(True, True)) != set(self.out_keys):
spec_keys = set(unravel_key_list(list(spec.keys(True, True))))
out_keys = set(unravel_key_list(self.out_keys))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like here

@vmoens vmoens merged commit 75a45be into main Jul 4, 2023
@vmoens vmoens deleted the fix_unravel branch July 4, 2023 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Refactoring Refactoring of an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot