Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI, Tests] Fix windows tests #2337

Merged
merged 2 commits into from
Jul 30, 2024
Merged

[CI, Tests] Fix windows tests #2337

merged 2 commits into from
Jul 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 30, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2337

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b2b2762 with merge base 371181c (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 30, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 30, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.1687ms 59.4524ms 16.8202 Ops/s 17.3130 Ops/s $\color{#d91a1a}-2.85\%$
test_sync 39.7371ms 33.0049ms 30.2985 Ops/s 31.5434 Ops/s $\color{#d91a1a}-3.95\%$
test_async 60.2316ms 30.9581ms 32.3018 Ops/s 32.7494 Ops/s $\color{#d91a1a}-1.37\%$
test_simple 0.5156s 0.4412s 2.2668 Ops/s 2.4804 Ops/s $\textbf{\color{#d91a1a}-8.61\%}$
test_transformed 0.6514s 0.5836s 1.7134 Ops/s 1.7900 Ops/s $\color{#d91a1a}-4.28\%$
test_serial 1.3600s 1.2866s 0.7773 Ops/s 0.7858 Ops/s $\color{#d91a1a}-1.08\%$
test_parallel 1.1879s 1.1112s 0.8999 Ops/s 0.8992 Ops/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-True-True-True-True] 0.2738ms 25.0019μs 39.9969 KOps/s 41.4800 KOps/s $\color{#d91a1a}-3.58\%$
test_step_mdp_speed[True-True-True-True-False] 67.4550μs 14.4019μs 69.4353 KOps/s 70.3896 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-True-True-False-True] 50.7760μs 14.2552μs 70.1497 KOps/s 71.4803 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-True-True-False-False] 47.5790μs 8.3073μs 120.3766 KOps/s 121.8837 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[True-True-False-True-True] 70.4520μs 26.5595μs 37.6513 KOps/s 38.3354 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-True-False-True-False] 76.9140μs 15.9515μs 62.6900 KOps/s 63.8466 KOps/s $\color{#d91a1a}-1.81\%$
test_step_mdp_speed[True-True-False-False-True] 61.3050μs 15.8284μs 63.1775 KOps/s 65.0708 KOps/s $\color{#d91a1a}-2.91\%$
test_step_mdp_speed[True-True-False-False-False] 64.0000μs 9.7906μs 102.1385 KOps/s 102.7168 KOps/s $\color{#d91a1a}-0.56\%$
test_step_mdp_speed[True-False-True-True-True] 60.5530μs 28.2414μs 35.4090 KOps/s 36.4538 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[True-False-True-True-False] 63.7100μs 17.5975μs 56.8262 KOps/s 58.3145 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[True-False-True-False-True] 78.5690μs 15.8000μs 63.2913 KOps/s 64.8053 KOps/s $\color{#d91a1a}-2.34\%$
test_step_mdp_speed[True-False-True-False-False] 55.7730μs 9.8238μs 101.7933 KOps/s 102.9282 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[True-False-False-True-True] 63.4090μs 29.5658μs 33.8229 KOps/s 34.9919 KOps/s $\color{#d91a1a}-3.34\%$
test_step_mdp_speed[True-False-False-True-False] 77.2850μs 19.0670μs 52.4466 KOps/s 53.6121 KOps/s $\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-False-False-False-True] 76.0730μs 17.1115μs 58.4402 KOps/s 59.5497 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-False-False-False-False] 41.2180μs 11.3120μs 88.4019 KOps/s 90.2860 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[False-True-True-True-True] 95.5690μs 28.0181μs 35.6912 KOps/s 36.6972 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-True-True-True-False] 49.2330μs 17.6176μs 56.7615 KOps/s 58.2250 KOps/s $\color{#d91a1a}-2.51\%$
test_step_mdp_speed[False-True-True-False-True] 73.5180μs 18.2412μs 54.8211 KOps/s 55.4114 KOps/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[False-True-True-False-False] 35.4560μs 11.1444μs 89.7310 KOps/s 90.7820 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[False-True-False-True-True] 90.8000μs 29.5221μs 33.8729 KOps/s 34.9193 KOps/s $\color{#d91a1a}-3.00\%$
test_step_mdp_speed[False-True-False-True-False] 55.8150μs 18.9880μs 52.6648 KOps/s 53.7887 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[False-True-False-False-True] 76.1730μs 19.8559μs 50.3629 KOps/s 51.8202 KOps/s $\color{#d91a1a}-2.81\%$
test_step_mdp_speed[False-True-False-False-False] 62.0870μs 12.4895μs 80.0675 KOps/s 81.2162 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[False-False-True-True-True] 3.5527ms 31.0907μs 32.1640 KOps/s 32.9517 KOps/s $\color{#d91a1a}-2.39\%$
test_step_mdp_speed[False-False-True-True-False] 74.2690μs 20.7077μs 48.2912 KOps/s 49.4414 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-False-True-False-True] 72.2460μs 19.6856μs 50.7986 KOps/s 51.0890 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-False-True-False-False] 77.9360μs 12.5706μs 79.5504 KOps/s 80.8897 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-False-False-True-True] 89.4980μs 32.4213μs 30.8439 KOps/s 31.5951 KOps/s $\color{#d91a1a}-2.38\%$
test_step_mdp_speed[False-False-False-True-False] 54.8630μs 21.8500μs 45.7666 KOps/s 46.4425 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[False-False-False-False-True] 80.6010μs 20.7897μs 48.1008 KOps/s 49.0652 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-False-False-False-False] 38.7320μs 13.9991μs 71.4329 KOps/s 73.4102 KOps/s $\color{#d91a1a}-2.69\%$
test_values[generalized_advantage_estimate-True-True] 10.7003ms 9.8427ms 101.5982 Ops/s 104.1863 Ops/s $\color{#d91a1a}-2.48\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.1698ms 33.9377ms 29.4657 Ops/s 27.5190 Ops/s $\textbf{\color{#35bf28}+7.07\%}$
test_values[td0_return_estimate-False-False] 0.2435ms 0.1828ms 5.4716 KOps/s 5.7118 KOps/s $\color{#d91a1a}-4.21\%$
test_values[td1_return_estimate-False-False] 28.6193ms 24.7175ms 40.4572 Ops/s 42.5454 Ops/s $\color{#d91a1a}-4.91\%$
test_values[vec_td1_return_estimate-False-False] 39.5075ms 34.1462ms 29.2858 Ops/s 27.1257 Ops/s $\textbf{\color{#35bf28}+7.96\%}$
test_values[td_lambda_return_estimate-True-False] 36.4076ms 34.8992ms 28.6539 Ops/s 29.4130 Ops/s $\color{#d91a1a}-2.58\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.8373ms 33.8565ms 29.5364 Ops/s 27.3812 Ops/s $\textbf{\color{#35bf28}+7.87\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5248ms 8.4109ms 118.8940 Ops/s 121.7256 Ops/s $\color{#d91a1a}-2.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4725ms 2.0339ms 491.6778 Ops/s 558.2787 Ops/s $\textbf{\color{#d91a1a}-11.93\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5298ms 0.3635ms 2.7507 KOps/s 2.8254 KOps/s $\color{#d91a1a}-2.64\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.9340ms 42.6579ms 23.4423 Ops/s 21.2460 Ops/s $\textbf{\color{#35bf28}+10.34\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.9081ms 3.0565ms 327.1673 Ops/s 327.4402 Ops/s $\color{#d91a1a}-0.08\%$
test_dqn_speed 6.6604ms 1.3326ms 750.3950 Ops/s 775.2058 Ops/s $\color{#d91a1a}-3.20\%$
test_ddpg_speed 3.4343ms 2.7703ms 360.9745 Ops/s 359.7469 Ops/s $\color{#35bf28}+0.34\%$
test_sac_speed 9.8642ms 8.2654ms 120.9867 Ops/s 117.1725 Ops/s $\color{#35bf28}+3.26\%$
test_redq_speed 14.1984ms 13.1993ms 75.7617 Ops/s 75.2873 Ops/s $\color{#35bf28}+0.63\%$
test_redq_deprec_speed 15.3305ms 13.6877ms 73.0583 Ops/s 73.8512 Ops/s $\color{#d91a1a}-1.07\%$
test_td3_speed 8.6648ms 8.2017ms 121.9266 Ops/s 124.5308 Ops/s $\color{#d91a1a}-2.09\%$
test_cql_speed 38.5183ms 36.4790ms 27.4130 Ops/s 28.0237 Ops/s $\color{#d91a1a}-2.18\%$
test_a2c_speed 11.2118ms 7.7150ms 129.6184 Ops/s 133.6406 Ops/s $\color{#d91a1a}-3.01\%$
test_ppo_speed 9.4585ms 7.8031ms 128.1542 Ops/s 130.0030 Ops/s $\color{#d91a1a}-1.42\%$
test_reinforce_speed 7.4534ms 6.5553ms 152.5481 Ops/s 152.1078 Ops/s $\color{#35bf28}+0.29\%$
test_iql_speed 33.9712ms 32.2780ms 30.9808 Ops/s 30.8305 Ops/s $\color{#35bf28}+0.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8592ms 4.8927ms 204.3842 Ops/s 198.1684 Ops/s $\color{#35bf28}+3.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5903ms 0.4767ms 2.0979 KOps/s 2.0982 KOps/s $\color{#d91a1a}-0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7882ms 0.4536ms 2.2044 KOps/s 2.1871 KOps/s $\color{#35bf28}+0.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2797ms 4.8492ms 206.2178 Ops/s 197.0999 Ops/s $\color{#35bf28}+4.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3749ms 0.4782ms 2.0911 KOps/s 2.0585 KOps/s $\color{#35bf28}+1.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6310ms 0.4522ms 2.2116 KOps/s 2.2137 KOps/s $\color{#d91a1a}-0.10\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4749ms 1.7098ms 584.8527 Ops/s 596.9282 Ops/s $\color{#d91a1a}-2.02\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3069ms 1.6149ms 619.2431 Ops/s 625.6061 Ops/s $\color{#d91a1a}-1.02\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.8323ms 5.1086ms 195.7500 Ops/s 193.9241 Ops/s $\color{#35bf28}+0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9396ms 0.6218ms 1.6083 KOps/s 1.6049 KOps/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9934ms 0.5908ms 1.6926 KOps/s 1.6837 KOps/s $\color{#35bf28}+0.53\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2572ms 4.9575ms 201.7132 Ops/s 197.3842 Ops/s $\color{#35bf28}+2.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7866ms 0.4769ms 2.0969 KOps/s 2.0747 KOps/s $\color{#35bf28}+1.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7908ms 0.4604ms 2.1720 KOps/s 2.1582 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.7312ms 4.9736ms 201.0635 Ops/s 197.6784 Ops/s $\color{#35bf28}+1.71\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6568ms 0.4756ms 2.1026 KOps/s 2.0892 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7862ms 0.4490ms 2.2274 KOps/s 2.1488 KOps/s $\color{#35bf28}+3.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5463ms 5.1759ms 193.2049 Ops/s 192.2118 Ops/s $\color{#35bf28}+0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0292ms 0.6334ms 1.5788 KOps/s 1.5872 KOps/s $\color{#d91a1a}-0.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.9254ms 0.5992ms 1.6688 KOps/s 1.6942 KOps/s $\color{#d91a1a}-1.50\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1308s 8.6695ms 115.3467 Ops/s 157.9171 Ops/s $\textbf{\color{#d91a1a}-26.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3899ms 13.0559ms 76.5940 Ops/s 77.6081 Ops/s $\color{#d91a1a}-1.31\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6598ms 1.0820ms 924.2193 Ops/s 882.3226 Ops/s $\color{#35bf28}+4.75\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1194s 6.1334ms 163.0415 Ops/s 162.5632 Ops/s $\color{#35bf28}+0.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.5487ms 13.2157ms 75.6677 Ops/s 63.1140 Ops/s $\textbf{\color{#35bf28}+19.89\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.6526ms 1.1156ms 896.3969 Ops/s 888.8181 Ops/s $\color{#35bf28}+0.85\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1266s 6.3798ms 156.7435 Ops/s 151.4708 Ops/s $\color{#35bf28}+3.48\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 20.4824ms 13.5561ms 73.7677 Ops/s 72.7309 Ops/s $\color{#35bf28}+1.43\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.2017ms 1.3607ms 734.9196 Ops/s 534.2676 Ops/s $\textbf{\color{#35bf28}+37.56\%}$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 30, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1070s 0.1068s 9.3617 Ops/s 8.5673 Ops/s $\textbf{\color{#35bf28}+9.27\%}$
test_sync 96.1735ms 95.4971ms 10.4715 Ops/s 10.4807 Ops/s $\color{#d91a1a}-0.09\%$
test_async 0.1734s 89.3301ms 11.1944 Ops/s 11.0283 Ops/s $\color{#35bf28}+1.51\%$
test_single_pixels 0.1172s 0.1169s 8.5525 Ops/s 8.4773 Ops/s $\color{#35bf28}+0.89\%$
test_sync_pixels 78.4425ms 74.9876ms 13.3355 Ops/s 13.4208 Ops/s $\color{#d91a1a}-0.64\%$
test_async_pixels 0.1375s 68.9499ms 14.5033 Ops/s 14.3909 Ops/s $\color{#35bf28}+0.78\%$
test_simple 0.7883s 0.7783s 1.2849 Ops/s 1.2567 Ops/s $\color{#35bf28}+2.25\%$
test_transformed 1.0906s 1.0200s 0.9804 Ops/s 0.9894 Ops/s $\color{#d91a1a}-0.91\%$
test_serial 2.2966s 2.2289s 0.4487 Ops/s 0.4484 Ops/s $\color{#35bf28}+0.07\%$
test_parallel 2.0078s 1.9509s 0.5126 Ops/s 0.5056 Ops/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[True-True-True-True-True] 87.1220μs 36.2720μs 27.5695 KOps/s 28.1373 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-True-True-True-False] 61.7710μs 21.0402μs 47.5280 KOps/s 48.7671 KOps/s $\color{#d91a1a}-2.54\%$
test_step_mdp_speed[True-True-True-False-True] 89.7120μs 20.6041μs 48.5340 KOps/s 49.7281 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[True-True-True-False-False] 30.5510μs 11.9264μs 83.8478 KOps/s 86.3307 KOps/s $\color{#d91a1a}-2.88\%$
test_step_mdp_speed[True-True-False-True-True] 0.2278ms 38.7370μs 25.8151 KOps/s 26.4269 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[True-True-False-True-False] 48.5010μs 23.0531μs 43.3781 KOps/s 43.7034 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-True-False-False-True] 0.2129ms 22.7856μs 43.8874 KOps/s 45.0508 KOps/s $\color{#d91a1a}-2.58\%$
test_step_mdp_speed[True-True-False-False-False] 31.4600μs 14.0746μs 71.0501 KOps/s 71.9320 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[True-False-True-True-True] 60.5610μs 39.6395μs 25.2274 KOps/s 24.9770 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-False-True-True-False] 49.5510μs 25.5330μs 39.1649 KOps/s 39.8540 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[True-False-True-False-True] 47.2710μs 22.8311μs 43.7999 KOps/s 45.1831 KOps/s $\color{#d91a1a}-3.06\%$
test_step_mdp_speed[True-False-True-False-False] 43.3510μs 14.1617μs 70.6131 KOps/s 71.8109 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-False-False-True-True] 80.3420μs 42.9308μs 23.2933 KOps/s 23.7385 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[True-False-False-True-False] 0.1193ms 27.3786μs 36.5248 KOps/s 36.9001 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-False-False-False-True] 42.0210μs 24.4518μs 40.8968 KOps/s 40.9059 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-False-False-False] 39.4310μs 16.2752μs 61.4430 KOps/s 62.9538 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-True-True-True] 0.1358ms 41.5345μs 24.0764 KOps/s 24.6167 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[False-True-True-True-False] 62.8520μs 25.2420μs 39.6165 KOps/s 40.3733 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-True-True-False-True] 48.1220μs 27.2223μs 36.7346 KOps/s 37.1207 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-True-True-False-False] 36.7610μs 15.8324μs 63.1617 KOps/s 62.4165 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-True-False-True-True] 0.2117ms 42.3428μs 23.6168 KOps/s 23.6481 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-False-True-False] 47.6310μs 27.5255μs 36.3299 KOps/s 36.8690 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[False-True-False-False-True] 0.1929ms 29.5834μs 33.8027 KOps/s 34.4900 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-True-False-False-False] 37.0110μs 18.0907μs 55.2769 KOps/s 55.6274 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-True-True-True] 4.0961ms 45.6368μs 21.9121 KOps/s 22.0922 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-False-True-True-False] 99.9730μs 29.6592μs 33.7164 KOps/s 34.0158 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-False-True-False-True] 45.1610μs 29.3133μs 34.1142 KOps/s 34.1012 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-False-True-False-False] 50.5310μs 17.8355μs 56.0679 KOps/s 54.9718 KOps/s $\color{#35bf28}+1.99\%$
test_step_mdp_speed[False-False-False-True-True] 67.4520μs 46.2521μs 21.6206 KOps/s 21.6916 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-False-False-True-False] 56.8610μs 31.8291μs 31.4178 KOps/s 31.7765 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[False-False-False-False-True] 50.6210μs 31.2543μs 31.9956 KOps/s 32.6547 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-False-False-False-False] 37.0010μs 20.3093μs 49.2386 KOps/s 49.7211 KOps/s $\color{#d91a1a}-0.97\%$
test_values[generalized_advantage_estimate-True-True] 24.4738ms 24.0353ms 41.6055 Ops/s 42.0031 Ops/s $\color{#d91a1a}-0.95\%$
test_values[vec_generalized_advantage_estimate-True-True] 89.6312ms 2.6883ms 371.9761 Ops/s 371.4564 Ops/s $\color{#35bf28}+0.14\%$
test_values[td0_return_estimate-False-False] 89.6220μs 65.4617μs 15.2761 KOps/s 15.5092 KOps/s $\color{#d91a1a}-1.50\%$
test_values[td1_return_estimate-False-False] 56.5139ms 54.1536ms 18.4660 Ops/s 18.4253 Ops/s $\color{#35bf28}+0.22\%$
test_values[vec_td1_return_estimate-False-False] 1.3936ms 1.0811ms 924.9683 Ops/s 930.4196 Ops/s $\color{#d91a1a}-0.59\%$
test_values[td_lambda_return_estimate-True-False] 90.2778ms 87.1547ms 11.4739 Ops/s 11.6194 Ops/s $\color{#d91a1a}-1.25\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4633ms 1.0842ms 922.3313 Ops/s 933.3881 Ops/s $\color{#d91a1a}-1.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0422ms 23.8502ms 41.9284 Ops/s 42.1581 Ops/s $\color{#d91a1a}-0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9325ms 0.7159ms 1.3969 KOps/s 1.3848 KOps/s $\color{#35bf28}+0.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8157ms 0.6752ms 1.4811 KOps/s 1.4648 KOps/s $\color{#35bf28}+1.11\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5702ms 1.4657ms 682.2769 Ops/s 681.4723 Ops/s $\color{#35bf28}+0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8725ms 0.6811ms 1.4681 KOps/s 1.4588 KOps/s $\color{#35bf28}+0.64\%$
test_dqn_speed 7.4370ms 1.3278ms 753.1042 Ops/s 741.7400 Ops/s $\color{#35bf28}+1.53\%$
test_ddpg_speed 2.9236ms 2.6799ms 373.1516 Ops/s 366.7260 Ops/s $\color{#35bf28}+1.75\%$
test_sac_speed 8.5183ms 7.7423ms 129.1607 Ops/s 130.0079 Ops/s $\color{#d91a1a}-0.65\%$
test_redq_speed 11.8374ms 10.0048ms 99.9515 Ops/s 99.4906 Ops/s $\color{#35bf28}+0.46\%$
test_redq_deprec_speed 11.3842ms 11.0542ms 90.4635 Ops/s 95.4664 Ops/s $\textbf{\color{#d91a1a}-5.24\%}$
test_td3_speed 7.7721ms 7.6335ms 131.0017 Ops/s 131.2303 Ops/s $\color{#d91a1a}-0.17\%$
test_cql_speed 25.8091ms 24.6222ms 40.6138 Ops/s 40.5626 Ops/s $\color{#35bf28}+0.13\%$
test_a2c_speed 5.6744ms 5.4064ms 184.9654 Ops/s 182.5428 Ops/s $\color{#35bf28}+1.33\%$
test_ppo_speed 5.9796ms 5.7295ms 174.5358 Ops/s 171.9077 Ops/s $\color{#35bf28}+1.53\%$
test_reinforce_speed 5.3981ms 4.4995ms 222.2489 Ops/s 228.2097 Ops/s $\color{#d91a1a}-2.61\%$
test_iql_speed 22.7323ms 19.3468ms 51.6883 Ops/s 51.9964 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8343ms 6.5245ms 153.2692 Ops/s 152.6450 Ops/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8612ms 0.5104ms 1.9591 KOps/s 1.9596 KOps/s $\color{#d91a1a}-0.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7057ms 0.4861ms 2.0571 KOps/s 2.0475 KOps/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8047ms 6.4653ms 154.6725 Ops/s 155.3625 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7092ms 0.5036ms 1.9857 KOps/s 1.6303 KOps/s $\textbf{\color{#35bf28}+21.80\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6994ms 0.4846ms 2.0634 KOps/s 2.0740 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2070ms 1.9317ms 517.6739 Ops/s 522.4677 Ops/s $\color{#d91a1a}-0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0382ms 1.8248ms 548.0110 Ops/s 549.1942 Ops/s $\color{#d91a1a}-0.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.8061ms 6.5915ms 151.7101 Ops/s 149.6350 Ops/s $\color{#35bf28}+1.39\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3113ms 0.6511ms 1.5359 KOps/s 1.5355 KOps/s $\color{#35bf28}+0.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8221ms 0.6289ms 1.5901 KOps/s 1.5925 KOps/s $\color{#d91a1a}-0.15\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7843ms 6.5685ms 152.2420 Ops/s 152.2240 Ops/s $\color{#35bf28}+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5391ms 0.5114ms 1.9553 KOps/s 1.4940 KOps/s $\textbf{\color{#35bf28}+30.88\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6892ms 0.4932ms 2.0277 KOps/s 1.9504 KOps/s $\color{#35bf28}+3.96\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7838ms 6.4709ms 154.5368 Ops/s 153.8852 Ops/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7332ms 0.5051ms 1.9800 KOps/s 1.9893 KOps/s $\color{#d91a1a}-0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3002ms 0.4909ms 2.0371 KOps/s 2.0505 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9164ms 6.7134ms 148.9547 Ops/s 150.3126 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2556ms 0.6603ms 1.5145 KOps/s 1.5250 KOps/s $\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8866ms 0.6375ms 1.5687 KOps/s 1.5775 KOps/s $\color{#d91a1a}-0.56\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1459s 7.9065ms 126.4789 Ops/s 127.1286 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.3291ms 15.8546ms 63.0731 Ops/s 62.9129 Ops/s $\color{#35bf28}+0.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.1945ms 1.3457ms 743.1152 Ops/s 814.6494 Ops/s $\textbf{\color{#d91a1a}-8.78\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1274s 7.5239ms 132.9090 Ops/s 132.4878 Ops/s $\color{#35bf28}+0.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6787ms 15.7182ms 63.6206 Ops/s 62.9582 Ops/s $\color{#35bf28}+1.05\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2080ms 1.2269ms 815.0881 Ops/s 811.9563 Ops/s $\color{#35bf28}+0.39\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1275s 10.1528ms 98.4951 Ops/s 129.4840 Ops/s $\textbf{\color{#d91a1a}-23.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.3042ms 15.8082ms 63.2585 Ops/s 62.8612 Ops/s $\color{#35bf28}+0.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5451ms 1.3855ms 721.7612 Ops/s 717.7379 Ops/s $\color{#35bf28}+0.56\%$

@vmoens vmoens added Tests Incomplete or broken unit tests CI Has to do with CI setup (e.g. wheels & builds, tests...) labels Jul 30, 2024
@vmoens vmoens merged commit b3f99b3 into main Jul 30, 2024
64 of 65 checks passed
@vmoens vmoens deleted the fix-windows-tests branch July 30, 2024 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Tests Incomplete or broken unit tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot