-
Notifications
You must be signed in to change notification settings - Fork 327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BugFix, Test] Fix flaky gym vecenvs tests #1727
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1727
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (3 Unrelated Failures)As of commit ce7425e with merge base 2bacb82 (): FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Dec 1, 2023
vmoens
added
bug
Something isn't working
Tests
Incomplete or broken unit tests
CI
Has to do with CI setup (e.g. wheels & builds, tests...)
and removed
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
labels
Dec 1, 2023
|
Name | Max | Mean | Ops | Ops on Repo HEAD
|
Change |
---|---|---|---|---|---|
test_single | 65.3896ms | 63.6198ms | 15.7184 Ops/s | 15.1268 Ops/s | |
test_sync | 39.5186ms | 34.2973ms | 29.1568 Ops/s | 27.8761 Ops/s | |
test_async | 61.3216ms | 32.9844ms | 30.3173 Ops/s | 30.3219 Ops/s | |
test_simple | 0.4845s | 0.4332s | 2.3086 Ops/s | 2.3253 Ops/s | |
test_transformed | 0.6368s | 0.5945s | 1.6821 Ops/s | 1.6550 Ops/s | |
test_serial | 1.3943s | 1.3421s | 0.7451 Ops/s | 0.7454 Ops/s | |
test_parallel | 1.3553s | 1.2896s | 0.7755 Ops/s | 0.7785 Ops/s | |
test_step_mdp_speed[True-True-True-True-True] | 0.1590ms | 22.9054μs | 43.6578 KOps/s | 43.2933 KOps/s | |
test_step_mdp_speed[True-True-True-True-False] | 33.7030μs | 13.8086μs | 72.4186 KOps/s | 72.5106 KOps/s | |
test_step_mdp_speed[True-True-True-False-True] | 53.9610μs | 14.0892μs | 70.9761 KOps/s | 71.6614 KOps/s | |
test_step_mdp_speed[True-True-True-False-False] | 33.5030μs | 8.3424μs | 119.8700 KOps/s | 119.7060 KOps/s | |
test_step_mdp_speed[True-True-False-True-True] | 52.9800μs | 24.3943μs | 40.9932 KOps/s | 41.4629 KOps/s | |
test_step_mdp_speed[True-True-False-True-False] | 39.1430μs | 15.3149μs | 65.2957 KOps/s | 67.2046 KOps/s | |
test_step_mdp_speed[True-True-False-False-True] | 42.0790μs | 15.3788μs | 65.0245 KOps/s | 65.0531 KOps/s | |
test_step_mdp_speed[True-True-False-False-False] | 33.5530μs | 9.7949μs | 102.0935 KOps/s | 104.2129 KOps/s | |
test_step_mdp_speed[True-False-True-True-True] | 71.5540μs | 26.1699μs | 38.2118 KOps/s | 38.4440 KOps/s | |
test_step_mdp_speed[True-False-True-True-False] | 40.9170μs | 16.5331μs | 60.4849 KOps/s | 60.0917 KOps/s | |
test_step_mdp_speed[True-False-True-False-True] | 44.4340μs | 15.4323μs | 64.7993 KOps/s | 65.7760 KOps/s | |
test_step_mdp_speed[True-False-True-False-False] | 35.2760μs | 9.7362μs | 102.7093 KOps/s | 105.3814 KOps/s | |
test_step_mdp_speed[True-False-False-True-True] | 51.7680μs | 27.1478μs | 36.8354 KOps/s | 37.4939 KOps/s | |
test_step_mdp_speed[True-False-False-True-False] | 65.5840μs | 17.9690μs | 55.6515 KOps/s | 56.9072 KOps/s | |
test_step_mdp_speed[True-False-False-False-True] | 42.8210μs | 16.4590μs | 60.7571 KOps/s | 61.3886 KOps/s | |
test_step_mdp_speed[True-False-False-False-False] | 49.2930μs | 11.0867μs | 90.1985 KOps/s | 92.8436 KOps/s | |
test_step_mdp_speed[False-True-True-True-True] | 53.6700μs | 25.9581μs | 38.5237 KOps/s | 38.7865 KOps/s | |
test_step_mdp_speed[False-True-True-True-False] | 47.0580μs | 16.3527μs | 61.1520 KOps/s | 60.7792 KOps/s | |
test_step_mdp_speed[False-True-True-False-True] | 41.9290μs | 17.4406μs | 57.3375 KOps/s | 57.2240 KOps/s | |
test_step_mdp_speed[False-True-True-False-False] | 43.6420μs | 10.8848μs | 91.8716 KOps/s | 92.3340 KOps/s | |
test_step_mdp_speed[False-True-False-True-True] | 59.2710μs | 26.9460μs | 37.1113 KOps/s | 37.4185 KOps/s | |
test_step_mdp_speed[False-True-False-True-False] | 52.7690μs | 17.7104μs | 56.4641 KOps/s | 56.8551 KOps/s | |
test_step_mdp_speed[False-True-False-False-True] | 47.8700μs | 18.7548μs | 53.3196 KOps/s | 53.2933 KOps/s | |
test_step_mdp_speed[False-True-False-False-False] | 41.8080μs | 12.1280μs | 82.4540 KOps/s | 83.4289 KOps/s | |
test_step_mdp_speed[False-False-True-True-True] | 60.1630μs | 28.4662μs | 35.1294 KOps/s | 35.4964 KOps/s | |
test_step_mdp_speed[False-False-True-True-False] | 0.1293ms | 19.0455μs | 52.5058 KOps/s | 52.3048 KOps/s | |
test_step_mdp_speed[False-False-True-False-True] | 51.4260μs | 18.7175μs | 53.4258 KOps/s | 53.2981 KOps/s | |
test_step_mdp_speed[False-False-True-False-False] | 35.1460μs | 12.0167μs | 83.2174 KOps/s | 82.3082 KOps/s | |
test_step_mdp_speed[False-False-False-True-True] | 96.2300μs | 29.1272μs | 34.3322 KOps/s | 34.3127 KOps/s | |
test_step_mdp_speed[False-False-False-True-False] | 45.6560μs | 20.0954μs | 49.7627 KOps/s | 50.2091 KOps/s | |
test_step_mdp_speed[False-False-False-False-True] | 55.2740μs | 19.5811μs | 51.0696 KOps/s | 51.2311 KOps/s | |
test_step_mdp_speed[False-False-False-False-False] | 51.8370μs | 13.1684μs | 75.9393 KOps/s | 77.3255 KOps/s | |
test_values[generalized_advantage_estimate-True-True] | 14.5786ms | 11.9582ms | 83.6244 Ops/s | 82.8402 Ops/s | |
test_values[vec_generalized_advantage_estimate-True-True] | 27.8271ms | 26.4634ms | 37.7881 Ops/s | 36.6280 Ops/s | |
test_values[td0_return_estimate-False-False] | 0.2550ms | 0.1779ms | 5.6223 KOps/s | 5.4920 KOps/s | |
test_values[td1_return_estimate-False-False] | 26.4177ms | 26.0314ms | 38.4152 Ops/s | 38.4247 Ops/s | |
test_values[vec_td1_return_estimate-False-False] | 37.8575ms | 27.1579ms | 36.8217 Ops/s | 37.4066 Ops/s | |
test_values[td_lambda_return_estimate-True-False] | 39.1043ms | 36.3872ms | 27.4822 Ops/s | 27.6983 Ops/s | |
test_values[vec_td_lambda_return_estimate-True-False] | 26.9707ms | 26.6732ms | 37.4909 Ops/s | 37.1897 Ops/s | |
test_gae_speed[generalized_advantage_estimate-False-1-512] | 8.1522ms | 8.0071ms | 124.8885 Ops/s | 123.6746 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 10.2764ms | 1.9929ms | 501.7923 Ops/s | 507.2493 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 7.5937ms | 0.4563ms | 2.1917 KOps/s | 2.2422 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 46.4884ms | 38.8844ms | 25.7173 Ops/s | 24.7194 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 10.5426ms | 2.5929ms | 385.6644 Ops/s | 392.0968 Ops/s | |
test_dqn_speed | 9.6325ms | 1.6286ms | 614.0296 Ops/s | 615.6502 Ops/s | |
test_ddpg_speed | 64.8815ms | 3.8223ms | 261.6245 Ops/s | 277.9591 Ops/s | |
test_sac_speed | 17.7106ms | 10.2099ms | 97.9441 Ops/s | 98.2369 Ops/s | |
test_redq_speed | 31.9191ms | 19.5919ms | 51.0414 Ops/s | 51.4205 Ops/s | |
test_redq_deprec_speed | 23.6187ms | 15.3224ms | 65.2638 Ops/s | 65.2829 Ops/s | |
test_td3_speed | 17.5828ms | 10.4800ms | 95.4200 Ops/s | 94.4166 Ops/s | |
test_cql_speed | 47.5621ms | 38.8042ms | 25.7704 Ops/s | 25.9658 Ops/s | |
test_a2c_speed | 16.0631ms | 8.1928ms | 122.0581 Ops/s | 122.3313 Ops/s | |
test_ppo_speed | 16.3462ms | 8.5322ms | 117.2027 Ops/s | 118.0058 Ops/s | |
test_reinforce_speed | 15.1376ms | 7.2910ms | 137.1560 Ops/s | 137.9686 Ops/s | |
test_iql_speed | 42.2276ms | 34.2823ms | 29.1696 Ops/s | 29.3533 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.8899ms | 1.9063ms | 524.5818 Ops/s | 529.6038 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.1315ms | 2.0346ms | 491.4956 Ops/s | 488.6864 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.8091ms | 2.0653ms | 484.1802 Ops/s | 490.8924 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 2.4125ms | 1.8979ms | 526.8921 Ops/s | 529.0243 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 2.7216ms | 2.0357ms | 491.2291 Ops/s | 498.5169 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.0706ms | 2.0390ms | 490.4298 Ops/s | 491.5684 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.4937ms | 1.8887ms | 529.4739 Ops/s | 531.0784 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.2797ms | 2.0420ms | 489.7163 Ops/s | 497.8884 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 2.9848ms | 2.0122ms | 496.9622 Ops/s | 495.1409 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 2.8857ms | 1.9181ms | 521.3483 Ops/s | 516.3640 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 2.9882ms | 2.0340ms | 491.6389 Ops/s | 491.9499 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 3.1131ms | 2.0313ms | 492.3037 Ops/s | 488.7198 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 2.2886ms | 1.8933ms | 528.1811 Ops/s | 523.6900 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.2643ms | 2.0443ms | 489.1706 Ops/s | 494.3904 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 2.8866ms | 2.0324ms | 492.0393 Ops/s | 495.7959 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 2.4218ms | 1.9117ms | 523.1057 Ops/s | 524.2613 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.3642ms | 2.0341ms | 491.6063 Ops/s | 493.7919 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.0877ms | 2.0753ms | 481.8612 Ops/s | 487.3067 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.1624s | 17.4161ms | 57.4181 Ops/s | 56.9979 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1036s | 16.1363ms | 61.9722 Ops/s | 61.1256 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 0.1022s | 16.1169ms | 62.0465 Ops/s | 61.2788 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1016s | 16.0102ms | 62.4603 Ops/s | 60.9453 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1006s | 17.6992ms | 56.4998 Ops/s | 55.0497 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 0.1028s | 16.0972ms | 62.1225 Ops/s | 61.5693 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1041s | 16.2016ms | 61.7224 Ops/s | 61.0717 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1028s | 16.1513ms | 61.9147 Ops/s | 60.5240 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 0.1030s | 16.0675ms | 62.2376 Ops/s | 61.4091 Ops/s |
|
Name | Max | Mean | Ops | Ops on Repo HEAD
|
Change |
---|---|---|---|---|---|
test_single | 0.1259s | 0.1246s | 8.0283 Ops/s | 8.3283 Ops/s | |
test_sync | 0.1056s | 0.1034s | 9.6736 Ops/s | 9.6038 Ops/s | |
test_async | 0.2676s | 99.7809ms | 10.0220 Ops/s | 9.8760 Ops/s | |
test_single_pixels | 0.1360s | 0.1344s | 7.4421 Ops/s | 7.5996 Ops/s | |
test_sync_pixels | 96.0951ms | 94.9250ms | 10.5346 Ops/s | 10.4113 Ops/s | |
test_async_pixels | 0.2436s | 91.3237ms | 10.9501 Ops/s | 10.7918 Ops/s | |
test_simple | 0.9667s | 0.9121s | 1.0963 Ops/s | 1.1249 Ops/s | |
test_transformed | 1.2181s | 1.1517s | 0.8683 Ops/s | 0.9015 Ops/s | |
test_serial | 2.6362s | 2.5649s | 0.3899 Ops/s | 0.4084 Ops/s | |
test_parallel | 2.5850s | 2.5220s | 0.3965 Ops/s | 0.4029 Ops/s | |
test_step_mdp_speed[True-True-True-True-True] | 0.1052ms | 35.9333μs | 27.8294 KOps/s | 29.4440 KOps/s | |
test_step_mdp_speed[True-True-True-True-False] | 73.0610μs | 21.0359μs | 47.5379 KOps/s | 49.8717 KOps/s | |
test_step_mdp_speed[True-True-True-False-True] | 56.8920μs | 21.1557μs | 47.2685 KOps/s | 49.5434 KOps/s | |
test_step_mdp_speed[True-True-True-False-False] | 34.0900μs | 12.2904μs | 81.3645 KOps/s | 85.9219 KOps/s | |
test_step_mdp_speed[True-True-False-True-True] | 67.8620μs | 37.6028μs | 26.5937 KOps/s | 28.1622 KOps/s | |
test_step_mdp_speed[True-True-False-True-False] | 47.6210μs | 22.7564μs | 43.9437 KOps/s | 46.3218 KOps/s | |
test_step_mdp_speed[True-True-False-False-True] | 46.1610μs | 22.6406μs | 44.1685 KOps/s | 45.7071 KOps/s | |
test_step_mdp_speed[True-True-False-False-False] | 34.6910μs | 14.0110μs | 71.3723 KOps/s | 74.0849 KOps/s | |
test_step_mdp_speed[True-False-True-True-True] | 77.7010μs | 39.2561μs | 25.4738 KOps/s | 26.5378 KOps/s | |
test_step_mdp_speed[True-False-True-True-False] | 61.3320μs | 24.8549μs | 40.2335 KOps/s | 42.5422 KOps/s | |
test_step_mdp_speed[True-False-True-False-True] | 52.4820μs | 22.5957μs | 44.2562 KOps/s | 45.6202 KOps/s | |
test_step_mdp_speed[True-False-True-False-False] | 60.1110μs | 14.2137μs | 70.3547 KOps/s | 73.9079 KOps/s | |
test_step_mdp_speed[True-False-False-True-True] | 74.3110μs | 40.6115μs | 24.6236 KOps/s | 24.9621 KOps/s | |
test_step_mdp_speed[True-False-False-True-False] | 49.2610μs | 26.9294μs | 37.1341 KOps/s | 40.0100 KOps/s | |
test_step_mdp_speed[True-False-False-False-True] | 43.3610μs | 24.3396μs | 41.0853 KOps/s | 42.7957 KOps/s | |
test_step_mdp_speed[True-False-False-False-False] | 35.9400μs | 15.9857μs | 62.5558 KOps/s | 65.8765 KOps/s | |
test_step_mdp_speed[False-True-True-True-True] | 82.5920μs | 39.0146μs | 25.6314 KOps/s | 26.9495 KOps/s | |
test_step_mdp_speed[False-True-True-True-False] | 42.0010μs | 24.8066μs | 40.3118 KOps/s | 42.9762 KOps/s | |
test_step_mdp_speed[False-True-True-False-True] | 74.4710μs | 27.8451μs | 35.9129 KOps/s | 38.6193 KOps/s | |
test_step_mdp_speed[False-True-True-False-False] | 43.4610μs | 16.1071μs | 62.0842 KOps/s | 62.9186 KOps/s | |
test_step_mdp_speed[False-True-False-True-True] | 88.2510μs | 41.5887μs | 24.0450 KOps/s | 25.4736 KOps/s | |
test_step_mdp_speed[False-True-False-True-False] | 56.1210μs | 26.9424μs | 37.1162 KOps/s | 38.8588 KOps/s | |
test_step_mdp_speed[False-True-False-False-True] | 49.5410μs | 29.4592μs | 33.9453 KOps/s | 35.8546 KOps/s | |
test_step_mdp_speed[False-True-False-False-False] | 45.7010μs | 18.0493μs | 55.4039 KOps/s | 57.6519 KOps/s | |
test_step_mdp_speed[False-False-True-True-True] | 68.0210μs | 43.8873μs | 22.7857 KOps/s | 24.3585 KOps/s | |
test_step_mdp_speed[False-False-True-True-False] | 75.4110μs | 29.1750μs | 34.2760 KOps/s | 36.3449 KOps/s | |
test_step_mdp_speed[False-False-True-False-True] | 53.2310μs | 29.3440μs | 34.0785 KOps/s | 36.2216 KOps/s | |
test_step_mdp_speed[False-False-True-False-False] | 40.4910μs | 18.2220μs | 54.8786 KOps/s | 57.9915 KOps/s | |
test_step_mdp_speed[False-False-False-True-True] | 72.1120μs | 44.4592μs | 22.4925 KOps/s | 23.5617 KOps/s | |
test_step_mdp_speed[False-False-False-True-False] | 55.8010μs | 30.5685μs | 32.7134 KOps/s | 34.2776 KOps/s | |
test_step_mdp_speed[False-False-False-False-True] | 68.0610μs | 31.0435μs | 32.2128 KOps/s | 34.4018 KOps/s | |
test_step_mdp_speed[False-False-False-False-False] | 40.9600μs | 20.1680μs | 49.5835 KOps/s | 52.7842 KOps/s | |
test_values[generalized_advantage_estimate-True-True] | 28.4948ms | 27.7524ms | 36.0329 Ops/s | 38.1265 Ops/s | |
test_values[vec_generalized_advantage_estimate-True-True] | 94.2162ms | 3.4586ms | 289.1302 Ops/s | 303.1103 Ops/s | |
test_values[td0_return_estimate-False-False] | 0.1041ms | 67.4986μs | 14.8151 KOps/s | 15.3106 KOps/s | |
test_values[td1_return_estimate-False-False] | 61.0230ms | 59.8807ms | 16.6999 Ops/s | 17.6895 Ops/s | |
test_values[vec_td1_return_estimate-False-False] | 2.0742ms | 1.7510ms | 571.0941 Ops/s | 577.3547 Ops/s | |
test_values[td_lambda_return_estimate-True-False] | 97.1831ms | 95.6702ms | 10.4526 Ops/s | 10.7968 Ops/s | |
test_values[vec_td_lambda_return_estimate-True-False] | 2.0223ms | 1.7448ms | 573.1336 Ops/s | 579.0881 Ops/s | |
test_gae_speed[generalized_advantage_estimate-False-1-512] | 26.8938ms | 26.1462ms | 38.2465 Ops/s | 41.3049 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] | 0.8956ms | 0.7308ms | 1.3684 KOps/s | 1.3812 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] | 0.7672ms | 0.7065ms | 1.4155 KOps/s | 1.4689 KOps/s | |
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] | 1.5726ms | 1.4910ms | 670.7115 Ops/s | 677.7248 Ops/s | |
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] | 1.0063ms | 0.7255ms | 1.3783 KOps/s | 1.4137 KOps/s | |
test_dqn_speed | 8.0018ms | 1.5551ms | 643.0645 Ops/s | 677.4807 Ops/s | |
test_ddpg_speed | 4.9502ms | 3.4807ms | 287.3006 Ops/s | 294.4372 Ops/s | |
test_sac_speed | 95.9624ms | 10.5448ms | 94.8331 Ops/s | 105.1488 Ops/s | |
test_redq_speed | 17.9672ms | 17.2235ms | 58.0601 Ops/s | 59.4185 Ops/s | |
test_redq_deprec_speed | 14.6213ms | 13.4863ms | 74.1493 Ops/s | 75.8608 Ops/s | |
test_td3_speed | 9.9512ms | 9.7815ms | 102.2338 Ops/s | 105.1084 Ops/s | |
test_cql_speed | 0.1325s | 36.1572ms | 27.6570 Ops/s | 30.8145 Ops/s | |
test_a2c_speed | 8.6730ms | 7.4156ms | 134.8515 Ops/s | 140.6877 Ops/s | |
test_ppo_speed | 9.1094ms | 7.7303ms | 129.3609 Ops/s | 135.1642 Ops/s | |
test_reinforce_speed | 7.7647ms | 6.4059ms | 156.1054 Ops/s | 159.2220 Ops/s | |
test_iql_speed | 29.6835ms | 27.8766ms | 35.8724 Ops/s | 36.8987 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.2049ms | 2.5422ms | 393.3634 Ops/s | 404.8537 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.7357ms | 2.7257ms | 366.8767 Ops/s | 335.1447 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.4539ms | 2.7256ms | 366.8905 Ops/s | 374.2603 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.2779ms | 2.5210ms | 396.6669 Ops/s | 402.7403 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 3.6837ms | 2.7135ms | 368.5302 Ops/s | 374.3756 Ops/s | |
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 3.9991ms | 2.7194ms | 367.7265 Ops/s | 377.6838 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.1681ms | 2.5322ms | 394.9122 Ops/s | 403.9659 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.9368ms | 2.7191ms | 367.7695 Ops/s | 374.8873 Ops/s | |
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.6693ms | 2.7187ms | 367.8249 Ops/s | 373.5567 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] | 3.5012ms | 2.5459ms | 392.7937 Ops/s | 402.0094 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] | 3.7019ms | 2.7288ms | 366.4616 Ops/s | 376.6586 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] | 4.1756ms | 2.7150ms | 368.3193 Ops/s | 375.3363 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] | 3.2263ms | 2.5094ms | 398.4964 Ops/s | 403.3910 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] | 4.2015ms | 2.7254ms | 366.9207 Ops/s | 376.7624 Ops/s | |
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] | 4.2373ms | 2.7285ms | 366.4956 Ops/s | 375.6281 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] | 3.7177ms | 2.5676ms | 389.4630 Ops/s | 404.0471 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] | 3.8750ms | 2.7376ms | 365.2772 Ops/s | 376.6299 Ops/s | |
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] | 3.8208ms | 2.7294ms | 366.3832 Ops/s | 376.1003 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] | 0.2256s | 20.2498ms | 49.3833 Ops/s | 52.1034 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] | 0.1300s | 18.1341ms | 55.1447 Ops/s | 57.2137 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] | 0.1281s | 18.0965ms | 55.2594 Ops/s | 57.3715 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] | 0.1286s | 15.9303ms | 62.7733 Ops/s | 57.0820 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] | 0.1282s | 18.2377ms | 54.8313 Ops/s | 65.8711 Ops/s | |
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] | 0.1279s | 18.2183ms | 54.8898 Ops/s | 57.0193 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] | 0.1299s | 18.1518ms | 55.0909 Ops/s | 56.9226 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] | 0.1293s | 18.1479ms | 55.1029 Ops/s | 57.1115 Ops/s | |
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] | 0.1284s | 18.2480ms | 54.8005 Ops/s | 57.1220 Ops/s |
facebook-github-bot
added
the
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Dec 1, 2023
vmoens
changed the title
[BugFix, Test] FIx flaky gym vecenvs tests
[BugFix, Test] Fix flaky gym vecenvs tests
Dec 4, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
bug
Something isn't working
CI
Has to do with CI setup (e.g. wheels & builds, tests...)
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Tests
Incomplete or broken unit tests
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.