Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Example,BugFix] Add a Async gym env example #2139

Merged
merged 3 commits into from
Apr 30, 2024
Merged

[Example,BugFix] Add a Async gym env example #2139

merged 3 commits into from
Apr 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 30, 2024 •
edited
Loading

Also solves a bug when multiple gym envs are created and their readers interact with each other

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2139

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 14 Unrelated Failures

As of commit 3a14155 with merge base 3c6b9c6 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 30, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 30, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 56.5091ms 54.7803ms 18.2547 Ops/s 18.0975 Ops/s $\color{#35bf28}+0.87\%$
test_sync 35.1090ms 30.7572ms 32.5127 Ops/s 30.1531 Ops/s $\textbf{\color{#35bf28}+7.83\%}$
test_async 55.9378ms 29.0481ms 34.4256 Ops/s 34.1776 Ops/s $\color{#35bf28}+0.73\%$
test_simple 0.4188s 0.3565s 2.8050 Ops/s 2.9084 Ops/s $\color{#d91a1a}-3.56\%$
test_transformed 0.4977s 0.4912s 2.0359 Ops/s 1.9929 Ops/s $\color{#35bf28}+2.16\%$
test_serial 1.2805s 1.2211s 0.8190 Ops/s 0.8002 Ops/s $\color{#35bf28}+2.34\%$
test_parallel 1.1018s 1.0381s 0.9633 Ops/s 0.9681 Ops/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-True-True-True-True] 0.2428ms 21.0144μs 47.5864 KOps/s 47.0303 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[True-True-True-True-False] 40.5560μs 12.8545μs 77.7937 KOps/s 72.0523 KOps/s $\textbf{\color{#35bf28}+7.97\%}$
test_step_mdp_speed[True-True-True-False-True] 70.3910μs 12.3759μs 80.8019 KOps/s 77.5774 KOps/s $\color{#35bf28}+4.16\%$
test_step_mdp_speed[True-True-True-False-False] 28.7040μs 7.5421μs 132.5889 KOps/s 132.4621 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-True-False-True-True] 81.8430μs 22.5866μs 44.2740 KOps/s 44.4307 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-False-True-False] 43.3410μs 14.2197μs 70.3250 KOps/s 70.3744 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[True-True-False-False-True] 50.3440μs 13.6446μs 73.2890 KOps/s 73.9083 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-False-False-False] 30.3760μs 8.8330μs 113.2118 KOps/s 113.5588 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-True-True-True] 85.2590μs 23.8563μs 41.9176 KOps/s 41.9740 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-False-True-True-False] 41.9380μs 15.4279μs 64.8177 KOps/s 64.4142 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-False-True-False-True] 49.2110μs 13.7506μs 72.7242 KOps/s 73.6993 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[True-False-True-False-False] 34.2440μs 8.8048μs 113.5740 KOps/s 115.6813 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[True-False-False-True-True] 60.7730μs 25.0221μs 39.9647 KOps/s 40.3511 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[True-False-False-True-False] 46.2360μs 16.6325μs 60.1234 KOps/s 60.3906 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-False-False-False-True] 45.0940μs 14.7087μs 67.9870 KOps/s 68.1175 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-False-False-False-False] 41.4480μs 9.9817μs 100.1835 KOps/s 101.0199 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-True-True-True-True] 58.8700μs 23.9067μs 41.8293 KOps/s 42.0141 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-True-True-True-False] 43.3210μs 15.5675μs 64.2364 KOps/s 64.5455 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-True-True-False-True] 77.2160μs 15.7834μs 63.3578 KOps/s 63.2762 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-True-True-False-False] 55.0030μs 9.9505μs 100.4973 KOps/s 101.3505 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-True-False-True-True] 76.0620μs 24.6826μs 40.5144 KOps/s 40.1265 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-True-False-True-False] 56.7150μs 16.6089μs 60.2087 KOps/s 59.9026 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-True-False-False-True] 50.0030μs 16.8914μs 59.2016 KOps/s 59.1794 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-True-False-False-False] 43.1400μs 11.1780μs 89.4618 KOps/s 89.7733 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-False-True-True-True] 62.7170μs 26.0722μs 38.3551 KOps/s 38.1469 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-True-True-False] 41.1170μs 17.8207μs 56.1144 KOps/s 55.6151 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-False-True-False-True] 52.5080μs 16.9888μs 58.8622 KOps/s 58.6568 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-False-True-False-False] 50.9650μs 11.1157μs 89.9629 KOps/s 90.2061 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-False-False-True-True] 38.9820μs 27.3282μs 36.5922 KOps/s 36.4338 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-False-True-False] 54.5020μs 18.9402μs 52.7978 KOps/s 52.1376 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[False-False-False-False-True] 59.8810μs 17.9207μs 55.8013 KOps/s 55.7755 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-False-False-False-False] 62.6660μs 12.2133μs 81.8778 KOps/s 80.5195 KOps/s $\color{#35bf28}+1.69\%$
test_values[generalized_advantage_estimate-True-True] 10.8435ms 9.6285ms 103.8587 Ops/s 103.4292 Ops/s $\color{#35bf28}+0.42\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.4944ms 33.4336ms 29.9100 Ops/s 29.1459 Ops/s $\color{#35bf28}+2.62\%$
test_values[td0_return_estimate-False-False] 0.2753ms 0.1958ms 5.1069 KOps/s 5.3865 KOps/s $\textbf{\color{#d91a1a}-5.19\%}$
test_values[td1_return_estimate-False-False] 25.9551ms 23.4592ms 42.6272 Ops/s 42.4322 Ops/s $\color{#35bf28}+0.46\%$
test_values[vec_td1_return_estimate-False-False] 34.9194ms 33.4723ms 29.8754 Ops/s 28.8265 Ops/s $\color{#35bf28}+3.64\%$
test_values[td_lambda_return_estimate-True-False] 37.0322ms 33.7629ms 29.6183 Ops/s 29.5160 Ops/s $\color{#35bf28}+0.35\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.1976ms 33.4801ms 29.8685 Ops/s 28.5584 Ops/s $\color{#35bf28}+4.59\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4134ms 8.2349ms 121.4349 Ops/s 120.7354 Ops/s $\color{#35bf28}+0.58\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2505ms 1.8039ms 554.3506 Ops/s 515.8260 Ops/s $\textbf{\color{#35bf28}+7.47\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6104ms 0.3517ms 2.8436 KOps/s 2.8304 KOps/s $\color{#35bf28}+0.47\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 42.6423ms 40.6434ms 24.6042 Ops/s 22.4046 Ops/s $\textbf{\color{#35bf28}+9.82\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8023ms 3.0800ms 324.6793 Ops/s 325.1095 Ops/s $\color{#d91a1a}-0.13\%$
test_dqn_speed 1.7169ms 1.3631ms 733.6262 Ops/s 724.3758 Ops/s $\color{#35bf28}+1.28\%$
test_ddpg_speed 3.6141ms 2.8717ms 348.2235 Ops/s 345.5632 Ops/s $\color{#35bf28}+0.77\%$
test_sac_speed 10.0567ms 8.6720ms 115.3133 Ops/s 112.4336 Ops/s $\color{#35bf28}+2.56\%$
test_redq_speed 14.7332ms 13.3763ms 74.7589 Ops/s 74.2311 Ops/s $\color{#35bf28}+0.71\%$
test_redq_deprec_speed 16.1416ms 14.2216ms 70.3157 Ops/s 68.2325 Ops/s $\color{#35bf28}+3.05\%$
test_td3_speed 17.6179ms 8.8101ms 113.5065 Ops/s 112.9361 Ops/s $\color{#35bf28}+0.51\%$
test_cql_speed 40.1394ms 37.9244ms 26.3683 Ops/s 26.5724 Ops/s $\color{#d91a1a}-0.77\%$
test_a2c_speed 12.2460ms 7.9811ms 125.2956 Ops/s 122.7432 Ops/s $\color{#35bf28}+2.08\%$
test_ppo_speed 14.4951ms 8.6056ms 116.2028 Ops/s 118.4608 Ops/s $\color{#d91a1a}-1.91\%$
test_reinforce_speed 7.4928ms 7.0779ms 141.2850 Ops/s 139.0196 Ops/s $\color{#35bf28}+1.63\%$
test_iql_speed 34.7836ms 33.8670ms 29.5273 Ops/s 29.3493 Ops/s $\color{#35bf28}+0.61\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.4221ms 2.4654ms 405.6107 Ops/s 431.3285 Ops/s $\textbf{\color{#d91a1a}-5.96\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2161ms 0.5192ms 1.9259 KOps/s 1.9536 KOps/s $\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7243ms 0.4917ms 2.0338 KOps/s 2.0425 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0285ms 2.4437ms 409.2195 Ops/s 406.9058 Ops/s $\color{#35bf28}+0.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0826ms 0.5155ms 1.9399 KOps/s 1.9610 KOps/s $\color{#d91a1a}-1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7815ms 0.4882ms 2.0485 KOps/s 2.0341 KOps/s $\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7305ms 1.2484ms 801.0399 Ops/s 801.0394 Ops/s $+0.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.7302ms 1.1848ms 844.0451 Ops/s 842.3954 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.6129ms 2.4823ms 402.8587 Ops/s 387.4240 Ops/s $\color{#35bf28}+3.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1263ms 0.6379ms 1.5677 KOps/s 1.5352 KOps/s $\color{#35bf28}+2.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8034ms 0.6116ms 1.6349 KOps/s 1.6110 KOps/s $\color{#35bf28}+1.49\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3584ms 2.3507ms 425.4022 Ops/s 407.8302 Ops/s $\color{#35bf28}+4.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0794ms 0.5161ms 1.9377 KOps/s 1.9143 KOps/s $\color{#35bf28}+1.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6372ms 0.4927ms 2.0298 KOps/s 2.0141 KOps/s $\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6049ms 2.3670ms 422.4689 Ops/s 408.9102 Ops/s $\color{#35bf28}+3.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7292ms 0.5183ms 1.9293 KOps/s 1.9265 KOps/s $\color{#35bf28}+0.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6018ms 0.4866ms 2.0551 KOps/s 2.0384 KOps/s $\color{#35bf28}+0.82\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.8498ms 2.4879ms 401.9388 Ops/s 371.8041 Ops/s $\textbf{\color{#35bf28}+8.10\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1356ms 0.6388ms 1.5654 KOps/s 1.5554 KOps/s $\color{#35bf28}+0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7910ms 0.6118ms 1.6346 KOps/s 1.6073 KOps/s $\color{#35bf28}+1.70\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1204s 8.1922ms 122.0679 Ops/s 124.2071 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.2072ms 12.4550ms 80.2888 Ops/s 78.9622 Ops/s $\color{#35bf28}+1.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.6552ms 1.0845ms 922.1101 Ops/s 910.1252 Ops/s $\color{#35bf28}+1.32\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1094s 5.8456ms 171.0681 Ops/s 168.2604 Ops/s $\color{#35bf28}+1.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 14.7213ms 12.2311ms 81.7590 Ops/s 78.7904 Ops/s $\color{#35bf28}+3.77\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8075ms 1.0799ms 926.0013 Ops/s 911.0316 Ops/s $\color{#35bf28}+1.64\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1135s 8.3806ms 119.3230 Ops/s 115.3733 Ops/s $\color{#35bf28}+3.42\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 15.4100ms 12.5577ms 79.6323 Ops/s 75.5537 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.2296ms 1.3974ms 715.6158 Ops/s 702.2752 Ops/s $\color{#35bf28}+1.90\%$

@vmoens vmoens changed the title [Example] Add a Async gym env example [Example,BugFix] Add a Async gym env example Apr 30, 2024
@vmoens vmoens added the bug Something isn't working label Apr 30, 2024
@vmoens vmoens merged commit c25ec59 into main Apr 30, 2024
16 of 20 checks passed
@vmoens vmoens deleted the gym-async-example branch April 30, 2024 08:41
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1162s 0.1154s 8.6645 Ops/s 8.8275 Ops/s $\color{#d91a1a}-1.85\%$
test_sync 99.6778ms 98.4541ms 10.1570 Ops/s 9.9303 Ops/s $\color{#35bf28}+2.28\%$
test_async 0.1890s 95.4500ms 10.4767 Ops/s 10.4352 Ops/s $\color{#35bf28}+0.40\%$
test_single_pixels 0.1945s 0.1317s 7.5903 Ops/s 7.9426 Ops/s $\color{#d91a1a}-4.44\%$
test_sync_pixels 0.1356s 83.5921ms 11.9629 Ops/s 12.0138 Ops/s $\color{#d91a1a}-0.42\%$
test_async_pixels 0.1577s 78.9984ms 12.6585 Ops/s 13.1144 Ops/s $\color{#d91a1a}-3.48\%$
test_simple 0.8185s 0.7546s 1.3252 Ops/s 1.3191 Ops/s $\color{#35bf28}+0.47\%$
test_transformed 1.0564s 1.0012s 0.9988 Ops/s 0.9955 Ops/s $\color{#35bf28}+0.33\%$
test_serial 2.4617s 2.4445s 0.4091 Ops/s 0.4188 Ops/s $\color{#d91a1a}-2.31\%$
test_parallel 2.3784s 2.3180s 0.4314 Ops/s 0.4356 Ops/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-True-True-True-True] 0.1136ms 33.3014μs 30.0288 KOps/s 30.1352 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-True-True-False] 40.9310μs 19.8264μs 50.4379 KOps/s 49.3121 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[True-True-True-False-True] 37.1900μs 18.6484μs 53.6240 KOps/s 53.7261 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[True-True-True-False-False] 32.0100μs 11.1881μs 89.3807 KOps/s 86.5925 KOps/s $\color{#35bf28}+3.22\%$
test_step_mdp_speed[True-True-False-True-True] 52.5310μs 35.0935μs 28.4953 KOps/s 28.7969 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[True-True-False-True-False] 44.9110μs 21.3835μs 46.7651 KOps/s 46.4299 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-True] 47.7210μs 20.3166μs 49.2209 KOps/s 48.9116 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-False-False-False] 30.2410μs 12.9711μs 77.0944 KOps/s 75.5422 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[True-False-True-True-True] 65.4710μs 36.5292μs 27.3754 KOps/s 26.8940 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-False-True-True-False] 46.0310μs 23.3408μs 42.8433 KOps/s 41.6546 KOps/s $\color{#35bf28}+2.85\%$
test_step_mdp_speed[True-False-True-False-True] 50.4610μs 20.5207μs 48.7313 KOps/s 48.2771 KOps/s $\color{#35bf28}+0.94\%$
test_step_mdp_speed[True-False-True-False-False] 29.9910μs 13.1509μs 76.0407 KOps/s 75.6670 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[True-False-False-True-True] 69.2920μs 37.9729μs 26.3345 KOps/s 25.6148 KOps/s $\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-False-False-True-False] 52.0710μs 25.0760μs 39.8787 KOps/s 39.0525 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-False-False-True] 56.0610μs 22.1736μs 45.0986 KOps/s 44.5809 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-False-False-False-False] 33.2710μs 14.8204μs 67.4744 KOps/s 65.0720 KOps/s $\color{#35bf28}+3.69\%$
test_step_mdp_speed[False-True-True-True-True] 63.2720μs 36.1510μs 27.6617 KOps/s 27.0784 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-True-True-True-False] 67.4120μs 23.6388μs 42.3034 KOps/s 41.8305 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[False-True-True-False-True] 39.4410μs 24.3953μs 40.9916 KOps/s 40.9808 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-True-False-False] 34.1110μs 14.7641μs 67.7317 KOps/s 66.1719 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-True-False-True-True] 65.4120μs 38.0606μs 26.2739 KOps/s 26.0851 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-True-False-True-False] 50.2310μs 25.4840μs 39.2403 KOps/s 39.2633 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-False-False-True] 47.3710μs 26.4888μs 37.7518 KOps/s 37.7322 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-False-False-False] 43.0710μs 16.6522μs 60.0523 KOps/s 58.4577 KOps/s $\color{#35bf28}+2.73\%$
test_step_mdp_speed[False-False-True-True-True] 71.9210μs 40.6824μs 24.5807 KOps/s 24.5492 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-True-True-False] 53.5610μs 27.5083μs 36.3527 KOps/s 35.7999 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[False-False-True-False-True] 45.9310μs 26.7299μs 37.4113 KOps/s 37.9424 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-False-True-False-False] 38.3410μs 16.7143μs 59.8291 KOps/s 59.4334 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-False-True-True] 54.8510μs 42.5135μs 23.5219 KOps/s 23.2993 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-False-True-False] 55.0210μs 29.2802μs 34.1528 KOps/s 33.3976 KOps/s $\color{#35bf28}+2.26\%$
test_step_mdp_speed[False-False-False-False-True] 64.3210μs 28.7011μs 34.8419 KOps/s 35.5660 KOps/s $\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-False-False-False-False] 35.4910μs 18.7872μs 53.2278 KOps/s 53.3003 KOps/s $\color{#d91a1a}-0.14\%$
test_values[generalized_advantage_estimate-True-True] 23.7389ms 23.1025ms 43.2853 Ops/s 43.6812 Ops/s $\color{#d91a1a}-0.91\%$
test_values[vec_generalized_advantage_estimate-True-True] 79.3660ms 3.1394ms 318.5288 Ops/s 316.3165 Ops/s $\color{#35bf28}+0.70\%$
test_values[td0_return_estimate-False-False] 93.0920μs 61.1529μs 16.3525 KOps/s 16.1711 KOps/s $\color{#35bf28}+1.12\%$
test_values[td1_return_estimate-False-False] 50.2403ms 49.5058ms 20.1996 Ops/s 18.9164 Ops/s $\textbf{\color{#35bf28}+6.78\%}$
test_values[vec_td1_return_estimate-False-False] 2.0532ms 1.7400ms 574.7078 Ops/s 573.2094 Ops/s $\color{#35bf28}+0.26\%$
test_values[td_lambda_return_estimate-True-False] 79.8803ms 78.7754ms 12.6943 Ops/s 11.8156 Ops/s $\textbf{\color{#35bf28}+7.44\%}$
test_values[vec_td_lambda_return_estimate-True-False] 2.1006ms 1.7378ms 575.4481 Ops/s 573.7318 Ops/s $\color{#35bf28}+0.30\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.3658ms 22.1978ms 45.0495 Ops/s 42.3828 Ops/s $\textbf{\color{#35bf28}+6.29\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8948ms 0.6669ms 1.4995 KOps/s 1.4105 KOps/s $\textbf{\color{#35bf28}+6.31\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6828ms 0.6245ms 1.6013 KOps/s 1.5462 KOps/s $\color{#35bf28}+3.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4977ms 1.4254ms 701.5345 Ops/s 687.9155 Ops/s $\color{#35bf28}+1.98\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9110ms 0.6486ms 1.5419 KOps/s 1.4839 KOps/s $\color{#35bf28}+3.91\%$
test_dqn_speed 2.0725ms 1.4247ms 701.9049 Ops/s 691.0012 Ops/s $\color{#35bf28}+1.58\%$
test_ddpg_speed 3.0314ms 2.8921ms 345.7654 Ops/s 338.4151 Ops/s $\color{#35bf28}+2.17\%$
test_sac_speed 8.6786ms 8.2601ms 121.0641 Ops/s 119.0381 Ops/s $\color{#35bf28}+1.70\%$
test_redq_speed 10.9303ms 10.1822ms 98.2109 Ops/s 95.4910 Ops/s $\color{#35bf28}+2.85\%$
test_redq_deprec_speed 11.8374ms 11.2556ms 88.8447 Ops/s 87.2128 Ops/s $\color{#35bf28}+1.87\%$
test_td3_speed 8.3658ms 8.2511ms 121.1960 Ops/s 120.3744 Ops/s $\color{#35bf28}+0.68\%$
test_cql_speed 26.7305ms 25.8693ms 38.6558 Ops/s 39.1648 Ops/s $\color{#d91a1a}-1.30\%$
test_a2c_speed 5.9238ms 5.6879ms 175.8108 Ops/s 175.4891 Ops/s $\color{#35bf28}+0.18\%$
test_ppo_speed 7.0197ms 5.9719ms 167.4507 Ops/s 167.1518 Ops/s $\color{#35bf28}+0.18\%$
test_reinforce_speed 5.0325ms 4.6802ms 213.6662 Ops/s 216.7385 Ops/s $\color{#d91a1a}-1.42\%$
test_iql_speed 20.9702ms 19.8396ms 50.4043 Ops/s 50.3750 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8432ms 2.7522ms 363.3392 Ops/s 360.3144 Ops/s $\color{#35bf28}+0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7330ms 0.5942ms 1.6831 KOps/s 1.6605 KOps/s $\color{#35bf28}+1.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.2654ms 0.5738ms 1.7427 KOps/s 1.7312 KOps/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.9694ms 2.7873ms 358.7745 Ops/s 359.9091 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7043ms 0.5875ms 1.7022 KOps/s 1.6809 KOps/s $\color{#35bf28}+1.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.3524ms 0.5672ms 1.7632 KOps/s 1.7638 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6090ms 1.4867ms 672.6219 Ops/s 661.7576 Ops/s $\color{#35bf28}+1.64\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5901ms 1.4169ms 705.7910 Ops/s 698.5969 Ops/s $\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1578ms 2.8989ms 344.9624 Ops/s 344.8185 Ops/s $\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.5827ms 0.7186ms 1.3916 KOps/s 1.3846 KOps/s $\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8476ms 0.6893ms 1.4508 KOps/s 1.4041 KOps/s $\color{#35bf28}+3.33\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.8622ms 2.7635ms 361.8566 Ops/s 359.7582 Ops/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.3914ms 0.5981ms 1.6720 KOps/s 1.6438 KOps/s $\color{#35bf28}+1.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7364ms 0.5712ms 1.7506 KOps/s 1.7093 KOps/s $\color{#35bf28}+2.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0271ms 2.7824ms 359.4027 Ops/s 355.6078 Ops/s $\color{#35bf28}+1.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7044ms 0.5898ms 1.6954 KOps/s 1.6590 KOps/s $\color{#35bf28}+2.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7031ms 0.5604ms 1.7846 KOps/s 1.7445 KOps/s $\color{#35bf28}+2.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0571ms 2.8945ms 345.4821 Ops/s 347.1677 Ops/s $\color{#d91a1a}-0.49\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6639ms 0.7253ms 1.3787 KOps/s 1.3734 KOps/s $\color{#35bf28}+0.39\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8551ms 0.6934ms 1.4421 KOps/s 1.4160 KOps/s $\color{#35bf28}+1.84\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1261s 9.4827ms 105.4557 Ops/s 104.1974 Ops/s $\color{#35bf28}+1.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.3466ms 14.9075ms 67.0804 Ops/s 67.1253 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.4446ms 1.2315ms 811.9924 Ops/s 847.3323 Ops/s $\color{#d91a1a}-4.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1190s 7.0962ms 140.9214 Ops/s 138.5009 Ops/s $\color{#35bf28}+1.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.4141ms 14.7614ms 67.7441 Ops/s 68.4375 Ops/s $\color{#d91a1a}-1.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1736ms 1.1939ms 837.5674 Ops/s 827.2393 Ops/s $\color{#35bf28}+1.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1191s 7.4459ms 134.3018 Ops/s 101.7356 Ops/s $\textbf{\color{#35bf28}+32.01\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1272s 17.4735ms 57.2296 Ops/s 66.5544 Ops/s $\textbf{\color{#d91a1a}-14.01\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.6944ms 1.5318ms 652.8234 Ops/s 590.5259 Ops/s $\textbf{\color{#35bf28}+10.55\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot