Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] single player for TicTacToe #2303

Merged
merged 3 commits into from
Jul 22, 2024
Merged

[Feature] single player for TicTacToe #2303

merged 3 commits into from
Jul 22, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 22, 2024 •
edited
Loading

@vmoens
Update
895bfad 
[ghstack-poisoned]
This was referenced Jul 22, 2024
Merged
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 22, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2303

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Pending, 4 Unrelated Failures

As of commit e47e628 with merge base c771e6e (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] single player for TicTacToe
98ce9b7 
ghstack-source-id: bd3aec9cbaffcb8fc9990ebd871dc6f9681782d8
Pull Request resolved: #2303
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 22, 2024
@vmoens
Update
886eef0 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] single player for TicTacToe
76b6b89 
ghstack-source-id: 05cc2b7afd5ed269d952b0044f790feee0585b69
Pull Request resolved: #2303
@vmoens vmoens added enhancement New feature or request Environments Adds or modifies an environment wrapper labels Jul 22, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.9604ms 57.6349ms 17.3506 Ops/s 17.0033 Ops/s $\color{#35bf28}+2.04\%$
test_sync 33.7012ms 31.6225ms 31.6231 Ops/s 31.1000 Ops/s $\color{#35bf28}+1.68\%$
test_async 68.5445ms 29.9608ms 33.3770 Ops/s 33.3000 Ops/s $\color{#35bf28}+0.23\%$
test_simple 0.4768s 0.4113s 2.4313 Ops/s 2.4126 Ops/s $\color{#35bf28}+0.77\%$
test_transformed 0.6239s 0.5675s 1.7621 Ops/s 1.7268 Ops/s $\color{#35bf28}+2.05\%$
test_serial 1.3214s 1.2628s 0.7919 Ops/s 0.7801 Ops/s $\color{#35bf28}+1.51\%$
test_parallel 1.1708s 1.1055s 0.9045 Ops/s 0.8934 Ops/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-True-True-True] 67.9270μs 25.3005μs 39.5250 KOps/s 39.3849 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-True-True-True-False] 45.8460μs 14.7593μs 67.7540 KOps/s 68.5259 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-True-True-False-True] 57.5970μs 14.5277μs 68.8338 KOps/s 68.1062 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-True-False-False] 29.0640μs 8.5120μs 117.4810 KOps/s 117.4860 KOps/s $-0.00\%$
test_step_mdp_speed[True-True-False-True-True] 75.6120μs 26.9700μs 37.0782 KOps/s 36.6883 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-False-True-False] 49.5430μs 17.2999μs 57.8039 KOps/s 61.3037 KOps/s $\textbf{\color{#d91a1a}-5.71\%}$
test_step_mdp_speed[True-True-False-False-True] 0.2623ms 17.1658μs 58.2554 KOps/s 61.0659 KOps/s $\color{#d91a1a}-4.60\%$
test_step_mdp_speed[True-True-False-False-False] 52.3170μs 10.0628μs 99.3756 KOps/s 99.4117 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-True-True-True] 85.8410μs 28.6919μs 34.8530 KOps/s 34.0851 KOps/s $\color{#35bf28}+2.25\%$
test_step_mdp_speed[True-False-True-True-False] 48.8820μs 18.0262μs 55.4747 KOps/s 55.4811 KOps/s $\color{#d91a1a}-0.01\%$
test_step_mdp_speed[True-False-True-False-True] 49.3720μs 16.1722μs 61.8346 KOps/s 60.9628 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-False-True-False-False] 58.4090μs 10.0071μs 99.9289 KOps/s 99.7886 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-False-True-True] 76.8630μs 30.1318μs 33.1876 KOps/s 33.4408 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-False-False-True-False] 49.2520μs 19.5508μs 51.1488 KOps/s 51.6259 KOps/s $\color{#d91a1a}-0.92\%$
test_step_mdp_speed[True-False-False-False-True] 59.0700μs 17.7553μs 56.3211 KOps/s 55.7219 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-False-False-False] 41.9190μs 11.4820μs 87.0930 KOps/s 86.4920 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[False-True-True-True-True] 62.6170μs 28.5562μs 35.0187 KOps/s 34.5006 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-True-True-True-False] 47.8600μs 18.0403μs 55.4313 KOps/s 56.0544 KOps/s $\color{#d91a1a}-1.11\%$
test_step_mdp_speed[False-True-True-False-True] 73.5370μs 18.6479μs 53.6253 KOps/s 53.3656 KOps/s $\color{#35bf28}+0.49\%$
test_step_mdp_speed[False-True-True-False-False] 35.9170μs 11.3128μs 88.3958 KOps/s 88.3066 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-False-True-True] 70.4120μs 30.2276μs 33.0824 KOps/s 32.7293 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-True-False-True-False] 57.9080μs 19.4858μs 51.3193 KOps/s 51.2873 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-True-False-False-True] 55.8650μs 20.5439μs 48.6762 KOps/s 48.9529 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-False] 33.9130μs 12.8760μs 77.6637 KOps/s 78.2648 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[False-False-True-True-True] 3.1993ms 32.2437μs 31.0138 KOps/s 30.9201 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[False-False-True-True-False] 69.6400μs 20.9541μs 47.7233 KOps/s 46.3246 KOps/s $\color{#35bf28}+3.02\%$
test_step_mdp_speed[False-False-True-False-True] 46.5970μs 20.2129μs 49.4734 KOps/s 48.5520 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[False-False-True-False-False] 38.0010μs 12.7967μs 78.1454 KOps/s 77.8746 KOps/s $\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-False-False-True-True] 75.4310μs 33.1888μs 30.1307 KOps/s 29.7594 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-False] 62.2460μs 22.3404μs 44.7620 KOps/s 44.2116 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-False-True] 52.5480μs 21.3268μs 46.8895 KOps/s 46.3321 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-False-False-False-False] 51.8170μs 14.2950μs 69.9546 KOps/s 70.2276 KOps/s $\color{#d91a1a}-0.39\%$
test_values[generalized_advantage_estimate-True-True] 11.1561ms 9.6193ms 103.9574 Ops/s 104.1545 Ops/s $\color{#d91a1a}-0.19\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.7334ms 35.4131ms 28.2381 Ops/s 27.5439 Ops/s $\color{#35bf28}+2.52\%$
test_values[td0_return_estimate-False-False] 0.2316ms 0.1806ms 5.5378 KOps/s 5.4416 KOps/s $\color{#35bf28}+1.77\%$
test_values[td1_return_estimate-False-False] 25.0955ms 23.9504ms 41.7530 Ops/s 42.3639 Ops/s $\color{#d91a1a}-1.44\%$
test_values[vec_td1_return_estimate-False-False] 37.6315ms 35.5339ms 28.1421 Ops/s 27.6299 Ops/s $\color{#35bf28}+1.85\%$
test_values[td_lambda_return_estimate-True-False] 38.4385ms 34.3600ms 29.1036 Ops/s 29.3818 Ops/s $\color{#d91a1a}-0.95\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.5922ms 35.5962ms 28.0929 Ops/s 27.5596 Ops/s $\color{#35bf28}+1.93\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 10.0039ms 8.4460ms 118.3992 Ops/s 119.8481 Ops/s $\color{#d91a1a}-1.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9725ms 1.8771ms 532.7239 Ops/s 420.3443 Ops/s $\textbf{\color{#35bf28}+26.74\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5578ms 0.3546ms 2.8203 KOps/s 2.7923 KOps/s $\color{#35bf28}+1.00\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.8847ms 45.7675ms 21.8496 Ops/s 21.1811 Ops/s $\color{#35bf28}+3.16\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0308ms 3.0307ms 329.9607 Ops/s 326.1093 Ops/s $\color{#35bf28}+1.18\%$
test_dqn_speed 1.5961ms 1.3569ms 736.9512 Ops/s 700.5757 Ops/s $\textbf{\color{#35bf28}+5.19\%}$
test_ddpg_speed 3.3141ms 2.8694ms 348.5088 Ops/s 334.8046 Ops/s $\color{#35bf28}+4.09\%$
test_sac_speed 9.1175ms 8.4754ms 117.9884 Ops/s 113.7190 Ops/s $\color{#35bf28}+3.75\%$
test_redq_speed 15.0999ms 13.3965ms 74.6462 Ops/s 72.5092 Ops/s $\color{#35bf28}+2.95\%$
test_redq_deprec_speed 14.1660ms 13.4283ms 74.4697 Ops/s 73.4349 Ops/s $\color{#35bf28}+1.41\%$
test_td3_speed 11.3908ms 8.5370ms 117.1371 Ops/s 114.1965 Ops/s $\color{#35bf28}+2.58\%$
test_cql_speed 38.7690ms 36.5152ms 27.3858 Ops/s 27.0860 Ops/s $\color{#35bf28}+1.11\%$
test_a2c_speed 9.0356ms 7.4611ms 134.0278 Ops/s 131.5528 Ops/s $\color{#35bf28}+1.88\%$
test_ppo_speed 9.3986ms 7.6965ms 129.9288 Ops/s 127.9829 Ops/s $\color{#35bf28}+1.52\%$
test_reinforce_speed 7.4480ms 6.5915ms 151.7100 Ops/s 148.7342 Ops/s $\color{#35bf28}+2.00\%$
test_iql_speed 34.4942ms 32.4656ms 30.8018 Ops/s 30.5345 Ops/s $\color{#35bf28}+0.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.8794ms 4.2772ms 233.7980 Ops/s 234.5517 Ops/s $\color{#d91a1a}-0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7741ms 0.4711ms 2.1225 KOps/s 2.0804 KOps/s $\color{#35bf28}+2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6335ms 0.4517ms 2.2136 KOps/s 2.1983 KOps/s $\color{#35bf28}+0.70\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.6326ms 4.2448ms 235.5813 Ops/s 233.4228 Ops/s $\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7517ms 0.4667ms 2.1426 KOps/s 2.0981 KOps/s $\color{#35bf28}+2.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7302ms 0.4486ms 2.2293 KOps/s 2.2113 KOps/s $\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0201ms 1.6947ms 590.0870 Ops/s 585.6066 Ops/s $\color{#35bf28}+0.77\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0127ms 1.6093ms 621.4064 Ops/s 619.0936 Ops/s $\color{#35bf28}+0.37\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9111ms 4.4536ms 224.5377 Ops/s 227.2591 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6273ms 0.6185ms 1.6169 KOps/s 1.6242 KOps/s $\color{#d91a1a}-0.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7804ms 0.5826ms 1.7166 KOps/s 1.6963 KOps/s $\color{#35bf28}+1.20\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2262ms 4.2722ms 234.0717 Ops/s 233.4752 Ops/s $\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5974ms 0.4716ms 2.1204 KOps/s 2.0829 KOps/s $\color{#35bf28}+1.80\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.1958ms 0.4637ms 2.1564 KOps/s 2.1877 KOps/s $\color{#d91a1a}-1.43\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.5953ms 4.2624ms 234.6075 Ops/s 236.3622 Ops/s $\color{#d91a1a}-0.74\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8063ms 0.4671ms 2.1407 KOps/s 1.5883 KOps/s $\textbf{\color{#35bf28}+34.78\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.1355ms 0.4529ms 2.2079 KOps/s 2.2147 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9062ms 4.4188ms 226.3077 Ops/s 226.2987 Ops/s $+0.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4151ms 0.6160ms 1.6232 KOps/s 1.6095 KOps/s $\color{#35bf28}+0.85\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1214s 0.7450ms 1.3422 KOps/s 1.6657 KOps/s $\textbf{\color{#d91a1a}-19.42\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1060s 5.9202ms 168.9120 Ops/s 165.5311 Ops/s $\color{#35bf28}+2.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.6580ms 12.8447ms 77.8529 Ops/s 76.4545 Ops/s $\color{#35bf28}+1.83\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.6610ms 1.1935ms 837.9024 Ops/s 816.2497 Ops/s $\color{#35bf28}+2.65\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1075s 5.8687ms 170.3947 Ops/s 125.0218 Ops/s $\textbf{\color{#35bf28}+36.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.8084ms 12.7963ms 78.1477 Ops/s 76.5569 Ops/s $\color{#35bf28}+2.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.6059ms 1.2777ms 782.6770 Ops/s 839.8162 Ops/s $\textbf{\color{#d91a1a}-6.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1059s 8.0492ms 124.2366 Ops/s 160.2051 Ops/s $\textbf{\color{#d91a1a}-22.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.8376ms 13.1771ms 75.8891 Ops/s 75.7110 Ops/s $\color{#35bf28}+0.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.9532ms 1.3289ms 752.4846 Ops/s 788.6578 Ops/s $\color{#d91a1a}-4.59\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1068s 0.1048s 9.5392 Ops/s 8.7328 Ops/s $\textbf{\color{#35bf28}+9.23\%}$
test_sync 95.0866ms 94.5113ms 10.5807 Ops/s 10.5880 Ops/s $\color{#d91a1a}-0.07\%$
test_async 0.1769s 89.3798ms 11.1882 Ops/s 11.3223 Ops/s $\color{#d91a1a}-1.18\%$
test_single_pixels 0.1152s 0.1146s 8.7275 Ops/s 8.6472 Ops/s $\color{#35bf28}+0.93\%$
test_sync_pixels 84.8568ms 82.1620ms 12.1711 Ops/s 13.4972 Ops/s $\textbf{\color{#d91a1a}-9.83\%}$
test_async_pixels 0.1400s 69.2471ms 14.4410 Ops/s 14.4768 Ops/s $\color{#d91a1a}-0.25\%$
test_simple 0.8562s 0.7833s 1.2766 Ops/s 1.2924 Ops/s $\color{#d91a1a}-1.23\%$
test_transformed 1.0790s 1.0017s 0.9983 Ops/s 1.0173 Ops/s $\color{#d91a1a}-1.87\%$
test_serial 2.2511s 2.1791s 0.4589 Ops/s 0.4581 Ops/s $\color{#35bf28}+0.18\%$
test_parallel 2.0028s 1.9388s 0.5158 Ops/s 0.5099 Ops/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-True-True-True] 97.1820μs 39.0591μs 25.6022 KOps/s 26.8312 KOps/s $\color{#d91a1a}-4.58\%$
test_step_mdp_speed[True-True-True-True-False] 47.3110μs 21.8699μs 45.7250 KOps/s 47.4785 KOps/s $\color{#d91a1a}-3.69\%$
test_step_mdp_speed[True-True-True-False-True] 54.8110μs 22.9867μs 43.5033 KOps/s 47.4454 KOps/s $\textbf{\color{#d91a1a}-8.31\%}$
test_step_mdp_speed[True-True-True-False-False] 31.8500μs 13.2512μs 75.4649 KOps/s 83.4415 KOps/s $\textbf{\color{#d91a1a}-9.56\%}$
test_step_mdp_speed[True-True-False-True-True] 67.8910μs 40.7733μs 24.5259 KOps/s 25.8010 KOps/s $\color{#d91a1a}-4.94\%$
test_step_mdp_speed[True-True-False-True-False] 48.8910μs 24.0340μs 41.6078 KOps/s 43.4492 KOps/s $\color{#d91a1a}-4.24\%$
test_step_mdp_speed[True-True-False-False-True] 48.3710μs 23.7992μs 42.0183 KOps/s 43.5856 KOps/s $\color{#d91a1a}-3.60\%$
test_step_mdp_speed[True-True-False-False-False] 34.3810μs 14.8299μs 67.4315 KOps/s 70.6553 KOps/s $\color{#d91a1a}-4.56\%$
test_step_mdp_speed[True-False-True-True-True] 73.4620μs 43.1214μs 23.1903 KOps/s 24.1521 KOps/s $\color{#d91a1a}-3.98\%$
test_step_mdp_speed[True-False-True-True-False] 44.2320μs 25.7457μs 38.8414 KOps/s 38.8393 KOps/s $+0.01\%$
test_step_mdp_speed[True-False-True-False-True] 72.9810μs 23.7694μs 42.0709 KOps/s 43.2458 KOps/s $\color{#d91a1a}-2.72\%$
test_step_mdp_speed[True-False-True-False-False] 41.3810μs 15.3818μs 65.0120 KOps/s 71.3628 KOps/s $\textbf{\color{#d91a1a}-8.90\%}$
test_step_mdp_speed[True-False-False-True-True] 74.4910μs 44.1763μs 22.6365 KOps/s 23.2014 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-False-False-True-False] 54.0410μs 28.1281μs 35.5517 KOps/s 36.4350 KOps/s $\color{#d91a1a}-2.42\%$
test_step_mdp_speed[True-False-False-False-True] 49.9610μs 26.4158μs 37.8562 KOps/s 39.6800 KOps/s $\color{#d91a1a}-4.60\%$
test_step_mdp_speed[True-False-False-False-False] 37.1010μs 17.0299μs 58.7203 KOps/s 62.3069 KOps/s $\textbf{\color{#d91a1a}-5.76\%}$
test_step_mdp_speed[False-True-True-True-True] 63.9210μs 41.9290μs 23.8499 KOps/s 24.1317 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-True-True-True-False] 44.8610μs 25.4868μs 39.2360 KOps/s 38.9932 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-True-False-True] 53.0310μs 27.3549μs 36.5566 KOps/s 36.3936 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-True-True-False-False] 40.6110μs 16.2792μs 61.4279 KOps/s 61.3855 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-False-True-True] 87.4720μs 44.1092μs 22.6710 KOps/s 23.2652 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-True-False-True-False] 45.1610μs 27.6359μs 36.1849 KOps/s 36.2712 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-False-False-True] 49.2110μs 29.0719μs 34.3975 KOps/s 33.6976 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-True-False-False-False] 37.9210μs 18.3773μs 54.4151 KOps/s 55.0100 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-True-True-True] 3.8388ms 46.9270μs 21.3097 KOps/s 21.6547 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-False-True-True-False] 65.9720μs 31.3409μs 31.9072 KOps/s 33.0163 KOps/s $\color{#d91a1a}-3.36\%$
test_step_mdp_speed[False-False-True-False-True] 76.9610μs 30.5081μs 32.7782 KOps/s 34.0524 KOps/s $\color{#d91a1a}-3.74\%$
test_step_mdp_speed[False-False-True-False-False] 44.5110μs 19.4030μs 51.5384 KOps/s 54.5584 KOps/s $\textbf{\color{#d91a1a}-5.54\%}$
test_step_mdp_speed[False-False-False-True-True] 69.6010μs 47.9345μs 20.8618 KOps/s 20.8422 KOps/s $\color{#35bf28}+0.09\%$
test_step_mdp_speed[False-False-False-True-False] 59.1010μs 32.6670μs 30.6119 KOps/s 30.7696 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-False-False-False-True] 54.0410μs 31.2778μs 31.9715 KOps/s 31.9024 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-False-False-False-False] 36.2210μs 20.2928μs 49.2787 KOps/s 49.1804 KOps/s $\color{#35bf28}+0.20\%$
test_values[generalized_advantage_estimate-True-True] 24.7382ms 24.1969ms 41.3277 Ops/s 42.4300 Ops/s $\color{#d91a1a}-2.60\%$
test_values[vec_generalized_advantage_estimate-True-True] 87.4574ms 2.6422ms 378.4757 Ops/s 370.7744 Ops/s $\color{#35bf28}+2.08\%$
test_values[td0_return_estimate-False-False] 87.9820μs 65.0132μs 15.3815 KOps/s 15.4170 KOps/s $\color{#d91a1a}-0.23\%$
test_values[td1_return_estimate-False-False] 58.4633ms 55.0832ms 18.1544 Ops/s 18.7097 Ops/s $\color{#d91a1a}-2.97\%$
test_values[vec_td1_return_estimate-False-False] 1.4106ms 1.0777ms 927.9428 Ops/s 931.7806 Ops/s $\color{#d91a1a}-0.41\%$
test_values[td_lambda_return_estimate-True-False] 93.1335ms 89.9484ms 11.1175 Ops/s 11.7156 Ops/s $\textbf{\color{#d91a1a}-5.11\%}$
test_values[vec_td_lambda_return_estimate-True-False] 1.4359ms 1.0744ms 930.7240 Ops/s 935.9698 Ops/s $\color{#d91a1a}-0.56\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.6302ms 24.5087ms 40.8018 Ops/s 42.0931 Ops/s $\color{#d91a1a}-3.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9286ms 0.7086ms 1.4113 KOps/s 1.4290 KOps/s $\color{#d91a1a}-1.24\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7402ms 0.6620ms 1.5105 KOps/s 1.5341 KOps/s $\color{#d91a1a}-1.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.4943ms 1.4593ms 685.2759 Ops/s 689.8976 Ops/s $\color{#d91a1a}-0.67\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7074ms 0.6775ms 1.4760 KOps/s 1.4939 KOps/s $\color{#d91a1a}-1.20\%$
test_dqn_speed 7.1509ms 1.4189ms 704.7888 Ops/s 705.5252 Ops/s $\color{#d91a1a}-0.10\%$
test_ddpg_speed 3.2336ms 2.8898ms 346.0462 Ops/s 345.3894 Ops/s $\color{#35bf28}+0.19\%$
test_sac_speed 0.1003s 9.1717ms 109.0316 Ops/s 118.9261 Ops/s $\textbf{\color{#d91a1a}-8.32\%}$
test_redq_speed 11.2316ms 10.3888ms 96.2578 Ops/s 94.3094 Ops/s $\color{#35bf28}+2.07\%$
test_redq_deprec_speed 11.4707ms 11.0616ms 90.4031 Ops/s 88.8263 Ops/s $\color{#35bf28}+1.78\%$
test_td3_speed 8.4518ms 8.2404ms 121.3538 Ops/s 120.0487 Ops/s $\color{#35bf28}+1.09\%$
test_cql_speed 25.7863ms 25.2153ms 39.6584 Ops/s 35.4041 Ops/s $\textbf{\color{#35bf28}+12.02\%}$
test_a2c_speed 6.2896ms 5.6128ms 178.1644 Ops/s 181.0428 Ops/s $\color{#d91a1a}-1.59\%$
test_ppo_speed 6.0498ms 5.8793ms 170.0875 Ops/s 171.4888 Ops/s $\color{#d91a1a}-0.82\%$
test_reinforce_speed 4.9107ms 4.5853ms 218.0886 Ops/s 220.5392 Ops/s $\color{#d91a1a}-1.11\%$
test_iql_speed 20.2832ms 19.4666ms 51.3699 Ops/s 51.5694 Ops/s $\color{#d91a1a}-0.39\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8893ms 5.7614ms 173.5689 Ops/s 172.2761 Ops/s $\color{#35bf28}+0.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7572ms 0.5038ms 1.9849 KOps/s 1.9791 KOps/s $\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6653ms 0.4820ms 2.0746 KOps/s 2.0598 KOps/s $\color{#35bf28}+0.72\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8919ms 5.6674ms 176.4486 Ops/s 176.6678 Ops/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6424ms 0.5008ms 1.9969 KOps/s 2.0084 KOps/s $\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6787ms 0.4776ms 2.0939 KOps/s 2.0965 KOps/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.0271ms 1.8937ms 528.0574 Ops/s 528.9399 Ops/s $\color{#d91a1a}-0.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 6.5221ms 1.8054ms 553.8806 Ops/s 556.8181 Ops/s $\color{#d91a1a}-0.53\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9627ms 5.8504ms 170.9290 Ops/s 168.9672 Ops/s $\color{#35bf28}+1.16\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4427ms 0.6456ms 1.5490 KOps/s 1.5500 KOps/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1516s 0.7525ms 1.3289 KOps/s 1.5983 KOps/s $\textbf{\color{#d91a1a}-16.86\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8562ms 5.7327ms 174.4392 Ops/s 174.2173 Ops/s $\color{#35bf28}+0.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.4969ms 0.5051ms 1.9796 KOps/s 1.9748 KOps/s $\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6666ms 0.4831ms 2.0699 KOps/s 2.0638 KOps/s $\color{#35bf28}+0.30\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9266ms 5.6576ms 176.7523 Ops/s 175.1910 Ops/s $\color{#35bf28}+0.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8677ms 0.4994ms 2.0024 KOps/s 1.9922 KOps/s $\color{#35bf28}+0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5991ms 0.4768ms 2.0972 KOps/s 2.0863 KOps/s $\color{#35bf28}+0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2247ms 5.9051ms 169.3459 Ops/s 170.4197 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4504ms 0.6446ms 1.5513 KOps/s 1.5463 KOps/s $\color{#35bf28}+0.32\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7817ms 0.6235ms 1.6039 KOps/s 1.5800 KOps/s $\color{#35bf28}+1.52\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1418s 7.8381ms 127.5819 Ops/s 127.3674 Ops/s $\color{#35bf28}+0.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.6509ms 15.6158ms 64.0378 Ops/s 63.7095 Ops/s $\color{#35bf28}+0.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.5254ms 1.3135ms 761.3222 Ops/s 806.1539 Ops/s $\textbf{\color{#d91a1a}-5.56\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1231s 7.4453ms 134.3129 Ops/s 132.4577 Ops/s $\color{#35bf28}+1.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1336s 18.0708ms 55.3379 Ops/s 63.5759 Ops/s $\textbf{\color{#d91a1a}-12.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8454ms 1.0743ms 930.7998 Ops/s 766.7500 Ops/s $\textbf{\color{#35bf28}+21.40\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1240s 7.6339ms 130.9952 Ops/s 130.5813 Ops/s $\color{#35bf28}+0.32\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.5202ms 15.8617ms 63.0448 Ops/s 63.7865 Ops/s $\color{#d91a1a}-1.16\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.2279ms 1.4668ms 681.7591 Ops/s 742.1718 Ops/s $\textbf{\color{#d91a1a}-8.14\%}$

@vmoens
Update
e47e628 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] single player for TicTacToe
f978684 
ghstack-source-id: 18fb525b49fb8b3b9aef8367a54070eb70fbb3c0
Pull Request resolved: #2303
@vmoens vmoens merged commit e47e628 into gh/vmoens/3/base Jul 22, 2024
45 of 51 checks passed
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] single player for TicTacToe
59c3374 
ghstack-source-id: 18fb525b49fb8b3b9aef8367a54070eb70fbb3c0
Pull Request resolved: #2303
@vmoens vmoens deleted the gh/vmoens/3/head branch July 22, 2024 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request Environments Adds or modifies an environment wrapper
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot