Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] TicTacToeEnv #2301

Merged
merged 3 commits into from
Jul 22, 2024
Merged

[Feature] TicTacToeEnv #2301

merged 3 commits into from
Jul 22, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 22, 2024 •
edited
Loading

@vmoens
Update
4683408 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 22, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2301

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Pending, 4 Unrelated Failures

As of commit a689eed with merge base c771e6e (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens mentioned this pull request Jul 22, 2024
Merged
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 22, 2024
@vmoens vmoens mentioned this pull request Jul 22, 2024
Merged
@vmoens
Update
85f0e15 
[ghstack-poisoned]
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 62.4776ms 57.5570ms 17.3741 Ops/s 17.0386 Ops/s $\color{#35bf28}+1.97\%$
test_sync 41.1893ms 32.3467ms 30.9151 Ops/s 31.4336 Ops/s $\color{#d91a1a}-1.65\%$
test_async 80.4314ms 30.5672ms 32.7149 Ops/s 33.5576 Ops/s $\color{#d91a1a}-2.51\%$
test_simple 0.4785s 0.4080s 2.4508 Ops/s 2.4228 Ops/s $\color{#35bf28}+1.16\%$
test_transformed 0.6204s 0.5615s 1.7810 Ops/s 1.7650 Ops/s $\color{#35bf28}+0.91\%$
test_serial 1.3296s 1.2676s 0.7889 Ops/s 0.7921 Ops/s $\color{#d91a1a}-0.41\%$
test_parallel 1.1961s 1.1244s 0.8894 Ops/s 0.9124 Ops/s $\color{#d91a1a}-2.52\%$
test_step_mdp_speed[True-True-True-True-True] 0.1984ms 25.8444μs 38.6931 KOps/s 39.6762 KOps/s $\color{#d91a1a}-2.48\%$
test_step_mdp_speed[True-True-True-True-False] 42.8900μs 15.0108μs 66.6187 KOps/s 68.1327 KOps/s $\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-True-True-False-True] 67.7970μs 14.9634μs 66.8296 KOps/s 68.7335 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[True-True-True-False-False] 0.1649ms 8.7112μs 114.7944 KOps/s 118.7064 KOps/s $\color{#d91a1a}-3.30\%$
test_step_mdp_speed[True-True-False-True-True] 72.6560μs 27.9415μs 35.7891 KOps/s 37.0239 KOps/s $\color{#d91a1a}-3.34\%$
test_step_mdp_speed[True-True-False-True-False] 58.7690μs 16.7482μs 59.7078 KOps/s 61.7239 KOps/s $\color{#d91a1a}-3.27\%$
test_step_mdp_speed[True-True-False-False-True] 0.1610ms 17.2060μs 58.1191 KOps/s 62.1020 KOps/s $\textbf{\color{#d91a1a}-6.41\%}$
test_step_mdp_speed[True-True-False-False-False] 50.9650μs 10.3388μs 96.7233 KOps/s 97.0235 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-False-True-True-True] 0.1056ms 29.7813μs 33.5781 KOps/s 35.0246 KOps/s $\color{#d91a1a}-4.13\%$
test_step_mdp_speed[True-False-True-True-False] 73.2270μs 18.4365μs 54.2401 KOps/s 56.0388 KOps/s $\color{#d91a1a}-3.21\%$
test_step_mdp_speed[True-False-True-False-True] 44.1320μs 16.7463μs 59.7147 KOps/s 61.9634 KOps/s $\color{#d91a1a}-3.63\%$
test_step_mdp_speed[True-False-True-False-False] 57.9680μs 10.3409μs 96.7034 KOps/s 99.8452 KOps/s $\color{#d91a1a}-3.15\%$
test_step_mdp_speed[True-False-False-True-True] 83.0450μs 31.1889μs 32.0627 KOps/s 33.2180 KOps/s $\color{#d91a1a}-3.48\%$
test_step_mdp_speed[True-False-False-True-False] 49.7820μs 20.2927μs 49.2787 KOps/s 51.0039 KOps/s $\color{#d91a1a}-3.38\%$
test_step_mdp_speed[True-False-False-False-True] 67.1350μs 17.9733μs 55.6381 KOps/s 56.5443 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[True-False-False-False-False] 53.2790μs 11.7662μs 84.9889 KOps/s 85.1250 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-True-True-True-True] 64.3900μs 29.8548μs 33.4955 KOps/s 34.9149 KOps/s $\color{#d91a1a}-4.07\%$
test_step_mdp_speed[False-True-True-True-False] 68.0770μs 18.7079μs 53.4535 KOps/s 55.9959 KOps/s $\color{#d91a1a}-4.54\%$
test_step_mdp_speed[False-True-True-False-True] 51.3760μs 19.7265μs 50.6932 KOps/s 52.5220 KOps/s $\color{#d91a1a}-3.48\%$
test_step_mdp_speed[False-True-True-False-False] 62.1570μs 11.6113μs 86.1232 KOps/s 87.2242 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-True-False-True-True] 66.7950μs 31.1334μs 32.1198 KOps/s 33.2802 KOps/s $\color{#d91a1a}-3.49\%$
test_step_mdp_speed[False-True-False-True-False] 51.5570μs 20.0220μs 49.9451 KOps/s 51.3252 KOps/s $\color{#d91a1a}-2.69\%$
test_step_mdp_speed[False-True-False-False-True] 72.3150μs 21.1977μs 47.1750 KOps/s 49.6105 KOps/s $\color{#d91a1a}-4.91\%$
test_step_mdp_speed[False-True-False-False-False] 50.6740μs 13.2831μs 75.2837 KOps/s 78.0063 KOps/s $\color{#d91a1a}-3.49\%$
test_step_mdp_speed[False-False-True-True-True] 4.2380ms 33.3253μs 30.0072 KOps/s 32.1965 KOps/s $\textbf{\color{#d91a1a}-6.80\%}$
test_step_mdp_speed[False-False-True-True-False] 59.5920μs 21.6148μs 46.2647 KOps/s 47.9227 KOps/s $\color{#d91a1a}-3.46\%$
test_step_mdp_speed[False-False-True-False-True] 65.1020μs 21.0405μs 47.5275 KOps/s 50.0554 KOps/s $\textbf{\color{#d91a1a}-5.05\%}$
test_step_mdp_speed[False-False-True-False-False] 62.0160μs 13.0984μs 76.3451 KOps/s 77.9869 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[False-False-False-True-True] 81.1010μs 34.5389μs 28.9528 KOps/s 30.5040 KOps/s $\textbf{\color{#d91a1a}-5.09\%}$
test_step_mdp_speed[False-False-False-True-False] 58.4090μs 23.3883μs 42.7564 KOps/s 44.7668 KOps/s $\color{#d91a1a}-4.49\%$
test_step_mdp_speed[False-False-False-False-True] 68.6280μs 22.0245μs 45.4040 KOps/s 46.9732 KOps/s $\color{#d91a1a}-3.34\%$
test_step_mdp_speed[False-False-False-False-False] 62.9670μs 14.4966μs 68.9816 KOps/s 70.2570 KOps/s $\color{#d91a1a}-1.82\%$
test_values[generalized_advantage_estimate-True-True] 10.0674ms 9.7179ms 102.9024 Ops/s 104.4063 Ops/s $\color{#d91a1a}-1.44\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.1173ms 33.4412ms 29.9032 Ops/s 29.7445 Ops/s $\color{#35bf28}+0.53\%$
test_values[td0_return_estimate-False-False] 0.2200ms 0.1719ms 5.8181 KOps/s 5.6423 KOps/s $\color{#35bf28}+3.12\%$
test_values[td1_return_estimate-False-False] 25.1046ms 23.7206ms 42.1575 Ops/s 41.8518 Ops/s $\color{#35bf28}+0.73\%$
test_values[vec_td1_return_estimate-False-False] 43.7004ms 33.9982ms 29.4133 Ops/s 29.6831 Ops/s $\color{#d91a1a}-0.91\%$
test_values[td_lambda_return_estimate-True-False] 35.7521ms 34.3362ms 29.1238 Ops/s 29.1782 Ops/s $\color{#d91a1a}-0.19\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.8816ms 33.6015ms 29.7606 Ops/s 29.6171 Ops/s $\color{#35bf28}+0.48\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6968ms 8.5156ms 117.4313 Ops/s 116.9582 Ops/s $\color{#35bf28}+0.40\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.1702ms 2.0014ms 499.6512 Ops/s 498.2443 Ops/s $\color{#35bf28}+0.28\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7809ms 0.3675ms 2.7213 KOps/s 2.8050 KOps/s $\color{#d91a1a}-2.98\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 40.0659ms 38.7118ms 25.8319 Ops/s 21.9748 Ops/s $\textbf{\color{#35bf28}+17.55\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1517ms 3.0490ms 327.9802 Ops/s 329.8161 Ops/s $\color{#d91a1a}-0.56\%$
test_dqn_speed 1.9872ms 1.3585ms 736.1085 Ops/s 726.1531 Ops/s $\color{#35bf28}+1.37\%$
test_ddpg_speed 3.6412ms 2.8787ms 347.3809 Ops/s 343.3732 Ops/s $\color{#35bf28}+1.17\%$
test_sac_speed 11.2524ms 8.5997ms 116.2831 Ops/s 117.1962 Ops/s $\color{#d91a1a}-0.78\%$
test_redq_speed 15.0818ms 13.1419ms 76.0926 Ops/s 75.7677 Ops/s $\color{#35bf28}+0.43\%$
test_redq_deprec_speed 15.7474ms 13.1181ms 76.2306 Ops/s 75.9297 Ops/s $\color{#35bf28}+0.40\%$
test_td3_speed 8.8064ms 8.4906ms 117.7772 Ops/s 117.2847 Ops/s $\color{#35bf28}+0.42\%$
test_cql_speed 42.2459ms 36.6594ms 27.2781 Ops/s 27.2780 Ops/s $+0.00\%$
test_a2c_speed 9.5468ms 7.5238ms 132.9115 Ops/s 134.3883 Ops/s $\color{#d91a1a}-1.10\%$
test_ppo_speed 9.2569ms 7.8003ms 128.2003 Ops/s 129.2930 Ops/s $\color{#d91a1a}-0.85\%$
test_reinforce_speed 8.1709ms 6.6717ms 149.8862 Ops/s 149.1322 Ops/s $\color{#35bf28}+0.51\%$
test_iql_speed 39.4661ms 33.1830ms 30.1359 Ops/s 28.1301 Ops/s $\textbf{\color{#35bf28}+7.13\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5676ms 4.4312ms 225.6733 Ops/s 231.2031 Ops/s $\color{#d91a1a}-2.39\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6778ms 0.4769ms 2.0968 KOps/s 2.0919 KOps/s $\color{#35bf28}+0.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6662ms 0.4497ms 2.2239 KOps/s 2.2135 KOps/s $\color{#35bf28}+0.47\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7219ms 4.3148ms 231.7626 Ops/s 236.5561 Ops/s $\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6761ms 0.4721ms 2.1181 KOps/s 2.1328 KOps/s $\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6531ms 0.4473ms 2.2358 KOps/s 2.2360 KOps/s $-0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 3.0711ms 1.6942ms 590.2376 Ops/s 586.2938 Ops/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2429ms 1.6009ms 624.6594 Ops/s 618.1720 Ops/s $\color{#35bf28}+1.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8191ms 4.4994ms 222.2535 Ops/s 220.1947 Ops/s $\color{#35bf28}+0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6412ms 0.6132ms 1.6308 KOps/s 1.6083 KOps/s $\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.4488ms 0.5910ms 1.6920 KOps/s 1.6848 KOps/s $\color{#35bf28}+0.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.5682ms 4.1961ms 238.3148 Ops/s 231.2556 Ops/s $\color{#35bf28}+3.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7409ms 0.4723ms 2.1172 KOps/s 2.0577 KOps/s $\color{#35bf28}+2.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6679ms 0.4498ms 2.2230 KOps/s 2.1974 KOps/s $\color{#35bf28}+1.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.3538ms 4.1580ms 240.5018 Ops/s 230.3308 Ops/s $\color{#35bf28}+4.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7667ms 0.4703ms 2.1264 KOps/s 1.9171 KOps/s $\textbf{\color{#35bf28}+10.92\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6094ms 0.4432ms 2.2565 KOps/s 2.1976 KOps/s $\color{#35bf28}+2.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.8142ms 4.3434ms 230.2366 Ops/s 228.9744 Ops/s $\color{#35bf28}+0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8847ms 0.6118ms 1.6345 KOps/s 1.6364 KOps/s $\color{#d91a1a}-0.11\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 7.4442ms 0.5906ms 1.6931 KOps/s 1.6765 KOps/s $\color{#35bf28}+0.99\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1248s 8.2137ms 121.7478 Ops/s 167.4388 Ops/s $\textbf{\color{#d91a1a}-27.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.5846ms 13.3168ms 75.0933 Ops/s 76.6041 Ops/s $\color{#d91a1a}-1.97\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.2363ms 1.1885ms 841.3759 Ops/s 916.7436 Ops/s $\textbf{\color{#d91a1a}-8.22\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1024s 5.7410ms 174.1861 Ops/s 170.7212 Ops/s $\color{#35bf28}+2.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.6850ms 13.4217ms 74.5062 Ops/s 66.7969 Ops/s $\textbf{\color{#35bf28}+11.54\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.7850ms 1.0773ms 928.2287 Ops/s 851.5148 Ops/s $\textbf{\color{#35bf28}+9.01\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1073s 6.0025ms 166.5985 Ops/s 167.0302 Ops/s $\color{#d91a1a}-0.26\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.9352ms 13.5094ms 74.0224 Ops/s 76.3671 Ops/s $\color{#d91a1a}-3.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.3336ms 1.3034ms 767.1957 Ops/s 807.6681 Ops/s $\textbf{\color{#d91a1a}-5.01\%}$

@vmoens vmoens added enhancement New feature or request Environments Adds or modifies an environment wrapper labels Jul 22, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 22, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1052s 0.1045s 9.5694 Ops/s 9.4842 Ops/s $\color{#35bf28}+0.90\%$
test_sync 94.7172ms 94.1670ms 10.6194 Ops/s 10.6328 Ops/s $\color{#d91a1a}-0.13\%$
test_async 0.1785s 90.2401ms 11.0816 Ops/s 11.2102 Ops/s $\color{#d91a1a}-1.15\%$
test_single_pixels 0.1156s 0.1149s 8.7070 Ops/s 8.6084 Ops/s $\color{#35bf28}+1.14\%$
test_sync_pixels 77.3392ms 74.4042ms 13.4401 Ops/s 13.5161 Ops/s $\color{#d91a1a}-0.56\%$
test_async_pixels 0.1424s 70.6607ms 14.1521 Ops/s 14.2332 Ops/s $\color{#d91a1a}-0.57\%$
test_simple 0.7610s 0.7602s 1.3155 Ops/s 1.3043 Ops/s $\color{#35bf28}+0.86\%$
test_transformed 1.0815s 1.0047s 0.9953 Ops/s 1.0039 Ops/s $\color{#d91a1a}-0.85\%$
test_serial 2.2718s 2.1946s 0.4557 Ops/s 0.4537 Ops/s $\color{#35bf28}+0.44\%$
test_parallel 2.0101s 1.9368s 0.5163 Ops/s 0.5116 Ops/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-True-True-True] 0.1178ms 36.2457μs 27.5895 KOps/s 27.0654 KOps/s $\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-True-True-True-False] 78.9810μs 21.0392μs 47.5303 KOps/s 47.7099 KOps/s $\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-True-True-False-True] 0.1458ms 20.6649μs 48.3913 KOps/s 47.3290 KOps/s $\color{#35bf28}+2.24\%$
test_step_mdp_speed[True-True-True-False-False] 36.6910μs 11.8422μs 84.4436 KOps/s 84.6969 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-True-True] 80.5010μs 38.8656μs 25.7297 KOps/s 25.1477 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-True-False-True-False] 42.9410μs 23.0825μs 43.3229 KOps/s 43.0805 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[True-True-False-False-True] 79.9710μs 22.8106μs 43.8393 KOps/s 43.4505 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-True-False-False-False] 31.9210μs 14.0868μs 70.9885 KOps/s 70.8796 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[True-False-True-True-True] 68.9010μs 41.8458μs 23.8972 KOps/s 23.7058 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-False-True-True-False] 51.0410μs 25.4799μs 39.2466 KOps/s 39.4654 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[True-False-True-False-True] 0.1868ms 22.7095μs 44.0345 KOps/s 42.9724 KOps/s $\color{#35bf28}+2.47\%$
test_step_mdp_speed[True-False-True-False-False] 0.1926ms 14.0962μs 70.9412 KOps/s 71.5272 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-False-False-True-True] 73.6920μs 43.1931μs 23.1518 KOps/s 22.8016 KOps/s $\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-False-True-False] 44.2500μs 27.5705μs 36.2706 KOps/s 36.5082 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-False-False-True] 50.3110μs 24.9065μs 40.1502 KOps/s 39.4928 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-False-False-False] 33.1210μs 16.1867μs 61.7790 KOps/s 61.8911 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-True-True-True] 60.5110μs 41.1086μs 24.3258 KOps/s 24.1202 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-True-True-True-False] 44.8210μs 25.6994μs 38.9115 KOps/s 39.7322 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-True-True-False-True] 43.8210μs 26.9786μs 37.0665 KOps/s 36.9598 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-True-False-False] 83.6910μs 15.9725μs 62.6076 KOps/s 62.7187 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-False-True-True] 73.2710μs 43.2464μs 23.1233 KOps/s 22.9409 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-True-False-True-False] 52.5510μs 27.5592μs 36.2855 KOps/s 36.2551 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-True-False-False-True] 91.4410μs 29.1384μs 34.3190 KOps/s 34.1378 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-False-False-False] 35.5610μs 18.0830μs 55.3005 KOps/s 54.1674 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[False-False-True-True-True] 4.8987ms 46.2856μs 21.6050 KOps/s 21.3591 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[False-False-True-True-False] 56.5710μs 30.1124μs 33.2089 KOps/s 33.0666 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-True-False-True] 51.3720μs 29.3563μs 34.0642 KOps/s 33.7506 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-False-True-False-False] 35.0010μs 18.1430μs 55.1176 KOps/s 53.7970 KOps/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[False-False-False-True-True] 0.1221ms 47.2721μs 21.1541 KOps/s 21.0205 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-False-True-False] 70.4410μs 32.0114μs 31.2388 KOps/s 31.0353 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-False-False-False-True] 75.9110μs 30.6674μs 32.6079 KOps/s 31.9490 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-False-False-False-False] 40.6400μs 20.2107μs 49.4788 KOps/s 49.2138 KOps/s $\color{#35bf28}+0.54\%$
test_values[generalized_advantage_estimate-True-True] 25.1394ms 24.2981ms 41.1554 Ops/s 40.2141 Ops/s $\color{#35bf28}+2.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 97.3219ms 2.8393ms 352.2044 Ops/s 371.3538 Ops/s $\textbf{\color{#d91a1a}-5.16\%}$
test_values[td0_return_estimate-False-False] 88.0220μs 65.3314μs 15.3066 KOps/s 15.1974 KOps/s $\color{#35bf28}+0.72\%$
test_values[td1_return_estimate-False-False] 59.6481ms 55.2703ms 18.0929 Ops/s 18.1477 Ops/s $\color{#d91a1a}-0.30\%$
test_values[vec_td1_return_estimate-False-False] 1.4693ms 1.1060ms 904.1997 Ops/s 918.0831 Ops/s $\color{#d91a1a}-1.51\%$
test_values[td_lambda_return_estimate-True-False] 94.7405ms 89.5573ms 11.1660 Ops/s 11.5534 Ops/s $\color{#d91a1a}-3.35\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4745ms 1.0827ms 923.6064 Ops/s 930.0468 Ops/s $\color{#d91a1a}-0.69\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.7933ms 24.1418ms 41.4219 Ops/s 40.9624 Ops/s $\color{#35bf28}+1.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9274ms 0.7072ms 1.4140 KOps/s 1.3806 KOps/s $\color{#35bf28}+2.42\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8169ms 0.6614ms 1.5119 KOps/s 1.4980 KOps/s $\color{#35bf28}+0.92\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.7011ms 1.4541ms 687.7115 Ops/s 681.7935 Ops/s $\color{#35bf28}+0.87\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8605ms 0.6745ms 1.4825 KOps/s 1.4691 KOps/s $\color{#35bf28}+0.92\%$
test_dqn_speed 7.9976ms 1.4509ms 689.2254 Ops/s 688.4917 Ops/s $\color{#35bf28}+0.11\%$
test_ddpg_speed 3.2002ms 2.9653ms 337.2381 Ops/s 338.3535 Ops/s $\color{#d91a1a}-0.33\%$
test_sac_speed 9.0138ms 8.4970ms 117.6887 Ops/s 118.2006 Ops/s $\color{#d91a1a}-0.43\%$
test_redq_speed 12.8761ms 11.1869ms 89.3901 Ops/s 91.2153 Ops/s $\color{#d91a1a}-2.00\%$
test_redq_deprec_speed 12.5370ms 11.7705ms 84.9585 Ops/s 86.8403 Ops/s $\color{#d91a1a}-2.17\%$
test_td3_speed 8.5957ms 8.4838ms 117.8721 Ops/s 116.9467 Ops/s $\color{#35bf28}+0.79\%$
test_cql_speed 27.5610ms 26.8292ms 37.2729 Ops/s 37.4922 Ops/s $\color{#d91a1a}-0.58\%$
test_a2c_speed 6.1940ms 5.8819ms 170.0122 Ops/s 174.4415 Ops/s $\color{#d91a1a}-2.54\%$
test_ppo_speed 6.9374ms 6.2512ms 159.9687 Ops/s 166.1854 Ops/s $\color{#d91a1a}-3.74\%$
test_reinforce_speed 5.1820ms 4.8024ms 208.2289 Ops/s 211.4878 Ops/s $\color{#d91a1a}-1.54\%$
test_iql_speed 21.8923ms 20.8485ms 47.9650 Ops/s 49.3395 Ops/s $\color{#d91a1a}-2.79\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0313ms 5.8516ms 170.8947 Ops/s 172.0070 Ops/s $\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7451ms 0.5266ms 1.8990 KOps/s 1.8786 KOps/s $\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7263ms 0.5068ms 1.9731 KOps/s 1.9453 KOps/s $\color{#35bf28}+1.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0898ms 5.7866ms 172.8145 Ops/s 174.5521 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0344ms 0.5165ms 1.9360 KOps/s 1.9089 KOps/s $\color{#35bf28}+1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7031ms 0.5021ms 1.9918 KOps/s 1.9787 KOps/s $\color{#35bf28}+0.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1621ms 1.9780ms 505.5637 Ops/s 491.1467 Ops/s $\color{#35bf28}+2.94\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0879ms 1.8952ms 527.6524 Ops/s 513.6753 Ops/s $\color{#35bf28}+2.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0862ms 5.9101ms 169.2018 Ops/s 169.1085 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1540s 0.8388ms 1.1922 KOps/s 1.4820 KOps/s $\textbf{\color{#d91a1a}-19.56\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8185ms 0.6552ms 1.5263 KOps/s 1.5318 KOps/s $\color{#d91a1a}-0.36\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.0465ms 5.8341ms 171.4050 Ops/s 173.8461 Ops/s $\color{#d91a1a}-1.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6862ms 0.5240ms 1.9083 KOps/s 1.8910 KOps/s $\color{#35bf28}+0.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6938ms 0.5069ms 1.9729 KOps/s 1.9569 KOps/s $\color{#35bf28}+0.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1045ms 5.7475ms 173.9891 Ops/s 173.9422 Ops/s $\color{#35bf28}+0.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6759ms 0.5185ms 1.9287 KOps/s 1.9276 KOps/s $\color{#35bf28}+0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.4179ms 0.5082ms 1.9676 KOps/s 1.9801 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2555ms 5.9908ms 166.9222 Ops/s 169.9373 Ops/s $\color{#d91a1a}-1.77\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1975ms 0.6775ms 1.4760 KOps/s 1.4651 KOps/s $\color{#35bf28}+0.75\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8561ms 0.6525ms 1.5325 KOps/s 1.5241 KOps/s $\color{#35bf28}+0.55\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1577s 8.2984ms 120.5045 Ops/s 123.6297 Ops/s $\color{#d91a1a}-2.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.2857ms 15.7911ms 63.3269 Ops/s 63.6579 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.4839ms 1.3780ms 725.7143 Ops/s 797.1342 Ops/s $\textbf{\color{#d91a1a}-8.96\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1339s 7.8001ms 128.2041 Ops/s 127.9255 Ops/s $\color{#35bf28}+0.22\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.7768ms 15.8799ms 62.9725 Ops/s 64.1981 Ops/s $\color{#d91a1a}-1.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.2390ms 1.2588ms 794.4214 Ops/s 737.3893 Ops/s $\textbf{\color{#35bf28}+7.73\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1433s 10.7953ms 92.6327 Ops/s 124.5188 Ops/s $\textbf{\color{#d91a1a}-25.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.2606ms 15.8775ms 62.9822 Ops/s 63.3780 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4199ms 1.4178ms 705.3027 Ops/s 716.2566 Ops/s $\color{#d91a1a}-1.53\%$

@vmoens
Update
a689eed 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] TicTacToeEnv
e2c329f 
ghstack-source-id: a48a28b322af074877e9a66261310ba41a9599f0
Pull Request resolved: #2301
@vmoens vmoens merged commit a689eed into gh/vmoens/1/base Jul 22, 2024
38 of 45 checks passed
vmoens added a commit that referenced this pull request Jul 22, 2024
@vmoens
[Feature] TicTacToeEnv
e28440f 
ghstack-source-id: a48a28b322af074877e9a66261310ba41a9599f0
Pull Request resolved: #2301
@vmoens vmoens deleted the gh/vmoens/1/head branch July 22, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request Environments Adds or modifies an environment wrapper
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot