Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Construct loss by providing lists of modules #2311

Merged
merged 3 commits into from
Jul 23, 2024
Merged

[Feature] Construct loss by providing lists of modules #2311

merged 3 commits into from
Jul 23, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 23, 2024 •
edited
Loading

@vmoens
Update
a4cdd5e 
[ghstack-poisoned]
This was referenced Jul 23, 2024
Merged
Merged
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 23, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2311

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending, 13 Unrelated Failures

As of commit 0cb0d40 with merge base 59c3374 (image):

NEW FAILURE - The following job has failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Jul 23, 2024
@vmoens
[Feature] Construct loss by providing lists of modules
8406130 
ghstack-source-id: 8b6fbc87aa9efba6139008d1f8472ea53bd7f8b0
Pull Request resolved: #2311
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 23, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.8789ms 57.7394ms 17.3192 Ops/s 16.9649 Ops/s $\color{#35bf28}+2.09\%$
test_sync 38.3675ms 32.0204ms 31.2301 Ops/s 31.0512 Ops/s $\color{#35bf28}+0.58\%$
test_async 52.0907ms 29.9091ms 33.4346 Ops/s 32.7333 Ops/s $\color{#35bf28}+2.14\%$
test_simple 0.4887s 0.4146s 2.4119 Ops/s 2.4278 Ops/s $\color{#d91a1a}-0.66\%$
test_transformed 0.6428s 0.5741s 1.7418 Ops/s 1.7689 Ops/s $\color{#d91a1a}-1.53\%$
test_serial 1.3553s 1.2763s 0.7835 Ops/s 0.7806 Ops/s $\color{#35bf28}+0.37\%$
test_parallel 1.1809s 1.1133s 0.8982 Ops/s 0.8813 Ops/s $\color{#35bf28}+1.92\%$
test_step_mdp_speed[True-True-True-True-True] 0.2083ms 24.7873μs 40.3432 KOps/s 40.3558 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-True-True-True-False] 42.3400μs 14.5768μs 68.6023 KOps/s 69.2075 KOps/s $\color{#d91a1a}-0.87\%$
test_step_mdp_speed[True-True-True-False-True] 43.5020μs 14.2643μs 70.1052 KOps/s 69.5988 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-True-True-False-False] 44.9140μs 8.3704μs 119.4687 KOps/s 117.7444 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[True-True-False-True-True] 0.3847ms 27.1628μs 36.8151 KOps/s 37.7326 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-True-False-True-False] 57.0070μs 16.0638μs 62.2519 KOps/s 62.7476 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-False-False-True] 63.4390μs 16.1603μs 61.8799 KOps/s 63.4396 KOps/s $\color{#d91a1a}-2.46\%$
test_step_mdp_speed[True-True-False-False-False] 35.8180μs 9.9731μs 100.2702 KOps/s 101.4054 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-False-True-True-True] 61.2950μs 28.2405μs 35.4101 KOps/s 35.5046 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-True-True-False] 43.7020μs 17.9008μs 55.8633 KOps/s 56.7927 KOps/s $\color{#d91a1a}-1.64\%$
test_step_mdp_speed[True-False-True-False-True] 72.4560μs 16.0409μs 62.3405 KOps/s 62.8048 KOps/s $\color{#d91a1a}-0.74\%$
test_step_mdp_speed[True-False-True-False-False] 0.1968ms 10.0846μs 99.1608 KOps/s 100.5393 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-False-False-True-True] 64.9820μs 29.9612μs 33.3765 KOps/s 33.9834 KOps/s $\color{#d91a1a}-1.79\%$
test_step_mdp_speed[True-False-False-True-False] 68.6490μs 19.4944μs 51.2967 KOps/s 52.5224 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[True-False-False-False-True] 59.4110μs 17.4895μs 57.1772 KOps/s 57.7870 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-False-False-False-False] 51.4770μs 11.4602μs 87.2583 KOps/s 87.9424 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[False-True-True-True-True] 78.9580μs 28.6321μs 34.9259 KOps/s 35.5059 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-True-True-True-False] 54.7130μs 17.9431μs 55.7317 KOps/s 56.7647 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[False-True-True-False-True] 91.7220μs 18.6420μs 53.6422 KOps/s 52.2580 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-True-True-False-False] 60.7340μs 11.2380μs 88.9836 KOps/s 90.8272 KOps/s $\color{#d91a1a}-2.03\%$
test_step_mdp_speed[False-True-False-True-True] 66.3650μs 29.4297μs 33.9793 KOps/s 34.0306 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-False-True-False] 59.7830μs 19.3552μs 51.6657 KOps/s 52.4393 KOps/s $\color{#d91a1a}-1.48\%$
test_step_mdp_speed[False-True-False-False-True] 58.5500μs 20.3246μs 49.2015 KOps/s 51.4814 KOps/s $\color{#d91a1a}-4.43\%$
test_step_mdp_speed[False-True-False-False-False] 66.0020μs 12.7618μs 78.3588 KOps/s 80.7667 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[False-False-True-True-True] 3.7114ms 31.0096μs 32.2480 KOps/s 32.1461 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-False-True-True-False] 54.0410μs 20.7112μs 48.2831 KOps/s 48.8096 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-False-True-False-True] 72.2870μs 19.5261μs 51.2135 KOps/s 50.8805 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-False-True-False-False] 61.7360μs 12.6662μs 78.9503 KOps/s 80.3186 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[False-False-False-True-True] 83.1560μs 32.6332μs 30.6436 KOps/s 30.9674 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[False-False-False-True-False] 58.2590μs 21.9979μs 45.4588 KOps/s 45.2328 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-False-False-True] 54.8020μs 20.9639μs 47.7011 KOps/s 48.0024 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-False-False-False] 49.9640μs 13.9478μs 71.6961 KOps/s 72.0629 KOps/s $\color{#d91a1a}-0.51\%$
test_values[generalized_advantage_estimate-True-True] 11.1893ms 9.4631ms 105.6740 Ops/s 108.6256 Ops/s $\color{#d91a1a}-2.72\%$
test_values[vec_generalized_advantage_estimate-True-True] 38.4336ms 33.7583ms 29.6224 Ops/s 27.5026 Ops/s $\textbf{\color{#35bf28}+7.71\%}$
test_values[td0_return_estimate-False-False] 0.2331ms 0.1673ms 5.9785 KOps/s 6.4356 KOps/s $\textbf{\color{#d91a1a}-7.10\%}$
test_values[td1_return_estimate-False-False] 24.2562ms 23.5175ms 42.5215 Ops/s 43.5727 Ops/s $\color{#d91a1a}-2.41\%$
test_values[vec_td1_return_estimate-False-False] 35.7683ms 33.7124ms 29.6627 Ops/s 27.3898 Ops/s $\textbf{\color{#35bf28}+8.30\%}$
test_values[td_lambda_return_estimate-True-False] 37.7402ms 34.1084ms 29.3183 Ops/s 30.1687 Ops/s $\color{#d91a1a}-2.82\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.8095ms 33.7522ms 29.6277 Ops/s 27.2068 Ops/s $\textbf{\color{#35bf28}+8.90\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.7119ms 8.2817ms 120.7475 Ops/s 125.1234 Ops/s $\color{#d91a1a}-3.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2271ms 1.9591ms 510.4326 Ops/s 502.9047 Ops/s $\color{#35bf28}+1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6561ms 0.3597ms 2.7805 KOps/s 2.8121 KOps/s $\color{#d91a1a}-1.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 41.9795ms 40.6826ms 24.5806 Ops/s 21.6825 Ops/s $\textbf{\color{#35bf28}+13.37\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8381ms 3.0262ms 330.4484 Ops/s 332.2761 Ops/s $\color{#d91a1a}-0.55\%$
test_dqn_speed 1.8484ms 1.3828ms 723.1460 Ops/s 715.9157 Ops/s $\color{#35bf28}+1.01\%$
test_ddpg_speed 3.7873ms 2.9386ms 340.2928 Ops/s 343.4403 Ops/s $\color{#d91a1a}-0.92\%$
test_sac_speed 10.0926ms 8.6281ms 115.9001 Ops/s 117.8942 Ops/s $\color{#d91a1a}-1.69\%$
test_redq_speed 14.6131ms 13.4923ms 74.1162 Ops/s 71.7569 Ops/s $\color{#35bf28}+3.29\%$
test_redq_deprec_speed 14.8832ms 13.7015ms 72.9845 Ops/s 75.6199 Ops/s $\color{#d91a1a}-3.49\%$
test_td3_speed 8.7545ms 8.4531ms 118.2997 Ops/s 118.0804 Ops/s $\color{#35bf28}+0.19\%$
test_cql_speed 38.5134ms 36.5597ms 27.3525 Ops/s 27.7987 Ops/s $\color{#d91a1a}-1.61\%$
test_a2c_speed 8.5651ms 7.4220ms 134.7338 Ops/s 132.4915 Ops/s $\color{#35bf28}+1.69\%$
test_ppo_speed 8.8511ms 7.7291ms 129.3813 Ops/s 125.6152 Ops/s $\color{#35bf28}+3.00\%$
test_reinforce_speed 8.0128ms 6.5979ms 151.5629 Ops/s 151.9280 Ops/s $\color{#d91a1a}-0.24\%$
test_iql_speed 34.5665ms 32.3574ms 30.9048 Ops/s 30.6730 Ops/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4905ms 4.9076ms 203.7664 Ops/s 204.0627 Ops/s $\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6914ms 0.4846ms 2.0635 KOps/s 2.0357 KOps/s $\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6339ms 0.4597ms 2.1755 KOps/s 2.1593 KOps/s $\color{#35bf28}+0.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9931ms 4.8759ms 205.0887 Ops/s 204.4616 Ops/s $\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.1206s 0.5680ms 1.7605 KOps/s 2.0685 KOps/s $\textbf{\color{#d91a1a}-14.89\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6761ms 0.4558ms 2.1939 KOps/s 2.1433 KOps/s $\color{#35bf28}+2.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 3.4936ms 1.7511ms 571.0859 Ops/s 571.4235 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1654ms 1.6139ms 619.6301 Ops/s 625.4639 Ops/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.3544ms 5.1093ms 195.7214 Ops/s 198.5567 Ops/s $\color{#d91a1a}-1.43\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9088ms 0.6335ms 1.5785 KOps/s 1.5992 KOps/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1586ms 0.6095ms 1.6406 KOps/s 1.6690 KOps/s $\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1010ms 4.9377ms 202.5229 Ops/s 203.4601 Ops/s $\color{#d91a1a}-0.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2182ms 0.4850ms 2.0617 KOps/s 2.0723 KOps/s $\color{#d91a1a}-0.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7601ms 0.4655ms 2.1482 KOps/s 2.1260 KOps/s $\color{#35bf28}+1.04\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9882ms 5.0342ms 198.6396 Ops/s 200.6013 Ops/s $\color{#d91a1a}-0.98\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8194ms 0.4939ms 2.0247 KOps/s 2.0808 KOps/s $\color{#d91a1a}-2.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7086ms 0.4707ms 2.1244 KOps/s 2.1848 KOps/s $\color{#d91a1a}-2.76\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.3931ms 5.1024ms 195.9862 Ops/s 199.3332 Ops/s $\color{#d91a1a}-1.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7526ms 0.6217ms 1.6084 KOps/s 1.5845 KOps/s $\color{#35bf28}+1.51\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6928ms 0.6019ms 1.6614 KOps/s 1.6427 KOps/s $\color{#35bf28}+1.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1303s 6.3276ms 158.0388 Ops/s 110.0774 Ops/s $\textbf{\color{#35bf28}+43.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.5011ms 12.9355ms 77.3065 Ops/s 78.0222 Ops/s $\color{#d91a1a}-0.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.7633ms 1.1963ms 835.8797 Ops/s 846.8311 Ops/s $\color{#d91a1a}-1.29\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1293s 6.2625ms 159.6796 Ops/s 163.9864 Ops/s $\color{#d91a1a}-2.63\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.4211ms 13.0336ms 76.7248 Ops/s 77.1361 Ops/s $\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.8618ms 1.1433ms 874.6638 Ops/s 842.5641 Ops/s $\color{#35bf28}+3.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1243s 8.7459ms 114.3399 Ops/s 148.0767 Ops/s $\textbf{\color{#d91a1a}-22.78\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.0665ms 13.3333ms 75.0003 Ops/s 75.7352 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.7612ms 1.2479ms 801.3191 Ops/s 835.7314 Ops/s $\color{#d91a1a}-4.12\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 23, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1069s 0.1054s 9.4858 Ops/s 8.6395 Ops/s $\textbf{\color{#35bf28}+9.80\%}$
test_sync 95.1722ms 93.2641ms 10.7222 Ops/s 10.5230 Ops/s $\color{#35bf28}+1.89\%$
test_async 0.1788s 89.7218ms 11.1456 Ops/s 11.2618 Ops/s $\color{#d91a1a}-1.03\%$
test_single_pixels 0.1158s 0.1155s 8.6577 Ops/s 8.6060 Ops/s $\color{#35bf28}+0.60\%$
test_sync_pixels 76.4797ms 75.0574ms 13.3231 Ops/s 13.5416 Ops/s $\color{#d91a1a}-1.61\%$
test_async_pixels 0.1396s 68.2166ms 14.6592 Ops/s 14.4204 Ops/s $\color{#35bf28}+1.66\%$
test_simple 0.8600s 0.7838s 1.2759 Ops/s 1.2404 Ops/s $\color{#35bf28}+2.86\%$
test_transformed 1.0813s 1.0067s 0.9934 Ops/s 0.9814 Ops/s $\color{#35bf28}+1.22\%$
test_serial 2.2684s 2.1977s 0.4550 Ops/s 0.4415 Ops/s $\color{#35bf28}+3.07\%$
test_parallel 2.0260s 1.9647s 0.5090 Ops/s 0.5093 Ops/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-True-True-True-True] 0.1115ms 36.9381μs 27.0723 KOps/s 26.0178 KOps/s $\color{#35bf28}+4.05\%$
test_step_mdp_speed[True-True-True-True-False] 46.3030μs 21.4017μs 46.7253 KOps/s 46.8724 KOps/s $\color{#d91a1a}-0.31\%$
test_step_mdp_speed[True-True-True-False-True] 0.1166ms 21.4389μs 46.6441 KOps/s 46.5876 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-True-True-False-False] 29.9820μs 12.1187μs 82.5174 KOps/s 82.6347 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-True-False-True-True] 70.1340μs 39.5470μs 25.2863 KOps/s 24.5125 KOps/s $\color{#35bf28}+3.16\%$
test_step_mdp_speed[True-True-False-True-False] 44.1630μs 23.4363μs 42.6689 KOps/s 42.3632 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-True] 47.3120μs 23.7296μs 42.1414 KOps/s 41.6255 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-False-False-False] 39.7620μs 14.4291μs 69.3046 KOps/s 69.3692 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-False-True-True-True] 76.5040μs 42.5370μs 23.5090 KOps/s 23.6151 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-True-True-False] 56.1130μs 25.9725μs 38.5023 KOps/s 38.5963 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-False-True-False-True] 47.4430μs 23.6667μs 42.2534 KOps/s 41.7775 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[True-False-True-False-False] 36.1120μs 14.4451μs 69.2276 KOps/s 69.8576 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-True-True] 71.5440μs 44.1371μs 22.6567 KOps/s 22.1805 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[True-False-False-True-False] 54.6130μs 28.0439μs 35.6583 KOps/s 35.2815 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-False-False-False-True] 0.2197ms 25.9139μs 38.5893 KOps/s 37.8856 KOps/s $\color{#35bf28}+1.86\%$
test_step_mdp_speed[True-False-False-False-False] 0.1281ms 16.7430μs 59.7266 KOps/s 59.5675 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-True-True-True] 0.2471ms 41.8706μs 23.8831 KOps/s 23.4197 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[False-True-True-True-False] 0.1158ms 25.9442μs 38.5443 KOps/s 38.3155 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-True-True-False-True] 61.4240μs 27.9665μs 35.7571 KOps/s 35.5660 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-True-True-False-False] 0.2185ms 17.0272μs 58.7294 KOps/s 60.3081 KOps/s $\color{#d91a1a}-2.62\%$
test_step_mdp_speed[False-True-False-True-True] 71.2140μs 44.0436μs 22.7048 KOps/s 22.1819 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-True-False-True-False] 0.1164ms 28.0234μs 35.6845 KOps/s 35.2718 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-True-False-False-True] 56.1330μs 30.0514μs 33.2763 KOps/s 32.7402 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-False-False-False] 42.4220μs 18.7108μs 53.4451 KOps/s 53.2745 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-False-True-True-True] 4.1187ms 47.2059μs 21.1838 KOps/s 20.9906 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-True-True-False] 59.6430μs 30.7186μs 32.5536 KOps/s 32.3367 KOps/s $\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-False-True] 55.6630μs 30.0332μs 33.2965 KOps/s 32.6825 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-True-False-False] 39.9020μs 18.8351μs 53.0922 KOps/s 53.5458 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[False-False-False-True-True] 85.8450μs 48.5958μs 20.5779 KOps/s 20.3597 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-False-False-True-False] 59.4230μs 32.6473μs 30.6304 KOps/s 30.4548 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-False-False-True] 0.1883ms 32.1741μs 31.0809 KOps/s 31.0069 KOps/s $\color{#35bf28}+0.24\%$
test_step_mdp_speed[False-False-False-False-False] 43.6230μs 20.9533μs 47.7251 KOps/s 48.6977 KOps/s $\color{#d91a1a}-2.00\%$
test_values[generalized_advantage_estimate-True-True] 23.9387ms 23.6067ms 42.3609 Ops/s 40.9440 Ops/s $\color{#35bf28}+3.46\%$
test_values[vec_generalized_advantage_estimate-True-True] 96.0680ms 2.8083ms 356.0914 Ops/s 366.2589 Ops/s $\color{#d91a1a}-2.78\%$
test_values[td0_return_estimate-False-False] 0.1023ms 63.9518μs 15.6368 KOps/s 15.3800 KOps/s $\color{#35bf28}+1.67\%$
test_values[td1_return_estimate-False-False] 54.3601ms 53.3809ms 18.7333 Ops/s 18.2502 Ops/s $\color{#35bf28}+2.65\%$
test_values[vec_td1_return_estimate-False-False] 1.3884ms 1.0676ms 936.6648 Ops/s 928.8654 Ops/s $\color{#35bf28}+0.84\%$
test_values[td_lambda_return_estimate-True-False] 87.2138ms 83.9454ms 11.9125 Ops/s 11.0288 Ops/s $\textbf{\color{#35bf28}+8.01\%}$
test_values[vec_td_lambda_return_estimate-True-False] 1.4313ms 1.0652ms 938.8306 Ops/s 934.0088 Ops/s $\color{#35bf28}+0.52\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0106ms 23.6987ms 42.1964 Ops/s 38.9062 Ops/s $\textbf{\color{#35bf28}+8.46\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9233ms 0.7249ms 1.3795 KOps/s 1.4001 KOps/s $\color{#d91a1a}-1.47\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8319ms 0.6748ms 1.4818 KOps/s 1.4711 KOps/s $\color{#35bf28}+0.73\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5999ms 1.4509ms 689.2474 Ops/s 682.9488 Ops/s $\color{#35bf28}+0.92\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8169ms 0.6654ms 1.5029 KOps/s 1.4848 KOps/s $\color{#35bf28}+1.21\%$
test_dqn_speed 7.4665ms 1.4312ms 698.7035 Ops/s 704.7610 Ops/s $\color{#d91a1a}-0.86\%$
test_ddpg_speed 3.2979ms 2.9220ms 342.2329 Ops/s 337.3966 Ops/s $\color{#35bf28}+1.43\%$
test_sac_speed 0.1040s 9.0511ms 110.4832 Ops/s 120.1503 Ops/s $\textbf{\color{#d91a1a}-8.05\%}$
test_redq_speed 11.8115ms 10.9706ms 91.1527 Ops/s 91.0933 Ops/s $\color{#35bf28}+0.07\%$
test_redq_deprec_speed 11.9487ms 11.0694ms 90.3389 Ops/s 85.2064 Ops/s $\textbf{\color{#35bf28}+6.02\%}$
test_td3_speed 9.7699ms 8.2725ms 120.8827 Ops/s 120.8605 Ops/s $\color{#35bf28}+0.02\%$
test_cql_speed 27.5475ms 26.3064ms 38.0135 Ops/s 34.9076 Ops/s $\textbf{\color{#35bf28}+8.90\%}$
test_a2c_speed 5.9209ms 5.6793ms 176.0777 Ops/s 171.4569 Ops/s $\color{#35bf28}+2.69\%$
test_ppo_speed 6.5498ms 5.9902ms 166.9404 Ops/s 161.9935 Ops/s $\color{#35bf28}+3.05\%$
test_reinforce_speed 6.3261ms 4.6343ms 215.7806 Ops/s 209.0726 Ops/s $\color{#35bf28}+3.21\%$
test_iql_speed 20.4878ms 19.9703ms 50.0745 Ops/s 49.0206 Ops/s $\color{#35bf28}+2.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.9489ms 6.7093ms 149.0472 Ops/s 149.9728 Ops/s $\color{#d91a1a}-0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9183ms 0.5197ms 1.9243 KOps/s 1.9208 KOps/s $\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6952ms 0.4971ms 2.0118 KOps/s 1.9980 KOps/s $\color{#35bf28}+0.69\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9969ms 6.6175ms 151.1151 Ops/s 151.8288 Ops/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7463ms 0.5138ms 1.9463 KOps/s 1.9524 KOps/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6406ms 0.4924ms 2.0311 KOps/s 2.0230 KOps/s $\color{#35bf28}+0.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1592ms 1.9369ms 516.2931 Ops/s 503.9648 Ops/s $\color{#35bf28}+2.45\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2055ms 1.8431ms 542.5586 Ops/s 529.3259 Ops/s $\color{#35bf28}+2.50\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.7056ms 6.9798ms 143.2705 Ops/s 146.5347 Ops/s $\color{#d91a1a}-2.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1475s 0.7872ms 1.2702 KOps/s 1.4942 KOps/s $\textbf{\color{#d91a1a}-14.99\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8349ms 0.6376ms 1.5685 KOps/s 1.4950 KOps/s $\color{#35bf28}+4.91\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8673ms 6.6922ms 149.4272 Ops/s 149.5076 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5577ms 0.5160ms 1.9379 KOps/s 1.9314 KOps/s $\color{#35bf28}+0.34\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7022ms 0.4968ms 2.0128 KOps/s 1.9961 KOps/s $\color{#35bf28}+0.83\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9675ms 6.6138ms 151.1990 Ops/s 150.8025 Ops/s $\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7001ms 0.5135ms 1.9474 KOps/s 1.9470 KOps/s $\color{#35bf28}+0.02\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.2613ms 0.4971ms 2.0119 KOps/s 2.0324 KOps/s $\color{#d91a1a}-1.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1517ms 6.8573ms 145.8297 Ops/s 146.4733 Ops/s $\color{#d91a1a}-0.44\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4212ms 0.6679ms 1.4973 KOps/s 1.1882 KOps/s $\textbf{\color{#35bf28}+26.01\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8457ms 0.6415ms 1.5588 KOps/s 1.5485 KOps/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1483s 8.0923ms 123.5745 Ops/s 130.6899 Ops/s $\textbf{\color{#d91a1a}-5.44\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.3290ms 16.2900ms 61.3873 Ops/s 60.6852 Ops/s $\color{#35bf28}+1.16\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.1800ms 1.3505ms 740.4644 Ops/s 857.8244 Ops/s $\textbf{\color{#d91a1a}-13.68\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1276s 7.6639ms 130.4819 Ops/s 131.8090 Ops/s $\color{#d91a1a}-1.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.7338ms 16.2771ms 61.4359 Ops/s 60.6796 Ops/s $\color{#35bf28}+1.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.4002ms 1.2694ms 787.7441 Ops/s 751.6010 Ops/s $\color{#35bf28}+4.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1299s 10.4225ms 95.9459 Ops/s 128.6270 Ops/s $\textbf{\color{#d91a1a}-25.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.2199ms 16.3671ms 61.0980 Ops/s 59.9528 Ops/s $\color{#35bf28}+1.91\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.6534ms 1.4289ms 699.8395 Ops/s 783.1654 Ops/s $\textbf{\color{#d91a1a}-10.64\%}$

@vmoens
Update
fc6a57d 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 23, 2024
@vmoens
[Feature] Construct loss by providing lists of modules
a0c2a3f 
ghstack-source-id: 439e9ba3a1fbba044891b28bcd0b61001c2b514a
Pull Request resolved: #2311
@vmoens
Update
0cb0d40 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jul 23, 2024
@vmoens
[Feature] Construct loss by providing lists of modules
6756ac6 
ghstack-source-id: a8eae5312f3a31f0a2d66798c734d7b0bd0bddd6
Pull Request resolved: #2311
@vmoens vmoens merged commit 0cb0d40 into gh/vmoens/7/base Jul 23, 2024
29 of 35 checks passed
vmoens added a commit that referenced this pull request Jul 23, 2024
@vmoens
[Feature] Construct loss by providing lists of modules
f840a1a 
ghstack-source-id: a8eae5312f3a31f0a2d66798c734d7b0bd0bddd6
Pull Request resolved: #2311
@vmoens vmoens deleted the gh/vmoens/7/head branch July 23, 2024 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot