Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix TD3 and compat with https://github.com/pytorch-labs/tensordict/pull/482 #1375

Merged
merged 2 commits into from
Jul 9, 2023
Merged

[BugFix] Fix TD3 and compat with https://github.com/pytorch-labs/tensordict/pull/482 #1375

merged 2 commits into from
Jul 9, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 9, 2023

cc @BY571

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 9, 2023
@vmoens
Merge remote-tracking branch 'origin/main' into fix_td3
34763b8 
# Conflicts:
#	torchrl/envs/utils.py
@github-actions
Copy link

github-actions bot commented Jul 9, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1200s 0.1195s 8.3697 Ops/s 8.3868 Ops/s $\color{#d91a1a}-0.20\%$
test_sync 67.7217ms 62.9461ms 15.8866 Ops/s 15.8880 Ops/s $-0.01\%$
test_async 0.1222s 61.2735ms 16.3203 Ops/s 16.0427 Ops/s $\color{#35bf28}+1.73\%$
test_simple 0.6069s 0.5489s 1.8219 Ops/s 1.8391 Ops/s $\color{#d91a1a}-0.93\%$
test_transformed 1.4234s 1.3727s 0.7285 Ops/s 0.7347 Ops/s $\color{#d91a1a}-0.84\%$
test_serial 1.8005s 1.7510s 0.5711 Ops/s 0.5770 Ops/s $\color{#d91a1a}-1.02\%$
test_parallel 1.7005s 1.4964s 0.6683 Ops/s 0.6493 Ops/s $\color{#35bf28}+2.91\%$
test_step_mdp_speed[True-True-True-True-True] 0.3074ms 41.5410μs 24.0726 KOps/s 23.7676 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[True-True-True-True-False] 0.3408ms 23.6753μs 42.2382 KOps/s 42.0692 KOps/s $\color{#35bf28}+0.40\%$
test_step_mdp_speed[True-True-True-False-True] 0.2079ms 29.2554μs 34.1817 KOps/s 33.5353 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[True-True-True-False-False] 0.1628ms 16.5062μs 60.5832 KOps/s 60.6493 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-True-False-True-True] 0.1017ms 43.0680μs 23.2191 KOps/s 23.0484 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-False-True-False] 0.1823ms 25.4877μs 39.2346 KOps/s 39.2870 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[True-True-False-False-True] 59.9000μs 30.9196μs 32.3419 KOps/s 31.9097 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-True-False-False-False] 70.1000μs 18.4012μs 54.3443 KOps/s 54.7541 KOps/s $\color{#d91a1a}-0.75\%$
test_step_mdp_speed[True-False-True-True-True] 70.7000μs 44.5482μs 22.4476 KOps/s 22.2935 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-True-True-False] 84.7000μs 26.9182μs 37.1496 KOps/s 36.7633 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[True-False-True-False-True] 57.1990μs 30.5368μs 32.7473 KOps/s 31.9136 KOps/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-True-False-False] 70.0000μs 17.9261μs 55.7847 KOps/s 55.0607 KOps/s $\color{#35bf28}+1.31\%$
test_step_mdp_speed[True-False-False-True-True] 0.1116ms 46.3686μs 21.5663 KOps/s 21.3350 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[True-False-False-True-False] 83.1000μs 28.8579μs 34.6526 KOps/s 34.8806 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-False-False-True] 83.3000μs 32.3620μs 30.9005 KOps/s 30.1900 KOps/s $\color{#35bf28}+2.35\%$
test_step_mdp_speed[True-False-False-False-False] 37.5000μs 19.6903μs 50.7864 KOps/s 50.4624 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-True-True-True-True] 95.4990μs 44.6726μs 22.3851 KOps/s 22.1190 KOps/s $\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-True-True-True-False] 56.0000μs 27.2944μs 36.6376 KOps/s 36.9035 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[False-True-True-False-True] 51.8000μs 35.7623μs 27.9624 KOps/s 27.5341 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-True-True-False-False] 82.3000μs 20.1098μs 49.7270 KOps/s 49.4262 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-True-False-True-True] 75.3000μs 46.2648μs 21.6147 KOps/s 21.1024 KOps/s $\color{#35bf28}+2.43\%$
test_step_mdp_speed[False-True-False-True-False] 0.1781ms 29.0079μs 34.4734 KOps/s 34.8378 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[False-True-False-False-True] 0.1037ms 37.5312μs 26.6445 KOps/s 26.4246 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-False-False-False] 78.0000μs 21.7850μs 45.9031 KOps/s 45.5624 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-True-True-True] 97.4990μs 47.6679μs 20.9785 KOps/s 20.5503 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[False-False-True-True-False] 57.4000μs 30.2582μs 33.0489 KOps/s 33.1539 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-False-True-False-True] 98.5000μs 37.5388μs 26.6391 KOps/s 26.6300 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-False-True-False-False] 42.7000μs 21.8159μs 45.8381 KOps/s 46.1587 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-False-True-True] 0.1095ms 49.0818μs 20.3742 KOps/s 20.2492 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-False-False-True-False] 91.7990μs 32.1421μs 31.1119 KOps/s 31.0743 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[False-False-False-False-True] 0.1719ms 38.9998μs 25.6411 KOps/s 25.7746 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-False-False-False] 78.7000μs 23.3715μs 42.7871 KOps/s 42.7033 KOps/s $\color{#35bf28}+0.20\%$
test_values[generalized_advantage_estimate-True-True] 14.1312ms 13.5687ms 73.6990 Ops/s 73.7941 Ops/s $\color{#d91a1a}-0.13\%$
test_values[vec_generalized_advantage_estimate-True-True] 56.8486ms 50.8596ms 19.6620 Ops/s 19.3079 Ops/s $\color{#35bf28}+1.83\%$
test_values[td0_return_estimate-False-False] 0.2953ms 0.1840ms 5.4361 KOps/s 4.9590 KOps/s $\textbf{\color{#35bf28}+9.62\%}$
test_values[td1_return_estimate-False-False] 13.5815ms 13.4015ms 74.6186 Ops/s 75.3499 Ops/s $\color{#d91a1a}-0.97\%$
test_values[vec_td1_return_estimate-False-False] 52.9564ms 50.7646ms 19.6987 Ops/s 19.5899 Ops/s $\color{#35bf28}+0.56\%$
test_values[td_lambda_return_estimate-True-False] 32.8729ms 32.5639ms 30.7088 Ops/s 30.9592 Ops/s $\color{#d91a1a}-0.81\%$
test_values[vec_td_lambda_return_estimate-True-False] 51.6358ms 50.6081ms 19.7597 Ops/s 19.3923 Ops/s $\color{#35bf28}+1.89\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.3475ms 12.2298ms 81.7676 Ops/s 82.4073 Ops/s $\color{#d91a1a}-0.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.9660ms 2.4589ms 406.6898 Ops/s 411.3818 Ops/s $\color{#d91a1a}-1.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1.5382ms 0.3948ms 2.5327 KOps/s 2.5554 KOps/s $\color{#d91a1a}-0.89\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 60.9504ms 51.4104ms 19.4513 Ops/s 18.9149 Ops/s $\color{#35bf28}+2.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 5.6604ms 3.7933ms 263.6251 Ops/s 256.0047 Ops/s $\color{#35bf28}+2.98\%$
test_dqn_speed 6.5055ms 1.5431ms 648.0307 Ops/s 654.3660 Ops/s $\color{#d91a1a}-0.97\%$
test_ddpg_speed 7.3767ms 2.4855ms 402.3399 Ops/s 412.7528 Ops/s $\color{#d91a1a}-2.52\%$
test_sac_speed 9.1074ms 7.6506ms 130.7094 Ops/s 133.4423 Ops/s $\color{#d91a1a}-2.05\%$
test_redq_speed 18.4684ms 13.7977ms 72.4758 Ops/s 73.8784 Ops/s $\color{#d91a1a}-1.90\%$
test_redq_deprec_speed 11.6860ms 11.1463ms 89.7158 Ops/s 89.0617 Ops/s $\color{#35bf28}+0.73\%$
test_td3_speed 11.4463ms 10.3791ms 96.3472 Ops/s 97.8508 Ops/s $\color{#d91a1a}-1.54\%$
test_cql_speed 26.4321ms 25.1695ms 39.7307 Ops/s 38.7085 Ops/s $\color{#35bf28}+2.64\%$
test_a2c_speed 5.8100ms 5.1449ms 194.3679 Ops/s 196.2716 Ops/s $\color{#d91a1a}-0.97\%$
test_ppo_speed 21.5392ms 5.8632ms 170.5556 Ops/s 179.1970 Ops/s $\color{#d91a1a}-4.82\%$
test_reinforce_speed 6.1426ms 4.0611ms 246.2412 Ops/s 247.8210 Ops/s $\color{#d91a1a}-0.64\%$
test_iql_speed 22.1670ms 20.3422ms 49.1590 Ops/s 49.4138 Ops/s $\color{#d91a1a}-0.52\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8960ms 3.3023ms 302.8215 Ops/s 294.8556 Ops/s $\color{#35bf28}+2.70\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 6.2040ms 3.4914ms 286.4203 Ops/s 282.9747 Ops/s $\color{#35bf28}+1.22\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.0544ms 3.4471ms 290.1029 Ops/s 254.8744 Ops/s $\textbf{\color{#35bf28}+13.82\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9617ms 3.3036ms 302.7007 Ops/s 293.0853 Ops/s $\color{#35bf28}+3.28\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.3897ms 3.4288ms 291.6443 Ops/s 286.8543 Ops/s $\color{#35bf28}+1.67\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.7172ms 3.4455ms 290.2349 Ops/s 283.0678 Ops/s $\color{#35bf28}+2.53\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1891s 3.9440ms 253.5483 Ops/s 294.5874 Ops/s $\textbf{\color{#d91a1a}-13.93\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.3992ms 3.4488ms 289.9519 Ops/s 282.6489 Ops/s $\color{#35bf28}+2.58\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 5.3332ms 3.4602ms 289.0046 Ops/s 279.8849 Ops/s $\color{#35bf28}+3.26\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.3813ms 3.3349ms 299.8567 Ops/s 289.2231 Ops/s $\color{#35bf28}+3.68\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.5100ms 3.4666ms 288.4630 Ops/s 283.5207 Ops/s $\color{#35bf28}+1.74\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 6.1837ms 3.4579ms 289.1926 Ops/s 282.0649 Ops/s $\color{#35bf28}+2.53\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.2039ms 3.3246ms 300.7922 Ops/s 291.8694 Ops/s $\color{#35bf28}+3.06\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.1950ms 3.4949ms 286.1327 Ops/s 283.3468 Ops/s $\color{#35bf28}+0.98\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 6.1975ms 3.4885ms 286.6578 Ops/s 283.5546 Ops/s $\color{#35bf28}+1.09\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5796ms 3.3320ms 300.1199 Ops/s 248.0924 Ops/s $\textbf{\color{#35bf28}+20.97\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.6465ms 3.4697ms 288.2122 Ops/s 287.1159 Ops/s $\color{#35bf28}+0.38\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.3163ms 3.4877ms 286.7256 Ops/s 282.6079 Ops/s $\color{#35bf28}+1.46\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2128s 30.2750ms 33.0306 Ops/s 35.1910 Ops/s $\textbf{\color{#d91a1a}-6.14\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1263s 26.0236ms 38.4266 Ops/s 35.5704 Ops/s $\textbf{\color{#35bf28}+8.03\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1326s 28.9471ms 34.5457 Ops/s 35.1523 Ops/s $\color{#d91a1a}-1.73\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1260s 26.0188ms 38.4338 Ops/s 38.3328 Ops/s $\color{#35bf28}+0.26\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1277s 28.2226ms 35.4326 Ops/s 35.5461 Ops/s $\color{#d91a1a}-0.32\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1275s 26.2672ms 38.0703 Ops/s 38.5606 Ops/s $\color{#d91a1a}-1.27\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1249s 28.2232ms 35.4318 Ops/s 35.5604 Ops/s $\color{#d91a1a}-0.36\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1267s 28.2924ms 35.3452 Ops/s 38.1483 Ops/s $\textbf{\color{#d91a1a}-7.35\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1306s 26.5193ms 37.7083 Ops/s 35.5393 Ops/s $\textbf{\color{#35bf28}+6.10\%}$

@github-actions
Copy link

github-actions bot commented Jul 9, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1813s 0.1782s 5.6114 Ops/s 5.5796 Ops/s $\color{#35bf28}+0.57\%$
test_sync 95.9381ms 92.4154ms 10.8207 Ops/s 10.4506 Ops/s $\color{#35bf28}+3.54\%$
test_async 0.1760s 91.6733ms 10.9083 Ops/s 10.5513 Ops/s $\color{#35bf28}+3.38\%$
test_simple 0.9491s 0.8570s 1.1669 Ops/s 1.1150 Ops/s $\color{#35bf28}+4.66\%$
test_transformed 2.1263s 2.0720s 0.4826 Ops/s 0.4628 Ops/s $\color{#35bf28}+4.28\%$
test_serial 2.8158s 2.7077s 0.3693 Ops/s 0.3559 Ops/s $\color{#35bf28}+3.77\%$
test_parallel 2.1740s 2.0883s 0.4789 Ops/s 0.4759 Ops/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-True-True-True-True] 0.4407ms 47.0628μs 21.2482 KOps/s 20.5950 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[True-True-True-True-False] 0.5404ms 27.0235μs 37.0048 KOps/s 35.3613 KOps/s $\color{#35bf28}+4.65\%$
test_step_mdp_speed[True-True-True-False-True] 1.3149ms 33.5061μs 29.8453 KOps/s 27.9368 KOps/s $\textbf{\color{#35bf28}+6.83\%}$
test_step_mdp_speed[True-True-True-False-False] 5.7404ms 19.8644μs 50.3413 KOps/s 49.0277 KOps/s $\color{#35bf28}+2.68\%$
test_step_mdp_speed[True-True-False-True-True] 2.8798ms 50.5249μs 19.7922 KOps/s 19.9820 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-True-False-True-False] 0.3768ms 29.0122μs 34.4682 KOps/s 33.3738 KOps/s $\color{#35bf28}+3.28\%$
test_step_mdp_speed[True-True-False-False-True] 1.5532ms 35.3406μs 28.2961 KOps/s 26.4230 KOps/s $\textbf{\color{#35bf28}+7.09\%}$
test_step_mdp_speed[True-True-False-False-False] 8.5080ms 21.6291μs 46.2340 KOps/s 44.1722 KOps/s $\color{#35bf28}+4.67\%$
test_step_mdp_speed[True-False-True-True-True] 2.0382ms 50.5855μs 19.7685 KOps/s 19.0587 KOps/s $\color{#35bf28}+3.72\%$
test_step_mdp_speed[True-False-True-True-False] 1.3042ms 31.1906μs 32.0609 KOps/s 31.0002 KOps/s $\color{#35bf28}+3.42\%$
test_step_mdp_speed[True-False-True-False-True] 3.0265ms 35.4391μs 28.2174 KOps/s 27.1835 KOps/s $\color{#35bf28}+3.80\%$
test_step_mdp_speed[True-False-True-False-False] 0.3418ms 20.4428μs 48.9170 KOps/s 46.7751 KOps/s $\color{#35bf28}+4.58\%$
test_step_mdp_speed[True-False-False-True-True] 0.8379ms 52.0721μs 19.2041 KOps/s 18.5714 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[True-False-False-True-False] 0.3103ms 32.0873μs 31.1649 KOps/s 30.0144 KOps/s $\color{#35bf28}+3.83\%$
test_step_mdp_speed[True-False-False-False-True] 3.4501ms 37.4300μs 26.7166 KOps/s 24.5437 KOps/s $\textbf{\color{#35bf28}+8.85\%}$
test_step_mdp_speed[True-False-False-False-False] 0.2817ms 23.1516μs 43.1936 KOps/s 42.1977 KOps/s $\color{#35bf28}+2.36\%$
test_step_mdp_speed[False-True-True-True-True] 0.7828ms 50.5372μs 19.7874 KOps/s 18.5886 KOps/s $\textbf{\color{#35bf28}+6.45\%}$
test_step_mdp_speed[False-True-True-True-False] 2.7390ms 31.3604μs 31.8873 KOps/s 30.7267 KOps/s $\color{#35bf28}+3.78\%$
test_step_mdp_speed[False-True-True-False-True] 2.9747ms 41.2031μs 24.2700 KOps/s 23.0531 KOps/s $\textbf{\color{#35bf28}+5.28\%}$
test_step_mdp_speed[False-True-True-False-False] 0.8377ms 23.5426μs 42.4762 KOps/s 39.9971 KOps/s $\textbf{\color{#35bf28}+6.20\%}$
test_step_mdp_speed[False-True-False-True-True] 2.8488ms 53.5389μs 18.6780 KOps/s 18.1707 KOps/s $\color{#35bf28}+2.79\%$
test_step_mdp_speed[False-True-False-True-False] 5.0187ms 33.4517μs 29.8938 KOps/s 29.6684 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-True-False-False-True] 0.4616ms 41.9149μs 23.8579 KOps/s 22.0122 KOps/s $\textbf{\color{#35bf28}+8.38\%}$
test_step_mdp_speed[False-True-False-False-False] 2.9257ms 25.0231μs 39.9631 KOps/s 36.2620 KOps/s $\textbf{\color{#35bf28}+10.21\%}$
test_step_mdp_speed[False-False-True-True-True] 0.3796ms 53.9936μs 18.5207 KOps/s 17.9172 KOps/s $\color{#35bf28}+3.37\%$
test_step_mdp_speed[False-False-True-True-False] 0.3586ms 33.5535μs 29.8031 KOps/s 28.9408 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[False-False-True-False-True] 80.5010μs 41.2068μs 24.2678 KOps/s 20.2124 KOps/s $\textbf{\color{#35bf28}+20.06\%}$
test_step_mdp_speed[False-False-True-False-False] 1.9888ms 24.4294μs 40.9344 KOps/s 35.7953 KOps/s $\textbf{\color{#35bf28}+14.36\%}$
test_step_mdp_speed[False-False-False-True-True] 0.6502ms 54.7762μs 18.2561 KOps/s 17.5962 KOps/s $\color{#35bf28}+3.75\%$
test_step_mdp_speed[False-False-False-True-False] 2.8354ms 36.3632μs 27.5003 KOps/s 27.3346 KOps/s $\color{#35bf28}+0.61\%$
test_step_mdp_speed[False-False-False-False-True] 9.8959ms 46.4425μs 21.5320 KOps/s 22.3256 KOps/s $\color{#d91a1a}-3.55\%$
test_step_mdp_speed[False-False-False-False-False] 0.3244ms 26.3875μs 37.8967 KOps/s 35.5803 KOps/s $\textbf{\color{#35bf28}+6.51\%}$
test_values[generalized_advantage_estimate-True-True] 26.0838ms 19.7725ms 50.5752 Ops/s 50.5104 Ops/s $\color{#35bf28}+0.13\%$
test_values[vec_generalized_advantage_estimate-True-True] 77.4513ms 65.9872ms 15.1544 Ops/s 14.5510 Ops/s $\color{#35bf28}+4.15\%$
test_values[td0_return_estimate-False-False] 0.4102ms 0.2681ms 3.7293 KOps/s 3.4793 KOps/s $\textbf{\color{#35bf28}+7.18\%}$
test_values[td1_return_estimate-False-False] 28.3757ms 19.3365ms 51.7157 Ops/s 53.1603 Ops/s $\color{#d91a1a}-2.72\%$
test_values[vec_td1_return_estimate-False-False] 75.3422ms 66.1812ms 15.1100 Ops/s 14.8580 Ops/s $\color{#35bf28}+1.70\%$
test_values[td_lambda_return_estimate-True-False] 61.6380ms 53.7122ms 18.6178 Ops/s 20.7237 Ops/s $\textbf{\color{#d91a1a}-10.16\%}$
test_values[vec_td_lambda_return_estimate-True-False] 76.4995ms 66.3994ms 15.0604 Ops/s 14.6642 Ops/s $\color{#35bf28}+2.70\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 18.7894ms 15.1746ms 65.8997 Ops/s 63.0563 Ops/s $\color{#35bf28}+4.51\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 28.3559ms 4.5952ms 217.6164 Ops/s 227.9750 Ops/s $\color{#d91a1a}-4.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 2.5723ms 0.7204ms 1.3881 KOps/s 1.3649 KOps/s $\color{#35bf28}+1.70\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 78.1320ms 73.8863ms 13.5343 Ops/s 13.3697 Ops/s $\color{#35bf28}+1.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 7.4588ms 5.4058ms 184.9877 Ops/s 181.6950 Ops/s $\color{#35bf28}+1.81\%$
test_dqn_speed 11.1128ms 2.5221ms 396.5016 Ops/s 402.0768 Ops/s $\color{#d91a1a}-1.39\%$
test_ddpg_speed 15.3022ms 4.7206ms 211.8352 Ops/s 220.7820 Ops/s $\color{#d91a1a}-4.05\%$
test_sac_speed 15.5866ms 12.7597ms 78.3717 Ops/s 77.8827 Ops/s $\color{#35bf28}+0.63\%$
test_redq_speed 31.7970ms 24.3434ms 41.0789 Ops/s 38.9641 Ops/s $\textbf{\color{#35bf28}+5.43\%}$
test_redq_deprec_speed 24.3088ms 19.5632ms 51.1164 Ops/s 52.0713 Ops/s $\color{#d91a1a}-1.83\%$
test_td3_speed 27.5245ms 16.9364ms 59.0444 Ops/s 60.5739 Ops/s $\color{#d91a1a}-2.53\%$
test_cql_speed 82.5958ms 58.6668ms 17.0454 Ops/s 18.8017 Ops/s $\textbf{\color{#d91a1a}-9.34\%}$
test_a2c_speed 21.1647ms 11.1086ms 90.0200 Ops/s 95.2596 Ops/s $\textbf{\color{#d91a1a}-5.50\%}$
test_ppo_speed 19.9285ms 10.8061ms 92.5405 Ops/s 92.4540 Ops/s $\color{#35bf28}+0.09\%$
test_reinforce_speed 10.0647ms 8.3806ms 119.3237 Ops/s 113.2688 Ops/s $\textbf{\color{#35bf28}+5.35\%}$
test_iql_speed 45.1703ms 41.8275ms 23.9077 Ops/s 22.9125 Ops/s $\color{#35bf28}+4.34\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 12.5491ms 3.9289ms 254.5238 Ops/s 263.3245 Ops/s $\color{#d91a1a}-3.34\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 6.5041ms 4.0127ms 249.2068 Ops/s 249.0993 Ops/s $\color{#35bf28}+0.04\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.1679s 4.6654ms 214.3446 Ops/s 246.1112 Ops/s $\textbf{\color{#d91a1a}-12.91\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1991ms 3.8644ms 258.7726 Ops/s 281.5374 Ops/s $\textbf{\color{#d91a1a}-8.09\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.9087ms 4.0141ms 249.1242 Ops/s 242.8440 Ops/s $\color{#35bf28}+2.59\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 7.2609ms 4.1205ms 242.6889 Ops/s 247.0279 Ops/s $\color{#d91a1a}-1.76\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.8089ms 4.4955ms 222.4428 Ops/s 252.9085 Ops/s $\textbf{\color{#d91a1a}-12.05\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 6.9474ms 4.1447ms 241.2715 Ops/s 240.2887 Ops/s $\color{#35bf28}+0.41\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.8434ms 4.1243ms 242.4634 Ops/s 247.2502 Ops/s $\color{#d91a1a}-1.94\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8151ms 3.8091ms 262.5309 Ops/s 258.8799 Ops/s $\color{#35bf28}+1.41\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 6.2167ms 4.0641ms 246.0561 Ops/s 210.0265 Ops/s $\textbf{\color{#35bf28}+17.15\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 13.4064ms 4.1959ms 238.3259 Ops/s 245.9376 Ops/s $\color{#d91a1a}-3.09\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9536ms 3.8281ms 261.2278 Ops/s 250.3536 Ops/s $\color{#35bf28}+4.34\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 7.7749ms 4.1226ms 242.5662 Ops/s 244.8559 Ops/s $\color{#d91a1a}-0.94\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 11.4682ms 4.1574ms 240.5363 Ops/s 249.2958 Ops/s $\color{#d91a1a}-3.51\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.5637ms 3.9035ms 256.1782 Ops/s 254.0922 Ops/s $\color{#35bf28}+0.82\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 10.6024ms 4.1523ms 240.8288 Ops/s 241.5973 Ops/s $\color{#d91a1a}-0.32\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 12.7960ms 4.1411ms 241.4844 Ops/s 248.2389 Ops/s $\color{#d91a1a}-2.72\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2758s 39.4598ms 25.3422 Ops/s 25.1321 Ops/s $\color{#35bf28}+0.84\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1666s 36.8436ms 27.1418 Ops/s 29.7257 Ops/s $\textbf{\color{#d91a1a}-8.69\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1619s 33.9588ms 29.4475 Ops/s 26.8602 Ops/s $\textbf{\color{#35bf28}+9.63\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1709s 37.2971ms 26.8118 Ops/s 27.4170 Ops/s $\color{#d91a1a}-2.21\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1818s 35.0237ms 28.5521 Ops/s 31.7213 Ops/s $\textbf{\color{#d91a1a}-9.99\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1661s 37.3231ms 26.7930 Ops/s 26.7590 Ops/s $\color{#35bf28}+0.13\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1645s 34.0227ms 29.3921 Ops/s 28.7461 Ops/s $\color{#35bf28}+2.25\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1702s 37.6201ms 26.5816 Ops/s 26.9471 Ops/s $\color{#d91a1a}-1.36\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1675s 34.5448ms 28.9479 Ops/s 28.7127 Ops/s $\color{#35bf28}+0.82\%$

@vmoens vmoens merged commit 771ef81 into main Jul 9, 2023
@vmoens vmoens deleted the fix_td3 branch July 11, 2023 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot