Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] VC1 integration #1211

Merged
merged 34 commits into from
Jul 7, 2023
Merged

[Feature] VC1 integration #1211

merged 34 commits into from
Jul 7, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented May 31, 2023 •
edited
Loading

Usage:

in_keys = ["pixels"]
out_keys = ["vec"]
vc1 = VC1Transform(
            in_keys=in_keys,
            out_keys=out_keys,
            del_keys=del_keys,
            model_name="default",
        )
base_env = ParallelEnv(4, lambda: SomeEnv().to(device))
transformed_env = TransformedEnv(base_env, vc1)

See tests for more info

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 31, 2023
@vmoens vmoens added the new algo New algorithm request or PR label Jun 9, 2023
@github-actions
Copy link

github-actions bot commented Jun 30, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1424s 0.1420s 7.0423 Ops/s 7.0972 Ops/s $\color{#d91a1a}-0.77\%$
test_sync 77.5384ms 74.4991ms 13.4230 Ops/s 11.5358 Ops/s $\textbf{\color{#35bf28}+16.36\%}$
test_async 0.1197s 72.6611ms 13.7625 Ops/s 13.8992 Ops/s $\color{#d91a1a}-0.98\%$
test_simple 0.6048s 0.5379s 1.8591 Ops/s 1.8513 Ops/s $\color{#35bf28}+0.42\%$
test_transformed 1.4007s 1.3651s 0.7325 Ops/s 0.7350 Ops/s $\color{#d91a1a}-0.34\%$
test_serial 1.7227s 1.6819s 0.5946 Ops/s 0.5806 Ops/s $\color{#35bf28}+2.41\%$
test_parallel 1.7750s 1.4937s 0.6695 Ops/s 0.6914 Ops/s $\color{#d91a1a}-3.17\%$
test_step_mdp_speed[True-True-True-True-True] 0.1344ms 41.9903μs 23.8150 KOps/s 23.8372 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-True-True-True-False] 43.6000μs 23.4051μs 42.7258 KOps/s 42.7060 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-True-True-False-True] 92.6000μs 30.3899μs 32.9057 KOps/s 32.6084 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[True-True-True-False-False] 35.2000μs 16.7638μs 59.6523 KOps/s 59.3144 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-True-False-True-True] 69.7000μs 43.1525μs 23.1736 KOps/s 23.2755 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-False-True-False] 0.1943ms 24.9950μs 40.0079 KOps/s 40.1559 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-False-True] 59.6000μs 32.2209μs 31.0358 KOps/s 31.2208 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-True-False-False-False] 69.2000μs 18.6017μs 53.7585 KOps/s 55.0169 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[True-False-True-True-True] 82.2000μs 45.2420μs 22.1034 KOps/s 22.4844 KOps/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-False-True-True-False] 69.0010μs 26.4242μs 37.8440 KOps/s 38.0110 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-False-True-False-True] 51.2000μs 32.1994μs 31.0565 KOps/s 31.4892 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-False-True-False-False] 37.5000μs 18.1179μs 55.1941 KOps/s 54.8974 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-False-False-True-True] 67.4000μs 46.5564μs 21.4793 KOps/s 21.8320 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[True-False-False-True-False] 72.4000μs 27.9850μs 35.7334 KOps/s 35.8398 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-False-False-True] 53.2000μs 33.3821μs 29.9562 KOps/s 29.6926 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-False-False-False] 40.5000μs 19.8451μs 50.3902 KOps/s 51.0716 KOps/s $\color{#d91a1a}-1.33\%$
test_step_mdp_speed[False-True-True-True-True] 87.7000μs 44.9550μs 22.2444 KOps/s 22.3063 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-True-True-True-False] 67.1010μs 26.4376μs 37.8249 KOps/s 37.8732 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-True-False-True] 0.1467ms 36.2732μs 27.5686 KOps/s 27.5057 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-True-True-False-False] 0.1600ms 19.9998μs 50.0005 KOps/s 49.5975 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[False-True-False-True-True] 67.5000μs 46.5746μs 21.4709 KOps/s 21.5938 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-True-False] 54.2000μs 28.1662μs 35.5036 KOps/s 36.0635 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[False-True-False-False-True] 76.7000μs 37.6651μs 26.5497 KOps/s 26.3304 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-False-False-False] 0.3922ms 21.5862μs 46.3259 KOps/s 46.5374 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-False-True-True-True] 0.5475ms 48.0390μs 20.8164 KOps/s 21.3048 KOps/s $\color{#d91a1a}-2.29\%$
test_step_mdp_speed[False-False-True-True-False] 54.7000μs 29.3919μs 34.0230 KOps/s 34.4177 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-False-True-False-True] 54.3000μs 38.3993μs 26.0422 KOps/s 26.7200 KOps/s $\color{#d91a1a}-2.54\%$
test_step_mdp_speed[False-False-True-False-False] 67.8000μs 21.2918μs 46.9665 KOps/s 46.7542 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-False-False-True-True] 84.9000μs 49.0685μs 20.3797 KOps/s 20.7667 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[False-False-False-True-False] 76.3000μs 30.8796μs 32.3839 KOps/s 32.5389 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-False-False-True] 81.8000μs 38.7823μs 25.7850 KOps/s 26.2863 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-False-False-False] 79.5000μs 22.8045μs 43.8510 KOps/s 43.8660 KOps/s $\color{#d91a1a}-0.03\%$
test_values[generalized_advantage_estimate-True-True] 13.8619ms 13.4486ms 74.3570 Ops/s 75.6557 Ops/s $\color{#d91a1a}-1.72\%$
test_values[vec_generalized_advantage_estimate-True-True] 54.9175ms 51.5888ms 19.3841 Ops/s 19.5784 Ops/s $\color{#d91a1a}-0.99\%$
test_values[td0_return_estimate-False-False] 0.3044ms 0.2508ms 3.9865 KOps/s 5.0187 KOps/s $\textbf{\color{#d91a1a}-20.57\%}$
test_values[td1_return_estimate-False-False] 13.5090ms 13.1432ms 76.0851 Ops/s 77.7611 Ops/s $\color{#d91a1a}-2.16\%$
test_values[vec_td1_return_estimate-False-False] 64.6101ms 51.4925ms 19.4203 Ops/s 19.4925 Ops/s $\color{#d91a1a}-0.37\%$
test_values[td_lambda_return_estimate-True-False] 31.3958ms 31.1978ms 32.0536 Ops/s 32.2147 Ops/s $\color{#d91a1a}-0.50\%$
test_values[vec_td_lambda_return_estimate-True-False] 58.0086ms 50.8130ms 19.6800 Ops/s 19.7547 Ops/s $\color{#d91a1a}-0.38\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.8211ms 11.5704ms 86.4277 Ops/s 86.4884 Ops/s $\color{#d91a1a}-0.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.4828ms 2.5196ms 396.8944 Ops/s 410.0381 Ops/s $\color{#d91a1a}-3.21\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5826ms 0.3834ms 2.6085 KOps/s 2.6010 KOps/s $\color{#35bf28}+0.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.2630ms 48.5027ms 20.6174 Ops/s 19.8181 Ops/s $\color{#35bf28}+4.03\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.4286ms 3.7584ms 266.0741 Ops/s 260.4997 Ops/s $\color{#35bf28}+2.14\%$
test_dqn_speed 6.3582ms 1.5209ms 657.5077 Ops/s 658.2989 Ops/s $\color{#d91a1a}-0.12\%$
test_ddpg_speed 4.5129ms 2.4864ms 402.1948 Ops/s 410.2346 Ops/s $\color{#d91a1a}-1.96\%$
test_sac_speed 7.7493ms 7.4477ms 134.2701 Ops/s 132.9707 Ops/s $\color{#35bf28}+0.98\%$
test_redq_speed 16.0709ms 12.7590ms 78.3761 Ops/s 75.0622 Ops/s $\color{#35bf28}+4.41\%$
test_redq_deprec_speed 11.8646ms 10.8666ms 92.0250 Ops/s 90.1936 Ops/s $\color{#35bf28}+2.03\%$
test_td3_speed 11.5427ms 10.2095ms 97.9482 Ops/s 96.1623 Ops/s $\color{#35bf28}+1.86\%$
test_cql_speed 28.1663ms 24.7954ms 40.3301 Ops/s 39.9543 Ops/s $\color{#35bf28}+0.94\%$
test_a2c_speed 5.8345ms 5.2029ms 192.2008 Ops/s 191.1344 Ops/s $\color{#35bf28}+0.56\%$
test_ppo_speed 6.4163ms 5.4794ms 182.5014 Ops/s 178.7248 Ops/s $\color{#35bf28}+2.11\%$
test_reinforce_speed 4.6141ms 4.0320ms 248.0134 Ops/s 242.1500 Ops/s $\color{#35bf28}+2.42\%$
test_iql_speed 21.7942ms 20.1687ms 49.5818 Ops/s 50.2260 Ops/s $\color{#d91a1a}-1.28\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.2461ms 3.6657ms 272.7968 Ops/s 269.1192 Ops/s $\color{#35bf28}+1.37\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.7630ms 3.8327ms 260.9115 Ops/s 264.4621 Ops/s $\color{#d91a1a}-1.34\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.7931ms 3.8163ms 262.0351 Ops/s 259.3928 Ops/s $\color{#35bf28}+1.02\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.3505ms 3.6115ms 276.8923 Ops/s 271.8406 Ops/s $\color{#35bf28}+1.86\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.6087ms 3.9385ms 253.9040 Ops/s 264.7520 Ops/s $\color{#d91a1a}-4.10\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 6.6272ms 3.8818ms 257.6094 Ops/s 233.9410 Ops/s $\textbf{\color{#35bf28}+10.12\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.3239ms 3.6426ms 274.5302 Ops/s 268.4647 Ops/s $\color{#35bf28}+2.26\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.8502ms 3.8203ms 261.7609 Ops/s 250.4627 Ops/s $\color{#35bf28}+4.51\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.0363ms 3.8558ms 259.3509 Ops/s 253.8774 Ops/s $\color{#35bf28}+2.16\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 89.1575ms 3.9730ms 251.7013 Ops/s 242.7338 Ops/s $\color{#35bf28}+3.69\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.6606ms 3.8084ms 262.5777 Ops/s 263.8998 Ops/s $\color{#d91a1a}-0.50\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 6.0532ms 3.8550ms 259.4027 Ops/s 236.0326 Ops/s $\textbf{\color{#35bf28}+9.90\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.6938ms 3.7296ms 268.1237 Ops/s 271.0232 Ops/s $\color{#d91a1a}-1.07\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 7.2214ms 3.7890ms 263.9210 Ops/s 259.9914 Ops/s $\color{#35bf28}+1.51\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 5.4246ms 3.7267ms 268.3345 Ops/s 257.5662 Ops/s $\color{#35bf28}+4.18\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 84.2008ms 3.8830ms 257.5296 Ops/s 252.1231 Ops/s $\color{#35bf28}+2.14\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.7557ms 3.7396ms 267.4117 Ops/s 265.3224 Ops/s $\color{#35bf28}+0.79\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1172s 4.1814ms 239.1524 Ops/s 263.4956 Ops/s $\textbf{\color{#d91a1a}-9.24\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2399s 30.5525ms 32.7305 Ops/s 31.9402 Ops/s $\color{#35bf28}+2.47\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1167s 27.7386ms 36.0509 Ops/s 34.4855 Ops/s $\color{#35bf28}+4.54\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1180s 27.7610ms 36.0217 Ops/s 34.3027 Ops/s $\textbf{\color{#35bf28}+5.01\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1189s 27.7593ms 36.0239 Ops/s 35.3596 Ops/s $\color{#35bf28}+1.88\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1179s 27.9679ms 35.7553 Ops/s 35.2677 Ops/s $\color{#35bf28}+1.38\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1175s 29.8646ms 33.4844 Ops/s 33.1507 Ops/s $\color{#35bf28}+1.01\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1194s 27.7767ms 36.0014 Ops/s 35.5396 Ops/s $\color{#35bf28}+1.30\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1217s 28.2777ms 35.3636 Ops/s 35.1466 Ops/s $\color{#35bf28}+0.62\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1256s 28.3188ms 35.3123 Ops/s 35.4688 Ops/s $\color{#d91a1a}-0.44\%$

@github-actions
Copy link

github-actions bot commented Jun 30, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}12$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.2595s 0.2512s 3.9805 Ops/s 4.0577 Ops/s $\color{#d91a1a}-1.90\%$
test_sync 0.1449s 0.1341s 7.4577 Ops/s 6.9641 Ops/s $\textbf{\color{#35bf28}+7.09\%}$
test_async 0.2021s 0.1341s 7.4552 Ops/s 7.7261 Ops/s $\color{#d91a1a}-3.51\%$
test_simple 1.1035s 1.0073s 0.9927 Ops/s 0.9797 Ops/s $\color{#35bf28}+1.34\%$
test_transformed 2.4845s 2.4229s 0.4127 Ops/s 0.4064 Ops/s $\color{#35bf28}+1.57\%$
test_serial 3.2309s 3.1542s 0.3170 Ops/s 0.3290 Ops/s $\color{#d91a1a}-3.65\%$
test_parallel 2.5364s 2.4744s 0.4041 Ops/s 0.4124 Ops/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-True-True-True] 6.0148ms 50.9591μs 19.6236 KOps/s 18.8219 KOps/s $\color{#35bf28}+4.26\%$
test_step_mdp_speed[True-True-True-True-False] 6.3728ms 29.5484μs 33.8428 KOps/s 33.9823 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-True-True-False-True] 4.0329ms 37.6621μs 26.5519 KOps/s 27.3126 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[True-True-True-False-False] 4.3949ms 21.7751μs 45.9241 KOps/s 46.1060 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[True-True-False-True-True] 2.1832ms 53.3299μs 18.7512 KOps/s 18.9551 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-False-True-False] 5.8874ms 32.0033μs 31.2468 KOps/s 32.7967 KOps/s $\color{#d91a1a}-4.73\%$
test_step_mdp_speed[True-True-False-False-True] 1.5772ms 38.2346μs 26.1543 KOps/s 24.7041 KOps/s $\textbf{\color{#35bf28}+5.87\%}$
test_step_mdp_speed[True-True-False-False-False] 2.6277ms 22.3864μs 44.6700 KOps/s 41.9118 KOps/s $\textbf{\color{#35bf28}+6.58\%}$
test_step_mdp_speed[True-False-True-True-True] 1.4170ms 53.4771μs 18.6996 KOps/s 18.7810 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-True-True-False] 1.6835ms 31.7916μs 31.4549 KOps/s 30.9651 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[True-False-True-False-True] 1.9427ms 38.4228μs 26.0262 KOps/s 25.8595 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-True-False-False] 2.5625ms 22.9648μs 43.5450 KOps/s 44.5742 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[True-False-False-True-True] 4.2899ms 56.2192μs 17.7875 KOps/s 18.0126 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[True-False-False-True-False] 3.1530ms 34.0292μs 29.3866 KOps/s 29.7090 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-False-False-False-True] 1.6514ms 40.4330μs 24.7323 KOps/s 24.7222 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-False-False-False-False] 6.6843ms 26.3786μs 37.9096 KOps/s 41.0171 KOps/s $\textbf{\color{#d91a1a}-7.58\%}$
test_step_mdp_speed[False-True-True-True-True] 1.6351ms 54.6969μs 18.2826 KOps/s 18.1520 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-True-True-True-False] 1.0827ms 32.5136μs 30.7564 KOps/s 30.8022 KOps/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[False-True-True-False-True] 5.7125ms 45.1585μs 22.1442 KOps/s 23.4717 KOps/s $\textbf{\color{#d91a1a}-5.66\%}$
test_step_mdp_speed[False-True-True-False-False] 2.8493ms 25.7609μs 38.8185 KOps/s 38.5639 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-True-False-True-True] 4.5176ms 60.3182μs 16.5787 KOps/s 17.1411 KOps/s $\color{#d91a1a}-3.28\%$
test_step_mdp_speed[False-True-False-True-False] 1.5808ms 33.7159μs 29.6596 KOps/s 28.2641 KOps/s $\color{#35bf28}+4.94\%$
test_step_mdp_speed[False-True-False-False-True] 3.5475ms 46.3501μs 21.5749 KOps/s 21.5218 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-True-False-False-False] 1.9142ms 26.8351μs 37.2647 KOps/s 36.4688 KOps/s $\color{#35bf28}+2.18\%$
test_step_mdp_speed[False-False-True-True-True] 4.8844ms 58.5916μs 17.0673 KOps/s 17.0304 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-False-True-True-False] 3.6402ms 35.8308μs 27.9089 KOps/s 27.4432 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-True-False-True] 0.2133ms 43.7752μs 22.8440 KOps/s 20.0578 KOps/s $\textbf{\color{#35bf28}+13.89\%}$
test_step_mdp_speed[False-False-True-False-False] 1.2713ms 26.1813μs 38.1953 KOps/s 37.3920 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-False-False-True-True] 1.6219ms 58.1794μs 17.1882 KOps/s 17.8030 KOps/s $\color{#d91a1a}-3.45\%$
test_step_mdp_speed[False-False-False-True-False] 3.5310ms 35.6133μs 28.0794 KOps/s 26.4694 KOps/s $\textbf{\color{#35bf28}+6.08\%}$
test_step_mdp_speed[False-False-False-False-True] 3.9848ms 47.0595μs 21.2497 KOps/s 21.6632 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-False-False-False] 1.4095ms 27.4111μs 36.4816 KOps/s 35.1073 KOps/s $\color{#35bf28}+3.91\%$
test_values[generalized_advantage_estimate-True-True] 31.6599ms 21.8116ms 45.8472 Ops/s 48.1596 Ops/s $\color{#d91a1a}-4.80\%$
test_values[vec_generalized_advantage_estimate-True-True] 93.2863ms 80.0870ms 12.4864 Ops/s 12.4979 Ops/s $\color{#d91a1a}-0.09\%$
test_values[td0_return_estimate-False-False] 0.6129ms 0.3957ms 2.5272 KOps/s 2.6544 KOps/s $\color{#d91a1a}-4.79\%$
test_values[td1_return_estimate-False-False] 28.1241ms 21.0135ms 47.5884 Ops/s 48.9631 Ops/s $\color{#d91a1a}-2.81\%$
test_values[vec_td1_return_estimate-False-False] 95.5193ms 79.6868ms 12.5491 Ops/s 12.8507 Ops/s $\color{#d91a1a}-2.35\%$
test_values[td_lambda_return_estimate-True-False] 66.6994ms 56.7180ms 17.6311 Ops/s 17.1657 Ops/s $\color{#35bf28}+2.71\%$
test_values[vec_td_lambda_return_estimate-True-False] 94.3614ms 79.2228ms 12.6226 Ops/s 12.8453 Ops/s $\color{#d91a1a}-1.73\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 21.2337ms 16.1340ms 61.9808 Ops/s 62.4849 Ops/s $\color{#d91a1a}-0.81\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 12.7692ms 5.1298ms 194.9409 Ops/s 190.1866 Ops/s $\color{#35bf28}+2.50\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 4.3147ms 0.7799ms 1.2823 KOps/s 1.1485 KOps/s $\textbf{\color{#35bf28}+11.65\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 95.6325ms 86.8136ms 11.5189 Ops/s 13.0819 Ops/s $\textbf{\color{#d91a1a}-11.95\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 9.1618ms 6.7285ms 148.6226 Ops/s 142.3744 Ops/s $\color{#35bf28}+4.39\%$
test_dqn_speed 12.7334ms 2.9585ms 338.0049 Ops/s 342.6976 Ops/s $\color{#d91a1a}-1.37\%$
test_ddpg_speed 10.9131ms 5.1411ms 194.5117 Ops/s 195.2596 Ops/s $\color{#d91a1a}-0.38\%$
test_sac_speed 24.0372ms 16.1059ms 62.0891 Ops/s 69.6603 Ops/s $\textbf{\color{#d91a1a}-10.87\%}$
test_redq_speed 0.1343s 29.8399ms 33.5122 Ops/s 37.3841 Ops/s $\textbf{\color{#d91a1a}-10.36\%}$
test_redq_deprec_speed 31.2629ms 24.5672ms 40.7047 Ops/s 39.6534 Ops/s $\color{#35bf28}+2.65\%$
test_td3_speed 25.2889ms 19.7708ms 50.5796 Ops/s 51.3147 Ops/s $\color{#d91a1a}-1.43\%$
test_cql_speed 72.0600ms 62.0382ms 16.1191 Ops/s 17.3308 Ops/s $\textbf{\color{#d91a1a}-6.99\%}$
test_a2c_speed 26.2432ms 12.5581ms 79.6301 Ops/s 76.2847 Ops/s $\color{#35bf28}+4.39\%$
test_ppo_speed 17.2666ms 13.1590ms 75.9937 Ops/s 69.4132 Ops/s $\textbf{\color{#35bf28}+9.48\%}$
test_reinforce_speed 14.4892ms 10.4095ms 96.0663 Ops/s 92.9211 Ops/s $\color{#35bf28}+3.38\%$
test_iql_speed 67.5116ms 56.0679ms 17.8355 Ops/s 19.2314 Ops/s $\textbf{\color{#d91a1a}-7.26\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8480ms 4.8201ms 207.4650 Ops/s 202.3595 Ops/s $\color{#35bf28}+2.52\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.7882ms 5.0089ms 199.6439 Ops/s 188.0422 Ops/s $\textbf{\color{#35bf28}+6.17\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 11.4016ms 5.0474ms 198.1237 Ops/s 189.4028 Ops/s $\color{#35bf28}+4.60\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 10.7267ms 4.9785ms 200.8654 Ops/s 172.4158 Ops/s $\textbf{\color{#35bf28}+16.50\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 9.3890ms 5.3374ms 187.3563 Ops/s 198.7161 Ops/s $\textbf{\color{#d91a1a}-5.72\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 14.9280ms 5.2260ms 191.3507 Ops/s 161.2552 Ops/s $\textbf{\color{#35bf28}+18.66\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.6210ms 4.9911ms 200.3564 Ops/s 203.8243 Ops/s $\color{#d91a1a}-1.70\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 14.1707ms 5.5375ms 180.5863 Ops/s 199.7725 Ops/s $\textbf{\color{#d91a1a}-9.60\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.7831ms 5.2492ms 190.5065 Ops/s 193.7787 Ops/s $\color{#d91a1a}-1.69\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 9.4872ms 5.2037ms 192.1714 Ops/s 171.9442 Ops/s $\textbf{\color{#35bf28}+11.76\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.4399ms 5.2782ms 189.4579 Ops/s 199.6368 Ops/s $\textbf{\color{#d91a1a}-5.10\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 9.7359ms 5.3472ms 187.0138 Ops/s 191.8463 Ops/s $\color{#d91a1a}-2.52\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 12.6641ms 5.2153ms 191.7441 Ops/s 198.7501 Ops/s $\color{#d91a1a}-3.53\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 12.6914ms 5.4910ms 182.1160 Ops/s 198.4702 Ops/s $\textbf{\color{#d91a1a}-8.24\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.7734ms 5.3092ms 188.3540 Ops/s 201.0214 Ops/s $\textbf{\color{#d91a1a}-6.30\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.7317ms 5.1516ms 194.1144 Ops/s 179.1300 Ops/s $\textbf{\color{#35bf28}+8.37\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 12.5949ms 5.2413ms 190.7911 Ops/s 196.2624 Ops/s $\color{#d91a1a}-2.79\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 11.1697ms 5.3496ms 186.9292 Ops/s 194.6051 Ops/s $\color{#d91a1a}-3.94\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3742s 51.5013ms 19.4170 Ops/s 19.8591 Ops/s $\color{#d91a1a}-2.23\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.2119s 45.1183ms 22.1639 Ops/s 23.5686 Ops/s $\textbf{\color{#d91a1a}-5.96\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.2042s 43.7936ms 22.8344 Ops/s 23.0763 Ops/s $\color{#d91a1a}-1.05\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.2006s 43.6167ms 22.9270 Ops/s 22.8871 Ops/s $\color{#35bf28}+0.17\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.2067s 44.6763ms 22.3832 Ops/s 24.9973 Ops/s $\textbf{\color{#d91a1a}-10.46\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.2118s 48.1698ms 20.7599 Ops/s 21.3880 Ops/s $\color{#d91a1a}-2.94\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.2232s 44.3061ms 22.5703 Ops/s 22.9026 Ops/s $\color{#d91a1a}-1.45\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.2059s 43.7253ms 22.8701 Ops/s 22.7841 Ops/s $\color{#35bf28}+0.38\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.2169s 45.0001ms 22.2222 Ops/s 23.2186 Ops/s $\color{#d91a1a}-4.29\%$

matteobettini
matteobettini reviewed Jul 7, 2023
View reviewed changes
torchrl/envs/transforms/vc1.py Outdated
model_name (str): one of resnet50, resnet34 or resnet18
in_keys (list of str): list of input keys. If left empty, the
"pixels" key is assumed.
out_keys (list of str, optional): list of output keys. If left empty,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NestedKey?

matteobettini
matteobettini reviewed Jul 7, 2023
View reviewed changes
test/test_transforms.py
def test_transform_no_env(self, device):
in_keys = ["pixels"]
del_keys = False
out_keys = ["vec"]
Copy link
Contributor

@matteobettini matteobettini Jul 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add some nested keys to these tests?

matteobettini
matteobettini approved these changes Jul 7, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, i left some minor comments

@vmoens vmoens merged commit 002a58a into main Jul 7, 2023
@vmoens vmoens deleted the vc1 branch July 7, 2023 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. new algo New algorithm request or PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot