Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix] Change ppo mujoco example to match paper results #1714

Merged
merged 6 commits into from
Nov 27, 2023
Merged

[BugFix] Change ppo mujoco example to match paper results #1714

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6a86d13
retrieve working script
albertbou92 Nov 23, 2023
4261911
fix script
albertbou92 Nov 24, 2023
cca4630
fix script
albertbou92 Nov 24, 2023
dfb24a2
fix script
albertbou92 Nov 27, 2023
9a06db6
fix script
albertbou92 Nov 27, 2023
2361765
format
albertbou92 Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
format
  • Loading branch information
@albertbou92
albertbou92 committed Nov 27, 2023
commit 236176579db336854035986adb9a01df580fdc0d
5 changes: 3 additions & 2 deletions examples/ppo/ppo_mujoco.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,9 @@ def main(cfg: "DictConfig"): # noqa: F821
"train/lr": alpha * cfg_optim_lr,
"train/sampling_time": sampling_time,
"train/training_time": training_time,
"train/clip_epsilon": alpha * cfg_loss_clip_epsilon if
cfg_loss_anneal_clip_eps else cfg_loss_clip_epsilon,
"train/clip_epsilon": alpha * cfg_loss_clip_epsilon
if cfg_loss_anneal_clip_eps
else cfg_loss_clip_epsilon,
}
)

Expand Down
5 changes: 4 additions & 1 deletion examples/ppo/utils_mujoco.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
# Environment utils
# --------------------------------------------------------------------


def make_env(env_name="HalfCheetah-v4", device="cpu"):
env = GymEnv(env_name, device=device)
env = TransformedEnv(env)
Expand Down Expand Up @@ -71,7 +72,9 @@ def make_ppo_models_state(proof_environment):
# Add state-independent normal scale
policy_mlp = torch.nn.Sequential(
policy_mlp,
AddStateIndependentNormalScale(proof_environment.action_spec.shape[-1], scale_lb=1e-8)
AddStateIndependentNormalScale(
proof_environment.action_spec.shape[-1], scale_lb=1e-8
),
)

# Add probabilistic sampling of the actions
Expand Down
Loading