Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BugFix] Discrete SAC rewrite #1461

Merged
merged 18 commits into from
Aug 30, 2023
Merged

[BugFix] Discrete SAC rewrite #1461

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d0efb65
amend
matteobettini Aug 14, 2023
1d33f01
amend
matteobettini Aug 14, 2023
2512843
amend
matteobettini Aug 14, 2023
7338ffc
doc
matteobettini Aug 14, 2023
179d473
fix bug
matteobettini Aug 17, 2023
30cafe7
align interface with future subloss exposure
matteobettini Aug 17, 2023
5d68c27
explicit None
matteobettini Aug 17, 2023
3a6d250
add vmas examples
matteobettini Aug 25, 2023
b845fbc
amend
matteobettini Aug 25, 2023
578efb2
chenge to mode
matteobettini Aug 26, 2023
9f68ba0
grad norm
matteobettini Aug 26, 2023
964e2fa
review fix
matteobettini Aug 26, 2023
b5447b4
ci
matteobettini Aug 26, 2023
5560f9b
add loss alpha
matteobettini Aug 26, 2023
97d2b26
amend
matteobettini Aug 26, 2023
8365de7
update old sac example
matteobettini Aug 26, 2023
eb6d68a
fix
matteobettini Aug 26, 2023
2e8d6ad
fix
matteobettini Aug 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update old sac example 
Signed-off-by: Matteo Bettini <matbet@meta.com>
  • Loading branch information
@matteobettini
matteobettini committed Aug 26, 2023
commit 8365de78283c11baac4d8bc75d5bc354f392a0bc
8 changes: 5 additions & 3 deletions examples/discrete_sac/discrete_sac.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import torch
import torch.cuda
import tqdm
from tensordict.nn import InteractionType
from tensordict.nn import InteractionType, TensorDictModule

from torch import nn, optim
from torchrl.collectors import SyncDataCollector
Expand All @@ -27,7 +27,7 @@
from torchrl.modules import MLP, SafeModule
from torchrl.modules.distributions import OneHotCategorical

from torchrl.modules.tensordict_module.actors import ProbabilisticActor, ValueOperator
from torchrl.modules.tensordict_module.actors import ProbabilisticActor

from torchrl.objectives import DiscreteSACLoss, SoftUpdate
from torchrl.record.loggers import generate_exp_name, get_logger
Expand Down Expand Up @@ -150,8 +150,9 @@ def env_factory(num_workers):
**qvalue_net_kwargs,
)

qvalue = ValueOperator(
qvalue = TensorDictModule(
in_keys=in_keys,
out_keys=["action_value"],
module=qvalue_net,
).to(device)

Expand All @@ -171,6 +172,7 @@ def env_factory(num_workers):
# Create SAC loss
loss_module = DiscreteSACLoss(
actor_network=model[0],
action_space=test_env.action_spec,
qvalue_network=model[1],
num_actions=num_actions,
num_qvalue_nets=2,
Expand Down