@matteobettini do you recall why we need to remove _reset from the td?
The problem is that we call reset() in TransformedEnv and we need "_reset" to be passed to the transforms reset() methods. Deleting it make them consider that "_reset" is all True by default

The idea there is to remove it because if users do not notice it is there and call reset again with the same tensordict it will still only reset those indexes and they might not notice. So i though of making it a consumable that once reaches the inner env is removed.

The problem is that right now the transform reset is called after the env reset. So this consumable logic consumes the _reset flag before reacheing the transforms. There is no workaroud to this right?

The same consumable logic is implemented also in the _BatchedEnvs so if we remove it we should remove it from there too

(and we might want to make the deletion in collectors)

I might be wrong (please correct me if I am) but isn't is actually useful to keep track of that?

It is also dangerous to keep the "action" entry or anything else FWIW. The real nasty thing is to control step_mdp such that its default behaviour does not pass useless info to the next step.

Yes, at the time i made it seemed less error prone to consume it, but we can treat it as we treat actions. And now I realize that transforms reset is called after the env one and they may need it.

If we integrate it, maybe we can remove the underscore. Or leave it (in which case the collector will discard it by default)

I would probably leave it as the private meaning is nice. Collectors will discard it automatically yes but they will also have to explicitly discard it here https://github.com/pytorch/rl/blob/main/torchrl/collectors/collectors.py#L620 i think, otherwise it will be carried to the next step. (Unless we implement the discarding logic in stp_mdp)