Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] NonTensorSpec calls NonTensorData with unexpected shape argument #2168

Closed
3 tasks done
maichmueller opened this issue May 22, 2024 · 5 comments · Fixed by #2169, #2172 or #2232
Closed
3 tasks done

[BUG] NonTensorSpec calls NonTensorData with unexpected shape argument #2168

maichmueller opened this issue May 22, 2024 · 5 comments · Fixed by #2169, #2172 or #2232
Assignees
Labels
bug Something isn't working

Comments

@maichmueller
Copy link

maichmueller commented May 22, 2024

Describe the bug

a NonTensorSpec apparently instantiates a NonTensorData object wrong on calls such as one, zero, and rand.

To Reproduce

import torchrl

torchrl.data.NonTensorSpec().one(shape=[1])

shows

Traceback (most recent call last):
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-23-02439521e062>", line 1, in <module>
    torchrl.data.NonTensorSpec().one(shape=[1])
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/torchrl/data/tensor_specs.py", line 1953, in one
    return NonTensorData(data=None, shape=self.shape, device=self.device)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/tensordict/tensorclass.py", line 389, in wrapper
    init(self, **kwargs)
TypeError: NonTensorData.__init__() got an unexpected keyword argument 'shape'

This currently comes up when I try to build an env with a NonTensorSpec and run check_env_specs(env) on it.

Expected behavior

Not throwing an error.

Screenshots

If applicable, add screenshots to help explain your problem.

System info

Describe the characteristic of your environment:

  • Describe how the library was installed: pip
  • Python version: 3.11.8
  • Ubuntu 22.04
import torchrl, numpy, sys

print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)
0.4.0 1.26.4 3.11.8 (main, Feb 26 2024, 21:39:34) [GCC 11.2.0] linux

Reason and Possible fixes

These lines seem to be at fault:

class NonTensorSpec(TensorSpec):
    """A spec for non-tensor data."""
...
    def rand(self, shape):
        return NonTensorData(data=None, shape=self.shape, device=self.device)

    def zero(self, shape):
        return NonTensorData(data=None, shape=self.shape, device=self.device)

    def one(self, shape):
        return NonTensorData(data=None, shape=self.shape, device=self.device)

Checklist

  • I have checked that there is no similar issue in the repo (required)
  • I have read the documentation (required)
  • I have provided a minimal working example to reproduce the bug (required)
@maichmueller maichmueller added the bug Something isn't working label May 22, 2024
@maichmueller maichmueller changed the title [BUG] [BUG] NonTensorSpec calls NonTensorData with unexpected shape argument May 22, 2024
@vmoens
Copy link
Contributor

vmoens commented May 22, 2024

Oh yeah that should be batch_size!
let me fix those

@vmoens
Copy link
Contributor

vmoens commented May 22, 2024

This is already fixed on main

def rand(self, shape):
return NonTensorData(data=None, batch_size=self.shape, device=self.device)
def zero(self, batch_size):
return NonTensorData(data=None, batch_size=self.shape, device=self.device)
def one(self, batch_size):
return NonTensorData(data=None, batch_size=self.shape, device=self.device)

I can add the test though

@vmoens vmoens closed this as completed May 22, 2024
@vmoens vmoens linked a pull request May 22, 2024 that will close this issue
@maichmueller
Copy link
Author

Hi @vmoens ,

I am afraid the fix on main...

pip install --no-deps git+https://github.com/pytorch/rl.git
.....
Successfully built torchrl
Installing collected packages: torchrl
  Attempting uninstall: torchrl
    Found existing installation: torchrl 0.4.0
    Uninstalling torchrl-0.4.0:
      Successfully uninstalled torchrl-0.4.0
Successfully installed torchrl-0.4.0+a93063b

does not work (unless I am calling things wrong in the first place):

torchrl.data.NonTensorSpec().one(shape=[1])
Traceback (most recent call last):
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-37af7c334d8b>", line 3, in <module>
    torchrl.data.NonTensorSpec().one(shape=[1])
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/torchrl/data/tensor_specs.py", line 1969, in one
    return NonTensorData(
           ^^^^^^^^^^^^^^
  File "/work/rleap1/michael.aichmueller/miniconda/envs/rgnn/lib/python3.11/site-packages/tensordict/tensorclass.py", line 382, in wrapper
    batch_size=torch.Size(batch_size),
               ^^^^^^^^^^^^^^^^^^^^^^
TypeError: torch.Size() takes an iterable of 'int' (item 1 is 'torch.Size')

@vmoens vmoens reopened this May 26, 2024
@vmoens vmoens linked a pull request May 26, 2024 that will close this issue
@c3-utsavdutta98
Copy link

@vmoens , thanks for your help on this^, ran into the same issue, am on the main branch now and still running into an issue.

For a bit of context, my ._step(), generates a td with a field "x", in my custom env.

"x" is itself a tensordict with a nested key structure, where the keys are not fixed (variable depending on the env).

I was trying a super simple observation spec with a blank NonTensorSpec:

self.observation_spec = CompositeSpec(
            updated_demand=NonTensorSpec(shape=(batch_size,)),
            shape=(batch_size, )
        )

And when instantiating the SyncDataCollector I eventually run into :

image

Just want to confirm this is a different issue.

@vmoens vmoens reopened this Jun 15, 2024
@vmoens
Copy link
Contributor

vmoens commented Jun 15, 2024

Interesting, that seems to be an issue with NonTensorData which receives smth unexpected during update.

I ran this code and it works fine on my end (on main branch), can you help me figure out how your example differs?

from torchrl.envs import GymEnv, Transform, check_env_specs
from torchrl.data import NonTensorSpec, TensorSpec
from torchrl.collectors import SyncDataCollector
from tensordict import TensorDictBase, TensorDict, NonTensorData

class AddNontTensorData(Transform):
    def _call(self, tensordict: TensorDictBase) -> TensorDictBase:
        tensordict["nt"] = "a string!"
        return tensordict

    def _reset(
        self, tensordict: TensorDictBase, tensordict_reset: TensorDictBase
    ) -> TensorDictBase:
        return tensordict_reset.set("nt", NonTensorData("reset!"))

    def transform_observation_spec(self, observation_spec: TensorSpec) -> TensorSpec:
        observation_spec["nt"] = NonTensorSpec(shape=())
        return observation_spec

env = GymEnv("CartPole-v1").append_transform(AddNontTensorData())
print(env.rollout(3))
check_env_specs(env)

collector = SyncDataCollector(env, frames_per_batch=200)
for data in collector:
    print(data)
    break

EDIT: the data from the collector is garbage but non tensor data is not yet supported by collectors "officially"

@vmoens vmoens linked a pull request Jun 15, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants