Description

• Add unittest_windows_optdepts_cpu workflow for python_version=3.7, 3.8, 3.9, 3.10 and
• Add unittest_windows_optdepts_gpu workflow for python_version=3.7, 3.8, 3.9, 3.10 and cuda=116 (workflows are not run for cuda=117 as done in the Torchvision repo)

Some unit tests are set to be skipped on Windows:

• test/test_rb_distributed.py::test_funcs: torch.distributed.rpc support is unstable for Windows
• test/test_rb.py:: TestPrototypeBuffers::{test_add,test_extend,test_sample,test_index} with rb_type= RemoteTensorDictReplayBuffer: torch.distributed.rpc support is unstable for Windows
• test_timeit.py is taking more time than allowed on Windows:

  File "C:\Users\circleci\project\test\test_helpers.py", line 935, in test_timeit
    assert abs(val1[0] - w1) < 1e-2
AssertionError: assert 0.01548680295944214 < 0.01
 +  where 0.01548680295944214 = abs((0.015586802959442139 - 0.0001))

• test_shared_.py::test_memmap : Call to td.to(SavedTensorDict) is failing (for all combinations of dtype and idx) with traceback:

Traceback (most recent call last):
  File "C:\Users\circleci\project\test\test_shared.py", line 178, in test_memmap
    td_saved = td.to(SavedTensorDict)
  File "C:\Users\circleci\project\env\lib\site-packages\tensordict\tensordict.py", line 2796, in to
    td = dest(
  File "C:\Users\circleci\project\env\lib\site-packages\tensordict\tensordict.py", line 4769, in __init__
    self._save(source)
  File "C:\Users\circleci\project\env\lib\site-packages\tensordict\tensordict.py", line 4779, in _save
    torch.save(tensordict, self.filename)
  File "C:\Users\circleci\project\env\lib\site-packages\torch\serialization.py", line 442, in save
    with _open_zipfile_writer(f) as opened_zipfile:
  File "C:\Users\circleci\project\env\lib\site-packages\torch\serialization.py", line 317, in _open_zipfile_writer
    return container(name_or_buffer)
  File "C:\Users\circleci\project\env\lib\site-packages\torch\serialization.py", line 290, in __init__
    super().__init__(torch._C.PyTorchFileWriter(str(name)))
RuntimeError: File C:\Users\circleci\AppData\Local\Temp\tmp8o43ldz4 cannot be opened.

• test_collector.py::test_output_device_consistency: Windows Access Violations and BrokenPipeError for certain combinations of python/policy_device/device/num_env
• test_collector.py::test_collector_batch_size: Test timeout (> 10 min) on CI pipeline Windows machine with GPU for num_env=3
• test_collector.py::test_collector_device_combinations: Access violation in torch.storage

Motivation and Context

TorchRL is currently built and tested for OsX and Linux only and there is some demand for a windows built.
This requires adding some test workflows for Windows.

Checklist

• I have read the CONTRIBUTION guide (required)
• I have updated the tests accordingly (required for a bug fix or a new feature).