Open
Description
In the _sanitize_pytorch_types function used in petastorm dataloaders, an exception is raised when an input is a string:
petastorm/petastorm/pytorch.py
Line 64 in 0b0775a
Is this really necessary? Maybe the default_collate function from PyTorch might not always like, although it seems to support it according to the documentation, but this prevents making other types of collate functions.
Would you consider removing the exception? Thank you.
Metadata
Metadata
Assignees
Labels
No labels
Activity
selitvin commentedon May 11, 2023
Sure! Would you like to propose a PR for that? Ideally, with a small unit test showing that it works.
freud14-tm commentedon Jul 11, 2023
It seems more complicated than I thought with the current code base of the library and I am not using the library right now. Still think the idea is worthwhile but feel free to close the issue if you wish. Otherwise, let it open in order to keep the idea close.
pratikgujjar commentedon Aug 23, 2023
Wondering if there is a reason for the string type to not be supported? A column in a parquet file could be all string labels. It is currently not possible to read this parquet file as a PyTorch dataloader becuase this line in the code.