-
Notifications
You must be signed in to change notification settings - Fork 587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hugging Face Hub Integration! #4193
Merged
Merged
Changes from 49 commits
Commits
Show all changes
69 commits
Select commit
Hold shift + click to select a range
a9eb9c7
initial sketch
jacobmarks cadc1be
always add filepath to sample dict
jacobmarks 7214599
image shape util usage
jacobmarks 8eaf809
change load_from_hub kwargs to named
jacobmarks 558497a
robust label field kwarg handling
jacobmarks 80b946e
enable config spec through kwargs
jacobmarks 0de7125
Speedup media download with threadpools
jacobmarks 72b381f
adds support for loading from gated repos
jacobmarks 3090f37
use threadpool to download masks
jacobmarks 50d8deb
default batch size config
jacobmarks 7225363
adds push_to_hub()
jacobmarks 3664981
adds loading FiftyOne datasets from hub
jacobmarks b63cdbf
clean up
jacobmarks 74e214c
handle no-label classification case
jacobmarks 639a7c2
rename hf util file
jacobmarks a539d76
adds standard module docstring
jacobmarks 8d49821
removes detection_format and mask_targets for now
jacobmarks cbafc83
lazy import huggingface_hub
jacobmarks 7fd50ba
remove unused import
jacobmarks 45aaefc
use TmpDir eta util
jacobmarks 0463937
fixes docstring indentation
jacobmarks 047947a
fix api docstring bullet list indent
jacobmarks 4e2f674
changing heading levels
jacobmarks 750c75a
change default push to public
jacobmarks 025a61e
basic push_to_hub docs
jacobmarks 97a7241
updating code snippet for loading dataset
jacobmarks 8eb5a0e
pushing to hub advanced usage section
jacobmarks a9d6801
examples for advanced push section
jacobmarks cc7f2c0
loading from hub via local config docs
jacobmarks edcbc4c
hub loading from kwargs docs
jacobmarks c6fe103
starting supported config fields section
jacobmarks 75b997d
documenting all supported config args
jacobmarks cc64b7b
documents split and subset config
jacobmarks fc9cfa8
documents download process params
jacobmarks 4c49b32
documents loading classification datasets
jacobmarks 0404864
documents loading detection datasets
jacobmarks e45317d
adds note about other detection formats
jacobmarks b510fff
documents loading segmentation datasets
jacobmarks b79fdf6
Documents loading unlabeled datasets
jacobmarks 75b60ef
documents advanced loading
jacobmarks 6114d74
adds breadcrumb to config args
jacobmarks d619c0e
Documents loading gated datasets
jacobmarks f0e82c2
adds num_workers arg for downloading
jacobmarks 75ee44c
remove unnecessary max_samples
jacobmarks de658d8
access warning
jacobmarks c771667
removes globals cache
jacobmarks b1e3882
implementation/doc updates
brimoor 7801a0a
Merge pull request #4223 from voxel51/hf-hub-integration-v2-brian
jacobmarks 3951c2c
bump huggingface_hub version for token auth
jacobmarks 8b74e3d
Update docs/source/integrations/huggingface.rst
jacobmarks dc3af96
Update docs/source/integrations/huggingface.rst
jacobmarks cd72833
Update docs/source/integrations/huggingface.rst
jacobmarks 9318690
Update docs/source/integrations/huggingface.rst
jacobmarks 43e6ce1
docs pass
brimoor 4484c2f
Merge branch 'hf-hub-integration-v2' of https://github.com/voxel51/fi…
brimoor 6cab1d0
makes fiftyone version req optional
jacobmarks e50e353
document min_fiftyone_version
jacobmarks 9e1e7d5
validation min_fiftyone_version
jacobmarks df1db86
remove unused import
jacobmarks e4e78ac
adds preview_path option to preview dataset
jacobmarks 40f7b92
documents preview_path
jacobmarks d5ea06e
improve name resolution
jacobmarks b71e3fb
only download necessary media files for FiftyOneDatasets
jacobmarks e2f55af
removing version validation
jacobmarks 6937faa
improve efficiency
jacobmarks 1179d9a
Merge pull request #4231 from voxel51/hf-hub-upgrades
jacobmarks a5291a6
Adds context without progress bars for upload/download
jacobmarks 91f026d
Merge pull request #4234 from voxel51/hf-hub-progress-bars
jacobmarks fd624fa
Update install instructions
jacobmarks File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit, but it would be nice if the screenshot showed the datasets viewer once the data has loaded successfully.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yes can absolutely change it! Great find
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: because the dataset is uploaded in FiftyOne format rather than Parquet, the dataset viewer does not know how to process it. I think perhaps the better thing to do would be to something integrate FiftyOne datasets on the hub with a public deployment like try.fiftyone.ai so people can "visualize with FiftyOne"