Skip to content

Commit

Permalink
Merge of PR #1022
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 434538342
  • Loading branch information
ychzhang authored and copybara-github committed Mar 14, 2022
1 parent 88ce4b9 commit 3dcd301
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions aqt/jax/imagenet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,8 @@ tfds.builder('imagenet2012').download_and_prepare(
"
```

Note that the preprocessing will extract and prepare the ImageNet dataset in a new folder defined by the environment variable `TFDS_DATA_DIR`, which is by default `~/tensorflow_datasets` under the home directory ([reference](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetBuilder)). If one has insufficient disk space under `/home` (about 150 GB), please use
`$ export TFDS_DATA_DIR=/path/to/preprocess/folder`
Note that the preprocessing will extract and prepare the ImageNet dataset in a new folder defined by the environment variable `TFDS_DATA_DIR`, which is by default `~/tensorflow_datasets` under the home directory ([reference](https://www.tensorflow.org/datasets/api_docs/python/tfds/core/DatasetBuilder)). If one has insufficient disk space under `/home` (about 150 GB), please use
`$ export TFDS_DATA_DIR=/path/to/preprocess/folder`
in the command line _before executing the tfds python code block_ to specify a different folder for storing the processed dataset.

After the processing ends, copy the `tensorflow_datasets` directory (where the processed data are stored) to the cloud storage bucket. It can then be used across different VMs.
Expand Down

0 comments on commit 3dcd301

Please sign in to comment.