Description of changes:

py1b is a shuffling algorithm that performs the final shuffle over fixed-size blocks instead of intra-shard like py1s. These units are presumably larger or much larger than single shards, leading to better shuffledness at the cost of having to download more shards to make progress.

naive is a shuffling algorithm that naively shuffles all-to-all. This is useful for single-node training on small data, where you want the most random shuffle possible. Statistically, this algorithm will result in all nodes downloading all shards, with those downloads all happening at the start of the epoch, bringing training to a crawl.

Also update the default shuffle algo from py1s to py1b.

Issue #, if available:

Merge Checklist:

Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

• I have read the contributor guidelines
• This is a documentation change or typo fix. If so, skip the rest of this checklist.
• I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the MosaicML team.
• I have updated any necessary documentation, including README and API docs (if appropriate).

Tests

• I ran pre-commit on my change. (check out the pre-commit section of prerequisites)
• I have added tests that prove my fix is effective or that my feature works (if appropriate).
• I ran the tests locally to make sure it pass. (check out testing)
• I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes.