This codebase contains code and models for paper Learning Vision from Models Rivals Learning Vision from Data:

@article{synclr2023,
  author  = {Tian, Yonglong and Fan, Lijie and Chen, Kaifeng and Katabi, Dina and Krishnan, Dilip and Isola, Phillip},
  journal = {Technical Report},
  title   = {Learning Vision from Models Rivals Learning Vision from Data},
  year    = {2023},
}

The pre-trained models can be downloaded from

• ViT-B/16, link
• ViT-L/14, link

For the SynCaps-150M generated in our paper, we will release it once the internal approval process is done.

For the generated images, we will try to see if we can release them.

Otherwise, the code to synthesize the captions and images can be found under the synthesis folder.

Check the README under the eval folder.

Our models were trained using Jax with Google internal computation frameworks. However, we provide a pytorch reference code under train_pytorch.

This is not an officially supported Google product.

Apache2 license.