[FLAVA]Use boolean pretrained flag to load ckpts #365
Conversation
[ghstack-poisoned]
ghstack-source-id: 9ea01321467e8e3bc8191969f7b09a4d1fd3cd1d Pull Request resolved: #365
facebook-github-bot
added
the
CLA Signed
Codecov ReportBase: 93.43% // Head: 93.43% // No change to project coverage 👍
Additional details and impacted files@@ Coverage Diff @@
## gh/ankitade/13/base #365 +/- ##
====================================================
Coverage 93.43% 93.43%
====================================================
Files 55 55
Lines 3262 3262
====================================================
Hits 3048 3048
Misses 214 214
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
@DeAnkita has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Using bool flag similar to omnivore since there is only one ckpt to load Test plan 1. Pretraining debug python -m flava.train config=flava/configs/pretraining/debug.yaml training.lightning.gpus=0 training.lightning.strategy=null Epoch 0: : 50it [03:10, 3.81s/it, loss=16.3, v_num=3, train/losses/mmm_text_loss=10.50, train/losses/mmm_image_loss=9.150, train/losses/itm_loss=0.838, train/losses/global_contrastive_loss=2.090, train/losses/mlm_loss=10.40, train/losses/mim_loss=9.180] 2. Finetune debug python -m flava.train config=flava/configs/pretraining/debug.yaml model.pretrained=True Epoch 0: : 50it [00:28, 1.78it/s, loss=6.78, v_num=4, train/losses/mmm_text_loss=0.456, train/losses/mmm_image_loss=6.150, train/losses/itm_loss=0.378, train/losses/global_contrastive_loss=3.170, 3. coco zero shot python -m flava.coco_zero_shot --data_root /datasets01/COCO/022719/val2017 --annotations /datasets01/COCO/022719/annotations/captions_val2017.json Differential Revision: [D41142483](https://our.internmc.facebook.com/intern/diff/D41142483) [ghstack-poisoned]
| if pretrained_model_key is not None: | ||
| flava.load_model(FLAVA_MODEL_MAPPING[pretrained_model_key]) | ||
| if pretrained: | ||
| flava.load_model(FLAVA_MODEL_MAPPING["flava_full"]) |
There was a problem hiding this comment.
i can move the flava_full string to constant in followup PR
|
@DeAnkita has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@DeAnkita has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Stack from ghstack (oldest at bottom):
Using bool flag similar to omnivore since there is only one ckpt to load
Test plan
Pretraining debug
python -m flava.train config=flava/configs/pretraining/debug.yaml training.lightning.gpus=0 training.lightning.strategy=null
Epoch 0: : 50it [03:10, 3.81s/it, loss=16.3, v_num=3, train/losses/mmm_text_loss=10.50, train/losses/mmm_image_loss=9.150, train/losses/itm_loss=0.838, train/losses/global_contrastive_loss=2.090, train/losses/mlm_loss=10.40, train/losses/mim_loss=9.180]
Finetune debug
python -m flava.train config=flava/configs/pretraining/debug.yaml model.pretrained=True
Epoch 0: : 50it [00:28, 1.78it/s, loss=6.78, v_num=4, train/losses/mmm_text_loss=0.456, train/losses/mmm_image_loss=6.150, train/losses/itm_loss=0.378, train/losses/global_contrastive_loss=3.170,
coco zero shot
python -m flava.coco_zero_shot --data_root /datasets01/COCO/022719/val2017 --annotations /datasets01/COCO/022719/annotations/captions_val2017.json
Differential Revision: D41142483