read training strategy from config (#66)

Summary: * Read training strategy from config file * Add `strategy: ddp` to all existing config files * Can also pass in strategy from the terminal by `python train.py config=configs/pretraining/debug.yaml training.lightning.strategy=ddp` * Priority: terminal argument > config Pull Request resolved: #66 Test Plan: logged `self.trainer.strategy` in `FLAVAPreTrainingLightningModule` test 1 * command: `python train.py config=configs/pretraining/debug.yaml` * config: `strategy: ddp` * result: `DDPStrategy` test 2 * command: `python train.py config=configs/pretraining/debug.yaml training.lightning.strategy=deepspeed` * config: `strategy: ddp` * result: `DeepSpeedStrategy` Reviewed By: ebsmothers Differential Revision: D37124420 Pulled By: katrina433 fbshipit-source-id: a02cafba7ad784963a0c27ad543256fd5daaa417
facebookresearch · Jun 13, 2022 · 3beffd9 · 3beffd9
1 parent 27a0d2e
commit 3beffd9
Show file tree

Hide file tree

Showing 4 changed files with 3 additions and 1 deletion.
diff --git a/examples/flava/configs/finetuning/qnli.yaml b/examples/flava/configs/finetuning/qnli.yaml
@@ -8,6 +8,7 @@ training:
     progress_bar_refresh_rate: 50
     val_check_interval: 1000
     num_sanity_val_steps: 0
+    strategy: ddp
   lightning_checkpoint:
     dirpath: "."
     filename: flava-{epoch:02d}-{step}

diff --git a/examples/flava/configs/finetuning/rendered_sst2.yaml b/examples/flava/configs/finetuning/rendered_sst2.yaml
@@ -8,6 +8,7 @@ training:
     progress_bar_refresh_rate: 50
     val_check_interval: 100
     num_sanity_val_steps: 0
+    strategy: ddp
   lightning_checkpoint:
     dirpath: "."
     filename: flava-{epoch:02d}-{step}

diff --git a/examples/flava/configs/pretraining/debug.yaml b/examples/flava/configs/pretraining/debug.yaml
@@ -7,6 +7,7 @@ training:
     progress_bar_refresh_rate: 50
     val_check_interval: 10000
     num_sanity_val_steps: 0
+    strategy: ddp
   lightning_checkpoint:
     dirpath: "."
     filename: flava-{epoch:02d}-{step}

diff --git a/examples/flava/train.py b/examples/flava/train.py
@@ -68,7 +68,6 @@ def main():
     trainer = Trainer(
         **OmegaConf.to_container(config.training.lightning),
         callbacks=callbacks,
-        strategy="ddp",
     )
     ckpt_path = config.training.lightning_load_from_checkpoint