This is the official repository for the Earth Intelligence Engine. This code trains and evaluates a deep generative vision model (GAN) to synthesize physically-consistent imagery of future floods. The code also trains a flood segmentation model on aerial imagery.

git clone --recursive git@github.com:blutjens/earth-eie.git
cd earth-eie
conda env create -f conda.yaml
conda activate eie_vision
pip install -e .

We recommend setting up your environment with conda. If you're unfamiliar with conda, read this intro.

Why git clone --recursive? Because we have at least one git submodules for hosting models. This means you'll need to run git submodule update when updating your remote.

Our full dataset, eie-earth-intelligence-engine, is available at huggingface. To download the dataset via git lfs please follow the instructions in the dataset README.md

• For the main model follow the notebook at link. This notebook contains the terminal commands to train the flood im2im model on xbd2xbd. After training the model is used to create predictions over the test set and the flood segmentation model is used to create flood masks of the generated imagery.
• Monitor the training by opening index.html

• The VAEGAN can be retrained with link.
• The hand-generated baseline can be created with [link]("sandbox/Color Baseline/Segment flood_color.ipynb").

• Evaluate the imagery with eval_main() as called in evaluate_notebook.ipynb

• Train, evaluate the flood segmentation model by following our other repository eie-flood-seg
• Copy and paste the model weights from checkpoints/temp/ into pretrained/

• Follow the notebook Train_conditional_binary_scratch_naip.ipynb

• Train an Arctic sea ice segmentation model with arctic-sea-ice-seg
• The code for generating reforestation visualizations is currently not available.

• xbd2xbd: Execute the steps in our eie-preprocessing repository to download and process the dataset. The first step will be to download the raw xBD flood imagery from xview by following the script at: eie-preprocessing/scripts/download_xBD_geotiles.sh
• xbd-seg: Hand-label data in xbd2xbd
• {naip2xbd, naip2hou, hou-seg}: Follow the instructions in the paper.
• arctic: Follow the instructions in full-pipeline/pipeline.sh in the arctic-sea-ice repository
• {forest, forest-gtm}: Follow the instructions in the paper.

• Visualize the generated imagery as a large geospatial map with align_slosh_w_naip.ipynb -> "Create large tif from generated imagery"

- archive: legacy code and documents
- configs: hyperparameters for the tested models
- data: placeholder for raw, interim, and processed data 
- docs: documentation, references, and figures
- pretrained: placeholder for model checkpoints
- results: generated imagery
- sandbox: prototyping scripts and notebooks
- scripts: important scripts and notebooks
- src: model source code, forked from existing git repositories
- temp: temporary results while training the models

@article{lutjens2024eie,
  author = {Lütjens, Björn and Leshchinskiy, Brandon and Boulais, Océane and Chishtie, Farrukh and Díaz-Rodríguez, Natalia and Masson-Forsythe, Margaux and Mata-Payerro, Ana and Requena-Mesa, Christian and Sankaranarayanan, Aruna and Piña, Aaron and Gal, Yarin and Raïssi, Chedy and Lavin, Alexander and Newman, Dava},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  title={Generating Physically-Consistent Satellite Imagery for Climate Visualizations}, 
  year = {2024},
  doi={10.1109/TGRS.2024.3493763}
}