by Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li and Liqing Zhang

CVPR2020

In this repository, we release the PyTorch implementation and model of DoveNet: Deep Image Harmonization via Domain Verification.

• Linux
• Python 3
• CPU or NVIDIA GPU + CUDA CuDNN

• Clone this repo:

git clone https://github.com/bcmi/Image_Harmonization_Datasets.git
cd Image_Harmonization_Datasets

• Download the iHarmony4 dataset.
• Install PyTorch 1.2 and other dependencies (e.g., torchvision, visdom and dominate).
  • For Conda users, you can create a new Conda environment using conda env create -f environment.yaml.

• To view training results and loss plots, run python -m visdom.server and click the URL http://localhost:8097.
• Train a model:

#!./scripts/train_dovenet.sh
python train.py  --dataset_root <path_to_iHarmony4_dataset> --name experiment_name  --model dovenet --dataset_mode iharmony4 --is_train 1  --gan_mode wgangp  --norm instance --no_flip --preprocess none --netG s2ad

Remember to specify dataset_root and name in the corresponding place.

To see more intermediate results, you can check out visdom or ./checkpoints/experiment_name/web/index.html.

• Test the model:

#!./scripts/test_dovenet.sh
python test.py --dataset_root <path_to_iHarmony4_dataset> --name experiment_name --model dovenet --dataset_mode iharmony4 --netG s2ad --is_train 0  --norm instance --no_flip --preprocess none --num_test 7404

Remember to specify dataset_root and name in the corresponding places.

When testing, it prints the results of evaluation metrics MSE and PSNR. It also saves the harmonized outputs in ./results/experiment_name/latest_test/images/

Our pre-trained model is available on Baidu Cloud (access code: 8q8a) (Alternative_address). Download and save it at ./checkpoints/experiment_name_pretrain/latest_net_G.pth.

As both instance normalization and batch normalization perform well for our task, the model we provided here is the one using batch normalization (specify the "norm" with "batch" to get reasonable results).

To test its performance on iHarmony4 dataset, using:

python test.py --dataset_root <path_to_iHarmony4_dataset> --name experiment_name_pretrain --model dovenet --dataset_mode iharmony4 --netG s2ad --is_train 0  --norm batch --no_flip --preprocess none --num_test 7404

Note to specify dataset_root and name in the corresponding place.

The expected quantitative results is as the following table. The expected results of our released model are slightly different from those reported in our paper as we are releasing an overall better model.

If you use this code for your research, please cite our paper using the following BibTeX [preprint]:

@inproceedings{DoveNet2020,
title={DoveNet: Deep Image Harmonization via Domain Verification},
author={Wenyan Cong and Jianfu Zhang and Li Niu and Liu Liu and Zhixin Ling and Weiyuan Li and Liqing Zhang},
booktitle={CVPR},
year={2020}}

Code borrows heavily from pix2pix.