The IllumHarmony-Dataset for SIDNet: Learning Shading-aware Illumination Descriptor for Image Harmonization (IEEE TETCI 2023).

We constructed a large-scale synthetic image harmonization dataset (called IllumHarmony-Dataset by us) where each pair of unharmonized and ground-truth images contains shading variation caused by different illumination. Compared to the widely-used iHarmony4 dataset, the shading variation between the unharmonized and ground-truth images in our dataset is really in line with real-world image harmonization. It poses a greater challenge to existing image harmonization works.

Dataset construciton. The pipeline of dataset construction is given below. NOTE THAT our collected models/illumination are all captured from the real world, with the aim of achieving photo-realistic renderings. More details on dataset construction can be found in our paper.

Dataset statistics. Our dataset has a total of 143,390 training images and 22,048 test images, which cover a wide range of scenes and illumination conditions. We split the training set and the test set into four categories based on illumination conditions as reported in the following Table.

High-quality examples from our IllumHarmony-Dataset are shown in the figure below.

The IllumHarmony-Dataset (~87GB) can be downloaded from OneDrive or BaiduCloud. This dataset is licensed under a LICENSE.

We provide python scripts in the foloder ./data_utils for loading data.

loading training data. By running this script, you can get the composite training data from the rendered raw data. Note that for the input unharmonized composite image, its foreground illumination is randomly selected to increase the diversity of the IllumHarmony-Dataset.

python data_utils/train_dataloader.py

loading test data. Run the following script to get not only the input unharmonized image, mask and GT, but also the albedo image and the shading image.

python data_utils/test_dataloader.py

If you want to use the final composed images (including input, mask, and GT) directly, please download them from OneDrive or BaiduCloud.

We provide the Python script in the folder ./blender for rendering foreground images using your hdr panoramas and mesh. Notes: The Python script is tested in blender-2.83.9-linux64.

./blender --background --python render_demo.py -- 0 0

In addition to the IllumHarmony-Dataset, we also built a small image harmonization dataset consisting of 58 real composite images for user study. We provided our SIDNet results on this small dataset to facilitate possible future comparative studies.

If you use these data in your research, please cite:

@article{hu2024sidnet,
  title={{SIDNet}: Learning Shading-Aware Illumination Descriptor for Image Harmonization},
  author={Hu, Zhongyun and Nsampi, Ntumba Elie and Wang, Xue and Wang, Qing},
  journal={IEEE Transactions on Emerging Topics in Computational Intelligence}, 
  year={2024},
  volume={8},
  number={2},
  pages={1290-1302}
}

If you have any questions, please contact zy_h@mail.nwpu.edu.cn.