Skip to content

Latest commit

 

History

History

segmentation

README.md

InternImage for Semantic Segmentation

This folder contains the implementation of the InternImage for semantic segmentation.

Our segmentation code is developed on top of MMSegmentation v0.27.0.

Usage

Install

  • Clone this repo:
git clone https://github.com/GNOEYHEAT/Fisheye-InternImage.git
cd Fisheye-InternImage/segmentation
  • Create a conda virtual environment and activate it:
conda create -n fisheye python=3.7 -y
conda activate fisheye

For examples, to install torch==1.11 with CUDA==11.3 and nvcc:

conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch -y
conda install -c conda-forge cudatoolkit-dev=11.3 -y # to install nvcc

  • Install other requirements:

    note: conda opencv will break torchvision as not to support GPU, so we need to install opencv using pip.

conda install -c conda-forge termcolor yacs pyyaml scipy seaborn pip -y 
pip install opencv-python
pip install yapf==0.40.1
  • Install timm and mmcv-full and `mmsegmentation':
pip install -U openmim
mim install mmcv-full==1.5.0
mim install mmsegmentation==0.27.0
pip install timm==0.6.11 mmdet==2.28.1

if "ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall." error occurs when installing openmim:

conda remove pyyaml
pip install -U openmim
mim install mmcv-full==1.5.0
mim install mmsegmentation==0.27.0
pip install timm==0.6.11 mmdet==2.28.1
  • Compile CUDA operators
cd ./ops_dcnv3
sh ./make.sh
# unit test (should see all checking is True)
python test.py
  • You can also install the operator using .whl files DCNv3-1.0-whl

Data Preparation

Prepare datasets according to the directory structure guideline

Fisheye Transformation

To transform original data into fisheye-data, run:

python fisheye_transform.py

Submission Guideline

To submit our model on test data, run:

python test.py <config-file> <checkpoint> --out <pickle-file>
python submit.py --pkl_name <pickle-file>

You can download checkpoint files from here. Then place it to segmentation/work_dirs/exp_04.

For example, to inference the exp_04:

python test.py configs/samsung/exp_04.py work_dirs/exp_04/best_mIoU_iter_22000.pth --out results/exp_04.pkl

For example, to submit the exp_04:

python submit.py --pkl_name exp_04

Training

To train our model on preprocess_data, run:

sh dist_train.sh <config-file> <gpu-num>

or another run with a single GPU:

python train.py <config-file>

For example, to train exp_04 with 2 GPU on 1 node (total batch size 4), run:

sh dist_train.sh configs/samsung/exp_04.py 2 -seed 826

Citation

@article{wang2022internimage,
  title={InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions},
  author={Wang, Wenhai and Dai, Jifeng and Chen, Zhe and Huang, Zhenhang and Li, Zhiqi and Zhu, Xizhou and Hu, Xiaowei and Lu, Tong and Lu, Lewei and Li, Hongsheng and others},
  journal={arXiv preprint arXiv:2211.05778},
  year={2022}
}

Acknowledgements

This code is heavily borrowed from INTERNIMAGE