• Documents for poster, slides, thesis, and main results are available in the docs and all of the results are in the corresponding notebooks.
• Code for the UKBB usecase is avialable in more-scenarios
• Project setup for UKBB usecase is in this readme

In healthcare, the workload is always increasing, and the workforce is limited, which calls for automation of certain tasks to save time. An example showcasing this idea is Rwanda, where in 2015, there were only 11 radiologists available for a population of 12 million [62]. Fairness is a major concern in in healthcare, where the models are used to make decisions that affect human lives.

We would like to leverage semi-supervised learning, patient textual information and Distribution Alignment [3] to mitigate bias. Using semi-supervised helps with scarcity of data in low-data regimes and provides a means of representation learning which could offer help with bias mitigation.

We employ a three-stage pipeline where we employ a number of techniques. For the preprocessing step we have data balanacing and slice selection. We employ data balancing based on protected attributes to build different train and test sets. Our slice selection strategy includes using either all slices or only those with the lowest error scores.

The second stage is method choice where we use two training methods: fully-supervised and the semi-supervised pipeline from UniMatch [1].

The third stage is the model choice where we tried three segmentation models: Unet [2], UNet Multi-Modal which uses image features and text embeddings from BERT [4], and UNet Multi-Task which learns both classification and segmentation.

• No technique could completely mitigate bias for all protected groups and intersectional groups.
• Lack of bias between groups does not preclude bias within intersectional groups.
• Incorporating ethnicity label into certain pipelines lowers overall performance and sometimes introduced disparities between more groups.
• We cannot definitively say that the cause of the bias is the encoding of sex or ethnicity features in the images.
• Using different test sets resulted in varied behavior when analyzing bias.

This codebase contains a strong re-implementation of FixMatch in the field of semi-supervised semantic segmentation, as well as the official PyTorch implementation of our UniMatch in the natural, remote sensing, and medical scenarios.

Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation
Lihe Yang, Lei Qi, Litong Feng, Wayne Zhang, Yinghuan Shi
In Conference on Computer Vision and Pattern Recognition (CVPR), 2023

We provide a list of Awesome Semi-Supervised Semantic Segmentation works.

You can check our training logs for convenient comparisons during reproducing.

Note: we have added and updated some results in our camera-ready version. Please refer to our latest version.

Labeled images are sampled from the original high-quality training set. Results are obtained by DeepLabv3+ based on ResNet-101 with training size 321.

Results are obtained by DeepLabv3+ based on ResNet-50/101. We reproduce U²PL results on ResNet-50.

Note: the results differ from our arXiv-V1 because we change the confidence threshold from 0.95 to 0, and change the ResNet output stride from 8 to 16. Therefore, it is currently more efficient to run.

You can click on the numbers to be directed to corresponding checkpoints.

Results are obtained by DeepLabv3+ based on Xception-65.

You can click on the numbers to be directed to corresponding checkpoints.

We also apply our UniMatch in the scenarios of semi-supervised remote sensing change detection and medical image segmentation, achieving tremendous improvements over previous methods:

• Remote Sensing Change Detection [training logs]
• Medical Image Segmentation [training logs]

cd UniMatch
conda create -n unimatch python=3.10.4
conda activate unimatch
pip install -r requirements.txt
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html

ResNet-50 | ResNet-101 | Xception-65

├── ./pretrained
    ├── resnet50.pth
    ├── resnet101.pth
    └── xception.pth

• Pascal: JPEGImages | SegmentationClass
• Cityscapes: leftImg8bit | gtFine
• COCO: train2017 | val2017 | masks

Please modify your dataset path in configuration files.

The groundtruth masks have already been pre-processed by us. You can use them directly.

├── [Your Pascal Path]
    ├── JPEGImages
    └── SegmentationClass
    
├── [Your Cityscapes Path]
    ├── leftImg8bit
    └── gtFine
    
├── [Your COCO Path]
    ├── train2017
    ├── val2017
    └── masks

# use torch.distributed.launch
sh scripts/train.sh <num_gpu> <port>
# to fully reproduce our results, the <num_gpu> should be set as 4 on all three datasets
# otherwise, you need to adjust the learning rate accordingly

# or use slurm
# sh scripts/slurm_train.sh <num_gpu> <port> <partition>

To train on other datasets or splits, please modify dataset and split in train.sh.

Modify the method from 'unimatch' to 'fixmatch' in train.sh.

Modify the method from 'unimatch' to 'supervised' in train.sh, and double the batch_size in configuration file if you use the same number of GPUs as semi-supervised setting (no need to change lr).

If you find this project useful, please consider citing:

@inproceedings{unimatch,
  title={Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation},
  author={Yang, Lihe and Qi, Lei and Feng, Litong and Zhang, Wayne and Shi, Yinghuan},
  booktitle={CVPR},
  year={2023}
}

We have some other works on semi-supervised semantic segmentation:

• [CVPR 2022] ST++
• [CVPR 2023] AugSeg
• [CVPR 2023] iMAS