This repo contains the code for our NeurIPS 2024 paper The Best of Both Worlds: On the Delimma of Out-of-Distribution Detection. We propose an learning objective for both OOD detection and generalization.
To get started with this repository, you'll need to follow these installation steps. Before proceeding, make sure you have anaconda installed.
pip3 install -r requirement.txt
We follow the POEM and OpenOOD to prepare the datasets. We provide links and instructions to download each dataset:
- CIFAR10 and CIFAR100: https://www.cs.toronto.edu/~kriz/cifar.html
- SVHN: download it and place it in the folder of
datasets/ood_datasets/svhn. - Textures: download it and place it in the folder of
datasets/ood_datasets/dtd. - Places365: download it and place it in the folder of
datasets/ood_datasets/places365. - LSUN: download it and place it in the folder of
datasets/ood_datasets/LSUN. - LSUN-resize: download it and place it in the folder of
datasets/ood_datasets/LSUN_resize. - iSUN: download it and place it in the folder of
datasets/ood_datasets/iSUN.
These datasets are used to evaluate OOD detection performance.
Our DUL is devised in a finetune manner. Follow the below instructions for standard pretraining:
cd CIFAR
python baseline.py --dataset cifar10
Then, finetune the pretrain model with the following instructions for both OOD detection and generalization purposes:
python finetune.py --method DUL --dataset cifar10 --epoch 20
python finetune.py --method DUL --dataset cifar100 --epoch 30
Evaluate the OOD detection and generalization performance with below instructions:
python test.py --dataset cifar10 --method DUL --score diff_entropy --epoch 19
python test.py --dataset cifar100 --method DUL --score diff_entropy --epoch 29
We are actively working on releasing the code for ImageNet experiments.
If you have any questions related to the code or the paper, feel free to email Qingyang Zhang (qingyangzhang@tju.edu.cn). If you encounter any problems when using the code, or want to report a bug, you can open an issue. Please try to specify the problem with details so we can help you better and quicker!
Please cite our paper if you find the repo helpful in your work:
@inproceedings{zhang2024best,
title={The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection},
author={Zhang, Qingyang and Feng, Qiuxuan and Zhou, Joey Tianyi and Bian, Yatao and Hu, Qinghua and Zhang, Changqing},
booktitle={Advances in Neural Information Processing Systems},
year={2024}
}
The code is developed based on Outlier Exposure, POEM and OpenOOD.