first commit

zzhanghub · Aug 4, 2020 · 074e35c · 074e35c
commit 074e35c
Show file tree

Hide file tree

Showing 9 changed files with 585 additions and 0 deletions.
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1 @@
+*__pycache__
diff --git a/READM.md b/READM.md
@@ -0,0 +1,120 @@
+<!-- PROJECT LOGO -->
+<br />
+<p align="center">
+  <a href="https://zhaozhang.net/coca.html">
+    <img src="img/eval_co-sod_logo.png" alt="Logo" width="210" height="100">
+  </a>
+
+  <h3 align="center">PyTorch-Based Evaluation Tool for Co-Saliency Detection</h3>
+
+  <p align="center">
+    Automatically evaluate 8 metrics and draw 3 types of curves
+    <br />
+    <a href="https://zhaozhang.net/coca.html"><strong>⭐ Project Home »</strong></a>
+    <br />
+  </p>
+</p>
+
+
+***
+**Eval Co-SOD** is an extended version of [Evaluate-SOD](https://github.com/Hanqer/Evaluate-SOD) for **co-saliency detection task**.
+It provides eight metrics and three curves:
+* Metrics:
+    * Mean Absolute Error (MAE)
+	* Maximum F-measure (max-Fm)
+	* Mean F-measure (mean-Fm)
+	* Maximum E-measure (max-Em)
+	* Mean E-measure (mean-Em)
+	* S-measure (Sm)
+	* Average Precision (AP)
+	* Area Under Curve (AUC)
+* Curves:
+	* Precision-Recall (PR) curve
+	* Receiver Operating Characteristic (ROC) curve
+	* F-measure curve
+
+
+evaluation tool for co-saliency detection methods.
+
+## Prerequisites
+* PyTorch >= 1.0
+
+
+## Usage
+
+### 1. Prepare your data
+The structure of `root_dir` should be organized as follows:
+```
+.
+├── gt
+│   ├── dataset1
+│   │   ├── accordion
+│   │   │   ├── 51499.png
+│   │   │   └── 186605.png
+│   │   └── alarm clock
+│   │       ├── 51499.png
+│   │       └── 186605.png
+│   ├── dataset2 ...
+│   └── dataset3 ...
+└── pred
+    └── method1
+    │   ├── dataset1
+    │   │   ├── accordion
+    │   │   │   ├── 51499.png
+    │   │   │   └── 186605.png
+    │   │   └── alarm clock
+    │   │       ├── 51499.png
+    │   │       └── 186605.png
+    │   ├── dataset2 ..
+    │   └── dataset3 ...
+    └──method2 ...
+```
+
+### 2. Evaluate on the 8 metrices
+1. Configure `eval.sh`
+```shell
+--methods method1+method2+method3 (Multiple items are connected with '+')
+--datasets dataset1+dataset2+dataset3
+--save_dir ../Result (Path to save results)
+--root_dir ../SalMaps
+```
+
+2. Run by
+```
+sh eval.sh
+```
+
+### 2. Draw the 3 types of curves
+1. Configure `plot_curve.sh`
+```shell
+--methods method1+method2+method3 (Multiple items are connected with '+')
+--datasets dataset1+dataset2+dataset3
+--out_dir ../Result/Curves (Path to save results)
+--res_dir ../Result
+```
+
+2. Run by
+```
+sh plot_curve.sh
+```
+
+## Citation
+If you find this tool is useful for your research, please cite the following papers.
+```
+@inproceedings{zhang2020gicd,
+ title={Gradient-Induced Co-Saliency Detection},
+ author={Zhang, Zhao and Jin, Wenda and Xu, Jun and Cheng, Ming-Ming},
+ booktitle={European Conference on Computer Vision (ECCV)},
+ year={2020}
+}
+
+@inproceedings{fan2020taking,
+  title={Taking a Deeper Look at the Co-salient Object Detection}, 
+  author={Fan, Deng-Ping and Lin, Zheng and Ji, Ge-Peng and Zhang, Dingwen and Fu, Huazhu and Cheng, Ming-Ming},   
+  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+  year={2020} 
+} 
+```
+
+## Contact
+If you have any questions, feel free to contact me via `zzhang🥳mail😲nankai😲edu😲cn`
diff --git a/Result/Detail/CoCA_GICD_retest.pth b/Result/Detail/CoCA_GICD_retest.pth
diff --git a/Result/result.txt b/Result/result.txt
@@ -0,0 +1 @@
+CoCA (GICD_retest): 0.1217 mae || 0.5107 max-fm || 0.5037 mean-fm || 0.7163 max-Emeasure || 0.7045 mean-Emeasure || 0.6566 S-measure || 0.4237 AP || 0.8009 AUC.
diff --git a/dataloader.py b/dataloader.py
@@ -0,0 +1,33 @@
+from torch.utils import data
+import os
+from PIL import Image
+
+
+class EvalDataset(data.Dataset):
+    def __init__(self, pred_root, label_root):
+        pred_dirs = os.listdir(pred_root)
+        label_dirs = os.listdir(label_root)
+
+        dir_name_list = []
+        for idir in pred_dirs:
+            if idir in label_dirs:
+                pred_names = os.listdir(os.path.join(pred_root, idir))
+                label_names = os.listdir(os.path.join(label_root, idir))
+                for iname in pred_names:
+                    if iname in label_names:
+                        dir_name_list.append(os.path.join(idir, iname))
+
+        self.image_path = list(
+            map(lambda x: os.path.join(pred_root, x), dir_name_list))
+        self.label_path = list(
+            map(lambda x: os.path.join(label_root, x), dir_name_list))
+
+    def __getitem__(self, item):
+        pred = Image.open(self.image_path[item]).convert('L')
+        gt = Image.open(self.label_path[item]).convert('L')
+        if pred.size != gt.size:
+            pred = pred.resize(gt.size, Image.BILINEAR)
+        return pred, gt
+
+    def __len__(self):
+        return len(self.image_path)
diff --git a/eval.sh b/eval.sh
@@ -0,0 +1 @@
+CUDA_VISIBLE_DEVICES=0 python main.py --methods GICD_retest --datasets CoCA --save_dir ./Result --root_dir ../SalMaps
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		CoCA (GICD_retest): 0.1217 mae \|\| 0.5107 max-fm \|\| 0.5037 mean-fm \|\| 0.7163 max-Emeasure \|\| 0.7045 mean-Emeasure \|\| 0.6566 S-measure \|\| 0.4237 AP \|\| 0.8009 AUC.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		CUDA_VISIBLE_DEVICES=0 python main.py --methods GICD_retest --datasets CoCA --save_dir ./Result --root_dir ../SalMaps