Skip to content

Commit

Permalink
Fixed and simplified the description of the readme files.
Browse files Browse the repository at this point in the history
  • Loading branch information
@lartpang
lartpang committed Dec 16, 2023
1 parent c413a23 commit 980b4da
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 43 deletions.
29 changes: 6 additions & 23 deletions readme.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

## Introduction

A simple and efficient implementation of SOD metrics
A simple and efficient implementation of SOD metrics.

- Based on `numpy` and `scipy`
- Verification based on Fan's matlab code <https://github.com/DengPingFan/CODToolbox>
Expand All @@ -23,7 +23,7 @@ Your improvements and suggestions are welcome.

### Related Projects

- A Python-based salient object detection and video object segmentation evaluation toolbox. <https://github.com/lartpang/Py-SOD-VOS-EvalToolkit>
- [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit): A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection

### Supported Metrics

Expand All @@ -35,7 +35,7 @@ Your improvements and suggestions are welcome.
| Multi-Scale IoU | bin | | `MSIoU` |
| E-measure ($E_{m}$) | max,avg,adp | | `Emeasure` |
| F-measure (old) ($F_{beta}$) | max,avg,adp | | `Fmeasure` |
| F-measure (old) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |
| F-measure (new) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |
| BER | max,avg,adp,bin | bin | `FmeasureV2`+`BERHandler` |
| Dice | max,avg,adp,bin | bin | `FmeasureV2`+`DICEHandler` |
| FPR | max,avg,adp,bin | bin | `FmeasureV2`+`FPRHandler` |
Expand All @@ -48,24 +48,6 @@ Your improvements and suggestions are welcome.
| Specificity | max,avg,adp,bin | bin | `FmeasureV2`+`SpecificityHandler` |
| TNR | max,avg,adp,bin | bin | `FmeasureV2`+`TNRHandler` |
| TPR | max,avg,adp,bin | bin | `FmeasureV2`+`TPRHandler` |
## TODO List

- [x] Speed up the calculation of Emeasure.
- [x] Add the necessary documentation for different functions.

## Compared with [Matlab Code from Fan](https://github.com/DengPingFan/CODToolbox)

In our comparison (the test code can be seen under the `test` folder), the result is consistent with Fan's code, as follows:

```text
ours: Smeasure:0.903; wFmeasure:0.558; MAE:0.037; adpEm:0.941; meanEm:0.957; maxEm:0.967; adpFm:0.582; meanFm:0.577; maxFm:0.589
matlab: Smeasure:0.903; wFmeasure:0.558; MAE:0.037; adpEm:0.941; meanEm:0.957; maxEm:0.967; adpFm:0.582; meanFm:0.577; maxFm:0.589.
```

**NOTE**

- The matlab code needs to change `Bi_sal(sal>threshold)=1;` to `Bi_sal(sal>=threshold)=1;` in <https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102>. For related discussion, please see [the issue](https://github.com/DengPingFan/CODToolbox/issues/1).
- 2021-12-20 (version `1.3.0`): Due to the difference between numpy and matlab, in version `1.2.x`, there are very slight differences on some metrics between the results of the matlab code and ours. The [recent PR](https://github.com/lartpang/PySODMetrics/pull/3) alleviated this problem. However, there are still very small differences on E-measure. The results in most papers are rounded off to three or four significant figures, so, there is no obvious difference between the new version and the version `1.2.x` for them.

## Usage

Expand All @@ -81,8 +63,9 @@ The core files are in the folder `py_sod_metrics`.

## Reference

- <https://github.com/DengPingFan/CODToolbox>
- By DengPingFan(<https://github.com/DengPingFan>)
- [Matlab Code](https://github.com/DengPingFan/CODToolbox) by DengPingFan(<https://github.com/DengPingFan>): In our comparison (the test code can be seen under the `test` folder), the result is consistent with the code.
- The matlab code needs to change `Bi_sal(sal>threshold)=1;` to `Bi_sal(sal>=threshold)=1;` in <https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102>. For related discussion, please see [the issue](https://github.com/DengPingFan/CODToolbox/issues/1).
- 2021-12-20 (version `1.3.0`): Due to the difference between numpy and matlab, in version `1.2.x`, there are very slight differences on some metrics between the results of the matlab code and ours. The [recent PR](https://github.com/lartpang/PySODMetrics/pull/3) alleviated this problem. However, there are still very small differences on E-measure. The results in most papers are rounded off to three or four significant figures, so, there is no obvious difference between the new version and the version `1.2.x` for them.
- <https://en.wikipedia.org/wiki/Precision_and_recall>

```text
Expand Down
27 changes: 7 additions & 20 deletions readme_zh.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,18 @@

## 介绍

一份简单有效的 SOD 指标实现
一份简单有效的 SOD 指标实现。

- 基于`numpy`和极少量`scipy.ndimage`代码
- 基于 DengPing Fan <https://github.com/DengPingFan/CODToolbox>
- 基于 DengPing Fan <https://github.com/DengPingFan/CODToolbox> 进行对比验证
- 结构简单，易于扩展
- 代码轻量且快速

欢迎您的改进和建议。

### 相关项目

- A Python-based salient object detection and video object segmentation evaluation toolbox. <https://github.com/lartpang/Py-SOD-VOS-EvalToolkit>
- [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit): A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection

### 支持的指标

Expand All @@ -35,7 +35,7 @@
| Multi-Scale IoU | bin | | `MSIoU` |
| E-measure ($E_{m}$) | max,avg,adp | | `Emeasure` |
| F-measure (old) ($F_{beta}$) | max,avg,adp | | `Fmeasure` |
| F-measure (old) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |
| F-measure (new) ($F_{beta}$, $F_{1}$) | max,avg,adp,bin | bin | `FmeasureV2`+`FmeasureHandler` |
| BER | max,avg,adp,bin | bin | `FmeasureV2`+`BERHandler` |
| Dice | max,avg,adp,bin | bin | `FmeasureV2`+`DICEHandler` |
| FPR | max,avg,adp,bin | bin | `FmeasureV2`+`FPRHandler` |
Expand All @@ -49,20 +49,6 @@
| TNR | max,avg,adp,bin | bin | `FmeasureV2`+`TNRHandler` |
| TPR | max,avg,adp,bin | bin | `FmeasureV2`+`TPRHandler` |

## 与[范的 Matlab 代码](https://github.com/DengPingFan/CODToolbox)的比较

在我们的测试中 (测试代码可见`test`文件夹下内容)，结果与 Fan 的代码一致，如下:

```text
ours: Smeasure:0.903; wFmeasure:0.558; MAE:0.037; adpEm:0.941; meanEm:0.957; maxEm:0.967; adpFm:0.582; meanFm:0.577; maxFm:0.589
matlab: Smeasure:0.903; wFmeasure:0.558; MAE:0.037; adpEm:0.941; meanEm:0.957; maxEm:0.967; adpFm:0.582; meanFm:0.577; maxFm:0.589.
```

**注意**

- matlab 代码需要将<https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102> 的`Bi_sal(sal>threshold)=1;`改为` Bi_sal(sal>=threshold)=1;`。细节可见 [相关讨论](https://github.com/DengPingFan/CODToolbox/issues/1)。
- 2021-12-20 (Version `1.3.0`)：由于 numpy 和 matlab 的不同，在 `1.2.x` 版本中，matlab 代码的结果与我们的结果在某些指标上存在非常细微的差异。[最近的 PR](https://github.com/lartpang/PySODMetrics/pull/3) 缓解了这个问题。但是，在 E-measure 上仍然存在非常小的差异。大多数论文中的结果都四舍五入到三四位有效数字，因此，新版本与“1.2.x”版本之间没有明显差异。

## 使用

核心文件在文件夹 `py_sod_metrics` 中。
Expand All @@ -77,8 +63,9 @@ matlab: Smeasure:0.903; wFmeasure:0.558; MAE:0.037; adpEm:0.941; meanEm:0.957; m

## 参考

- <https://github.com/DengPingFan/CODToolbox>
- By DengPingFan(<https://github.com/DengPingFan>)
- [Matlab Code](https://github.com/DengPingFan/CODToolbox) by DengPingFan(<https://github.com/DengPingFan>): 在我们的测试中 (测试代码可见`test`文件夹下内容)，结果与 Fan 的代码一致。
- matlab 代码需要将<https://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102> 的`Bi_sal(sal>threshold)=1;`改为` Bi_sal(sal>=threshold)=1;`。细节可见 [相关讨论](https://github.com/DengPingFan/CODToolbox/issues/1)。
- 2021-12-20 (Version `1.3.0`)：由于 numpy 和 matlab 的不同，在 `1.2.x` 版本中，matlab 代码的结果与我们的结果在某些指标上存在非常细微的差异。[最近的 PR](https://github.com/lartpang/PySODMetrics/pull/3) 缓解了这个问题。但是，在 E-measure 上仍然存在非常小的差异。大多数论文中的结果都四舍五入到三四位有效数字，因此，新版本与“1.2.x”版本之间没有明显差异。
- <https://en.wikipedia.org/wiki/Precision_and_recall>

```text
Expand Down

0 comments on commit 980b4da

Please sign in to comment.