Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics review #13

Merged
merged 18 commits into from
Sep 30, 2020
Merged

Metrics review #13

merged 18 commits into from
Sep 30, 2020

Conversation

markusuwe
Copy link
Contributor

@markusuwe markusuwe commented Sep 21, 2020

Checking each of the metrics so we have peace of mind that these implementations are 100% correct or at least ambiguities are documented.

All values are reasonable now with the exception of UIQ which dropped by about 50 %. Can you please have a thorough look if I introduced an error there?

  • RMSE
  • PSNR
  • SSIM
  • FSIM
  • ISSM
  • SRE
  • SAM
  • UIQ

@markusuwe markusuwe requested a review from nekhtiari September 21, 2020 12:51
@markusuwe markusuwe marked this pull request as ready for review September 25, 2020 07:42
Copy link
Contributor

@rodrigoalmeida94 rodrigoalmeida94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

"""
Structural SIMularity index
"""
_assert_image_shapes_equal(org_img, pred_img, "SSIM")

return structural_similarity(org_img, pred_img, data_range=data_range, multichannel=True)
return structural_similarity(org_img, pred_img, data_range=max_p, multichannel=True)


def sliding_window(image, stepSize, windowSize):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question of the processing time,

  • should we make the step size and window size exposed in the parameters? Just in case we want to speed it up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Did that.
BTW, it might now be faster as the compute-intensive operations only have to consider one band and not all of them.

Comment on lines +251 to +252
# The original paper states that SAM values are expressed as radians, while e.g. Lanares
# et al. (2018) use degrees. We therefore made this configurable, with degree the default
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment on lines +5 to +12
* <i><a href="https://en.wikipedia.org/wiki/Root-mean-square_deviation">Root mean square error (RMSE)</a></i>,
* <i><a href="https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio">Peak signal-to-noise ratio (PSNR)</a></i>,
* <i><a href="https://en.wikipedia.org/wiki/Structural_similarity">Structural Similarity Index (SSIM)</a></i>,
* <i><a href="https://www4.comp.polyu.edu.hk/~cslzhang/IQA/TIP_IQA_FSIM.pdf">Feature-based similarity index (FSIM)</a></i>,
* <i><a href="https://www.tandfonline.com/doi/full/10.1080/22797254.2019.1628617">Information theoretic-based Statistic Similarity Measure (ISSM)</a></i>,
* <i><a href="https://www.sciencedirect.com/science/article/abs/pii/S0924271618302636">Signal to reconstruction error ratio (SRE)</a></i>,
* <i><a href="https://ntrs.nasa.gov/citations/19940012238">Spectral angle mapper (SAM)</a></i>, and
* <i><a href="https://ece.uwaterloo.ca/~z70wang/publications/quality_2c.pdf">Universal image quality index (UIQ)</a></i>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@nekhtiari
Copy link
Owner

LGTM

@markusuwe markusuwe merged commit f0ebc7a into master Sep 30, 2020
@markusuwe markusuwe deleted the metrics_review branch October 15, 2020 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants