Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cv2.normalize with NORM_MINMAX produces small values below set minimum on float32 #26588

Open
3 of 4 tasks
isenberg opened this issue Dec 7, 2024 · 11 comments
Open
3 of 4 tasks

Comments

@isenberg
Copy link

isenberg commented Dec 7, 2024

System Information

OpenCV python version: 4.10.0
Operating System / Platform: macOS 15.1.1 arm64 M2
Python version: 3.9.6 (/usr/bin/python3 provided by macOS)

Detailed description

The following code shows unexpected small negative numbers when trying to normalize an image between 0 and 1 while a comparison, which is mathematically not exactly the same, works fine. Note: The comparison img1 is keeping the offset from 0, I just used it for a quick check.

This appears to be the same as #6125 and #6170 which apparently has been fixed once in dev but then has been everted due to performance impacts.

This bug, if I don't have a misunderstanding of the OpenCV NORM_MINMAX definition, is serious as even small negative numbers can lead to critical follow up errors. For safety one can of course follow it with a img = np.clip(img, 0, 1) which is advised anyway for critical applications, but the result is mathematically slightly wrong.

If the performance impact can still be seen today with a fix, then I suggest to add at least internally a clip.

Datatype of img is np.float32. Changing to np.double like suggested in the older bug doesn't change anything.

print(cv2.__version__)
print(img.min(axis=(0, 1)), img.max(axis=(0, 1)))
img1 = cv2.normalize(img, None, 0, 1, norm_type=cv2.NORM_MINMAX)
img2 = img / max(img.max(axis=(0, 1)))
print(img1.min(axis=(0, 1)), img1.max(axis=(0, 1)))
print(img2.min(axis=(0, 1)), img2.max(axis=(0, 1)))

4.10.0
[0.02032561 0.04122998 0.04707103] [0.6832291  0.98649204 0.9428176 ]
[-1.0251444e-10  2.1636410e-02  2.7682003e-02] [0.68611723 0.99999994 0.95479614]
[0.02060392 0.04179454 0.04771557] [0.6925845 1.        0.9557276]

If needed I guess I would be able to attach a reproducer input image and code.

Steps to reproduce

On arm64 macOS:

#!/usr/bin/env python3
import cv2
import numpy as np
m = np.array([[ 1888, 1692, 369, 263, 199, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 575, 574, 63, 12]], dtype=np.float32)
print(cv2.__version__)
print(m.min(axis=(0, 1)), m.max(axis=(0, 1)))
mn = cv2.normalize(m, None, 0, 1, norm_type=cv2.NORM_MINMAX)
print(mn.min(axis=(0, 1)), mn.max(axis=(0, 1)))

Writes wrong result, small negative min:

% ~/python311/bin/python3 testnorm.py
4.10.0
12.0 1888.0
-2.3283064e-10 1.0

Issue submission checklist

  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files (videos, images, onnx, etc)
@AliHaroonT
Copy link

Any update regarding this? I'm also facing same issue.
Right now I'm manually normalizing the pixel values using the solution mentioned here (https://stackoverflow.com/questions/29100722/equivalent-im2double-function-in-opencv-python)

@denshade
Copy link

Not sure if useful, but this seems to work fine on Windows.
The above code yields the following on Windows:
4.10.0
[190 145 161] [249 244 228]
[0 0 0] [1 1 1]
[0.76305221 0.58232932 0.64658635] [1. 0.97991968 0.91566265]

@isenberg
Copy link
Author

Still produces the wrong results on arm64 macOS when using float 32.

Reproducer:

#!/usr/bin/env python3
import cv2
import numpy as np
m = np.array([[ 1888, 1692, 369, 263, 199, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 575, 574, 63, 12]], dtype=np.float32)
print(cv2.__version__)
print(m.min(axis=(0, 1)), m.max(axis=(0, 1)))
mn = cv2.normalize(m, None, 0, 1, norm_type=cv2.NORM_MINMAX)
print(mn.min(axis=(0, 1)), mn.max(axis=(0, 1)))

Result is small negative which is wrong:

% ~/python311/bin/python3 testnorm.py
4.10.0
12.0 1888.0
-2.3283064e-10 1.0

@denshade
Copy link

Confirmed reproducer triggers a problem on Windows as well.

@isenberg isenberg changed the title cv2.normalize with NORM_MINMAX produces small values below set minimum cv2.normalize with NORM_MINMAX produces small values below set minimum on float32 Dec 20, 2024
@denshade
Copy link

denshade commented Dec 21, 2024

I was able to reproduce it in the C++ code

TEST(Normalize, regression_5877_inplace_change_type)
{
    std::vector<float> initial_values = { 1888, 1692, 369, 263, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 12 };
    Mat m(Size(16, 1), CV_32F, initial_values.data());
    normalize(m, m, 0, 1, NORM_MINMAX);
    float value = m.at<float>(0, 15);
    ASSERT_EQ(0, value);
}

test_arithm.cpp(2228): error: Expected equality of these values:
0
value
Which is: -2.32831e-10

@denshade
Copy link

I've traced the problem to the following line
opencv\modules\core\src\norm.cpp:1388. It's obviously a rounding error.

        if( rtype == CV_32F )
        {
            scale = (float)scale; //<---
            shift = (float)dmin - (float)(smin*scale);
        }

Changing the code to this:

        if( rtype == CV_32F )
        {
            scale = scale; //<--- Or you can just omit this line.
            shift = (float)dmin - (float)(smin*scale);
        }

Not rounding this yields
2.32831e-10 Which is > 0.

@isenberg
Copy link
Author

The line scale = (float)scale indeed doesn't look like making sense as smin is a double and in the next line the result is rounded anyway.

@denshade
Copy link

Added a small PR for this:
#26658 Feedback welcome.

@denshade
Copy link

I've created some code to find the smallest fit within the boundaries.

It appears tests are failing because it expects:

    TEST_CYCLE() cv::normalize(src, dst, 20., 100., NORM_MINMAX);
    SANITY_CHECK(dst, 1e-6, ERROR_RELATIVE);

expect_max evaluates to 100,
actual_max evaluates to 99.997268676757812, and
eps evaluates to 9.9999999999999991e-05.

If I'm not mistaking this means it allows to be outside of the interval. I hope an active contributor with more context can pitch in and decide what is most important:
a. the normalized results must fall within the interval
b. the normalized results may fall outside of the interval, in such a way that it is minimized.

@isenberg
Copy link
Author

That's too much change now and could cause the performance problems again the first try years ago caused which lead to being it reverted.

What about just clamping the result to the min value if it exceeds it? I.e. just adding something like this after shift = dmin - smin*scale;

if (shift < dmin)
shift = dmin

@isenberg
Copy link
Author

isenberg commented Dec 22, 2024

Ah I see the problem, the actual calculation is done in convertTo and not in the function where the current changes apply. Then the performance change risk is low.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants