Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test for Core_DotProduct.accuracy fails on RISC-V RVV #24358

Closed
1 of 4 tasks
hanliutong opened this issue Oct 4, 2023 · 5 comments · Fixed by #26666
Closed
1 of 4 tasks

Test for Core_DotProduct.accuracy fails on RISC-V RVV #24358

hanliutong opened this issue Oct 4, 2023 · 5 comments · Fixed by #26666
Assignees
@mshabunin
Labels
bug platform: riscv
Milestone
4.11.0

Comments

@hanliutong
Copy link
Contributor

System Information

OpenCV version: 4.8.0
Operating System / Platform: Ubuntu 22.04
Compiler & compiler version: Clang 16.0.6
QEMU: qemu-riscv64 version 8.0.3 (from RVV docker file: opencv-infrastructure/opencv-gha-dockerfile#20)

Detailed description

Test for Core_DotProduct.accuracy fails on RVV

If and only if this code block is enabled for RVV , unit test Core_DotProduct.accuracy fails due to too big difference (=1.20209e-08 > 1.11022e-12).

$ ~/sc-dt/tools/bin/qemu-riscv64 -cpu rv64,v=true,vlen=128,vext_spec=v1.0 -L ~/tool/rvbuild/sysroot/ ./bin/opencv_test_core --gtest_filter="Core_DotProduct.accuracy*"

/home/hanliutong/project/opencv/modules/ts/src/ts.cpp:618: Failure
Failed

        failure reason: Bad accuracy
        test case #4
        seed: 00000000000c5a60
-----------------------------------
        LOG:
output: Too big difference (=1.20209e-08 > 1.11022e-12) at element 0
input array 0 type=32sC1, size=(6, 5)
input array 1 type=32sC1, size=(6, 5)
ref output array 0 type=64fC1, size=(1, 4)
test_case_idx = 4

-----------------------------------

[  FAILED  ] Core_DotProduct.accuracy (20 ms)
[----------] 1 test from Core_DotProduct (22 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (30 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Core_DotProduct.accuracy

Steps to reproduce

  1. Enable the code block for RVV by adding CV_SIMD_SCALABE on line 2551
--- #if CV_SIMD_64F
+++ #if CV_SIMD_64F || CV_SIMD_SCALABLE_64F
  1. Rebuild and run the test

Issue submission checklist

  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
  • I updated to the latest OpenCV version and the issue is still there
  • There is reproducer code and related data files (videos, images, onnx, etc)
@hanliutong hanliutong added the bug label Oct 4, 2023
@hanliutong hanliutong mentioned this issue Oct 4, 2023
Merged
6 tasks
@asmorkalov asmorkalov changed the title Test for Core_DotProduct.accuracy fails on RVV Test for Core_DotProduct.accuracy fails on RISC-V RVV Oct 4, 2023
@hanliutong
Copy link
Contributor Author

Another failure that may be related: Imgproc_Hist_Compare.accuracy in Imgproc module,

[----------] 1 test from Imgproc_Hist_Compare
[ RUN      ] Imgproc_Hist_Compare.accuracy
/build/4_x_riscv_rvv-clang-qemu-lin64/opencv/modules/ts/src/ts.cpp:618: Failure
Failed

	failure reason: Bad accuracy
	test case #31
	seed: 00000000000c5a7b
-----------------------------------
	LOG:
The comparison result using the method #0 (Correlation)
	is inaccurate (=-0.001257, should be =-0.00125681)
test_case_idx = 31

-----------------------------------
	CONSOLE: ...................
-----------------------------------

[  FAILED  ] Imgproc_Hist_Compare.accuracy (121 ms)

@asmorkalov asmorkalov added this to the 4.11.0 milestone May 17, 2024
@asmorkalov
Copy link
Contributor

@hanliutong Is the issue still relevant?

@hanliutong
Copy link
Contributor Author

Yes, the issue still here. Just tested with clang 19.1 and gcc 14.2.

When we try to enable the CV_SIMD_SCALABLE_64F for dotProd_32s, whether use clang or gcc, the difference is too big (BPI-F3 board: 3.53886e-11 > 1.11022e-12; QEMU: 2.54209e-09 > 1.11022e-12)

BTW, do we really need precision e-12, how about relaxed accuracy requirements can be accepted to pass the test?

@asmorkalov
Copy link
Contributor

@vpisarev @mshabunin What do you think?

@mshabunin
Copy link
Contributor

Yes, I believe we can relax precision requirement for these tests.

@mshabunin mshabunin mentioned this issue Dec 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mshabunin mshabunin

Labels
bug platform: riscv
Projects
None yet
Milestone
4.11.0
Development

Successfully merging a pull request may close this issue.

RISC-V: enabled intrinsics in dotProd, relaxed test thresholds mshabunin/opencv
3 participants
@asmorkalov @mshabunin @hanliutong