You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ERROR: Formula evaluation gave negative result for link 12
or link 2 is outputted, but the test requires
ERROR: Formula evaluation gave negative result for link 10
I assumed Windows was downloading a huge update during office hours again, or something else particular to my local test environment was causing the weird behaviour. But this test behaves very strangely in CI too. So I'm not sure if the test is flakey, overly reliant on the test environment, if it relies on a stochastic function within sDNA, or if there really is a bug (if so, how important is this part of the test? - do tests need to fail if error messages are wrong, failing for failing wrongly?).
I was able to get all the other tests to pass, by simply persevering, and rerunning any failed jobs. But not the Python 3.8 test, yet:
The text was updated successfully, but these errors were encountered:
JamesParrott
changed the title
The test in debug_test.py / quick_test.bat, that diffes against correctout.txt randomly fails. Stochastic behaviour (flakey test) or bug?
The test in debug_test.py / quick_test.bat, that diffs against correctout.txt randomly fails. Stochastic behaviour (flakey test) or bug?
Feb 14, 2024
Temporary work around. This issue is currently ignored in the regression tests by setting the environment variable ALLOW_NEGATIVE_FORMULA_ERROR_ON_ANY_LINK_PRESENT
Sometimes:
or link 2 is outputted, but the test requires
I assumed Windows was downloading a huge update during office hours again, or something else particular to my local test environment was causing the weird behaviour. But this test behaves very strangely in CI too. So I'm not sure if the test is flakey, overly reliant on the test environment, if it relies on a stochastic function within sDNA, or if there really is a bug (if so, how important is this part of the test? - do tests need to fail if error messages are wrong, failing for failing wrongly?).
I was able to get all the other tests to pass, by simply persevering, and rerunning any failed jobs. But not the Python 3.8 test, yet:
https://github.com/JamesParrott/sdna_plus/actions/runs/7907698164/job/21586911903
However it passed fine in Python 3.8.10 locally just now, so I doubt there's a Python 3.8 only bug:
The text was updated successfully, but these errors were encountered: