Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some cleanup for new executor test code, which accidentally fixes ASAN failures in ExecutorTestHelper #11283

Merged
merged 15 commits into from
Mar 19, 2020

Conversation

markuspf
Copy link
Contributor

@markuspf markuspf commented Mar 16, 2020

This PR escalated a bit, because while I was fixing a memory leak, it turned out that the Executor tests had multiple unrelated AqlItemBlockManager instances around which removed each other's dependencies blocks, leading to crashes.

To fix this, the ExecutorTestHelper now gets an AqlItemBlockManger on construction, and the whole Pipeline API is gone and hidden inside the test helper.

I also deleted a lot of unused test code, in particular the SubqueryExecutor test, which did nothing. Lets not pretend there are tests for things that aren't there.

@markuspf markuspf requested a review from mchacki March 16, 2020 18:25
@@ -74,7 +74,6 @@ set(ARANGODB_TESTS_SOURCES
Aql/SortLimit-test.cpp
Aql/SpliceSubqueryOptimizerRuleTest.cpp
Aql/SplicedSubqueryIntegrationTest.cpp
Aql/SubqueryExecutorTest.cpp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmmh, why does this fix remove the SubqueryExecutorTest and its 500 lines of test code?
Is that intentional? If so, why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test was not doing anything (most of the test code was #if 0d out), and rather than pretend that such a test is there, I decided to delete it.

We have an internal planning ticket to implement tests for this executor.

@jsteemann
Copy link
Contributor

jsteemann commented Mar 17, 2020

@markuspf : just tried this branch and ran the tests with it.
no leaks, but test execution stops very early with an assertion failure:

[----------] 4 tests from IdExecutionBlockTest
[ RUN      ] IdExecutionBlockTest.test_initialize_cursor_get
[       OK ] IdExecutionBlockTest.test_initialize_cursor_get (0 ms)
[ RUN      ] IdExecutionBlockTest.test_initialize_cursor_skip
[       OK ] IdExecutionBlockTest.test_initialize_cursor_skip (0 ms)
[ RUN      ] IdExecutionBlockTest.test_initialize_cursor_fullCount
[       OK ] IdExecutionBlockTest.test_initialize_cursor_fullCount (0 ms)
[ RUN      ] IdExecutionBlockTest.test_hardlimit_single_row_fetcher
2020-03-17T15:15:04Z [8524] FATAL assertion failed in /home/jsteemann/ArangoAsan/arangod/Aql/AqlItemBlockManager.cpp:174: blocks[numItems] == nullptr

I think this is a side effect of something else being broken.

@jsteemann
Copy link
Contributor

I tried the latest version and it still fails, though much later now:

[----------] 48 tests from SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest
[ RUN      ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/0
[       OK ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/0 (0 ms)
[ RUN      ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/1
[       OK ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/1 (1 ms)
[ RUN      ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/2
[       OK ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/2 (0 ms)
[ RUN      ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/3
[       OK ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery_empty_input/3 (0 ms)
[ RUN      ] SplicedSubqueryIntegrationTest/SplicedSubqueryIntegrationTest.single_subquery/0
2020-03-18T10:05:29Z [27663] FATAL assertion failed in /home/jsteemann/ArangoAsan/arangod/Aql/InputAqlItemRow.cpp:151: registerId < getNrRegisters()
Wed Mar 18 2020 11:05:30 GMT+0100 (Central European Standard Time) Finished: ABORTED Signal: 6 Time elapsed: 560.2714965343475 - 
``

@markuspf
Copy link
Contributor Author

…/arangodb into bug-fix/test-asan-fixes-cleanup
@markuspf markuspf changed the title Fix ASAN failures in ExecutorTestHelper and remove SubqueryExecutorTest Some cleanup for new executor test code, which accidentally fixes ASAN failures in ExecutorTestHelper Mar 18, 2020
@markuspf
Copy link
Contributor Author

@markuspf
Copy link
Contributor Author

@markuspf
Copy link
Contributor Author

Jenkins blue now

@mchacki mchacki merged commit 9998b4e into devel Mar 19, 2020
@mchacki mchacki deleted the bug-fix/test-asan-fixes-cleanup branch March 19, 2020 10:38
ObiWahn added a commit that referenced this pull request Mar 27, 2020
…ql-functions

* origin/devel: (25 commits)
  Bug fix/fixes 20200318 (#11319)
  updated CHANGELOG
  Feature/out of search in range (#11324)
  fix "fix" for collection figures (#11323)
  updated CHANGELOG
  compilation fixes for clang-10s more strict checking (#11316)
  fix failing query (#11317)
  KShortestPathsExecutor must reset its KShortestPathFinder, including all caches. (#11312)
  Feature/aql subquery execution block impl execute implementation expected number of rows (#11274)
  Add DTRACE points to measure request timings. (#11245)
  USE_STRICT_OPENSSL is Off by default
  Fix usesRevisionAsDocumentId population and add syncByRevision flag (#11314)
  Traversal Bugfix  (#11310)
  Bug fix/issue 11275 (#11299)
  added simple test (#11301)
  Fix some typos (#10173)
  Documentation/typos 2020-01-24 (#10975)
  Update CHANGELOG
  Some cleanup for new executor test code, which accidentally fixes ASAN failures in ExecutorTestHelper (#11283)
  LZ4 update (#11306)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants