Add Exclusion Filters #705
Conversation
glubsy
commented
Sep 1, 2020
•
glubsy
commented
- Allow users to specify files to filter out during scan by using regular expressions
- Set the default directory states according to regular expressions.
- File names and file paths are matched against regular expressions: only regexes with path separators are matches against the full path to the file, other regexes are matched against the filename only. This allows for more precise filtering.
- Two different implementations: one around a python list, and one around a python dict, but same features (I think the dict implementation might be a bit more efficient, so we use that by default now).
- Regexes can be combined (union) for possibly better performance, although my experiments didn't show meaningful results.
- Also added test suites.
* Added initial draft for test suit * Fixed small logging bug
* Concatenating regexes into one Pattern might yield better performance under (un)certain conditions. * Filenames are tested against regexes with no os.sep in them. This may or may not be what we want to do. And alternative would be to test against the whole (absolute) path of each file, which would filter more agressively.
* We need to set the Details Dialog's previous instance to None when recreating a new Results window otherwise Qt crashes since we are probably dereferencing a dangling reference. * Also fixes Results tab not showing up when selecting it from the View menu.
* The custom QStackWidget+QTabBar class did not manage the tabs properly because the indices in the stackwidget were not aligned with the ones in the tab bar. * Properly disable exclude list action when it is the currently displayed widget. * Merge action callbacks for triggering ignore list or exclude list to avoid repeating code and remove unused checks for tab visibility. * Remove unused SetTabVisible() function.
* When recreating the Results window, the menubar had duplicate items added each time. * Removing the underlying C++ object is apparently enough to fix the issue. * SetParent(None) can still be used in case of floating windows
* Add testing feature to Exclusion dialog to allow users to test regexes against an arbitrary string. * Fixed test suites. * Improve comments and help dialog box.
|
This is based on the details dialog improvement branch. Regarding the regex union feature, here is a script I wrote to roughly compare performance based on time. |
* In some themes, the color picker widgets get stretched, while the color picker for the details dialog group doesn't. This should keep them a bit more consistent across themes.
* Sometimes, the splitter doesn't fully reach the table maximum height, and the scrollbar is still displayed on the right because a few pixels are still hidden. * It seems the splitter handle counts towards the total height of the widget (the table), so we add it to the maximum height of the table * The scrollbar disappears when we reach just above the actual table's height
|
Side note: just noticed detwinner also added a similar regex-based filter for file paths. 😉 |
There was a problem hiding this comment.
Few minor things here, also I am getting failures on the new directories and exclude tests.
FAILED core/tests/directories_test.py::TestExcludeList::test_exclude_refined - AssertionError: 6 != 5
FAILED core/tests/directories_test.py::TestExcludeList::test_get_state_returns_excluded_for_hidden_directories_and_files - AssertionError: 3 != 1
FAILED core/tests/directories_test.py::TestExcludeDict::test_exclude_refined - AssertionError: 6 != 5
FAILED core/tests/directories_test.py::TestExcludeDict::test_get_state_returns_excluded_for_hidden_directories_and_files - AssertionError: 3 != 1
FAILED core/tests/directories_test.py::TestExcludeListunion::test_exclude_refined - AssertionError: 6 != 5
FAILED core/tests/directories_test.py::TestExcludeListunion::test_get_state_returns_excluded_for_hidden_directories_and_files - AssertionError: 3 != 1
FAILED core/tests/directories_test.py::TestExcludeDictunion::test_exclude_refined - AssertionError: 6 != 5
FAILED core/tests/directories_test.py::TestExcludeDictunion::test_get_state_returns_excluded_for_hidden_directories_and_files - AssertionError: 3 != 1
FAILED core/tests/exclude_test.py::TestCaseCompiledList::test_compiled_files - AssertionError: 1 != 6
FAILED core/tests/exclude_test.py::TestCaseCompiledDict::test_compiled_files - AssertionError: 1 != 6
|
That's odd. I don't get any failure during the tests. The Continuous Integration system doesn't seem to produce these either. Could it be that these occur on Windows only? Edit: nevermind, I forgot you had set up a Windows machine for the CI too. Still no error there. |
|
@glubsy the tox environment for windows does not actually run the tests... so the CI will not catch that. I should change it to actually run the test, all it does right now is run build and packaging. |
|
@glubsy I updated the CI to actually run the tests on windows, which should help show these failures, we do have a minor conflict with my changes in the tox.ini now as I brought in another change there. I took a brief look at the tests that were failing and it seems to be that some of these have path separators in the regex which might explain why they are not filtering out when they should be (windows paths using different separator). |
|
Sorry for the late reply. I could indeed reproduce the issue on Windows 7 and already fixed half of it. I'll push the rest of the changes in a bit (it's 5AM here, I have to take a break). Thanks for updating the CI! |
Regexes did not match properly because the separator for Windows is '\\'
Dark green lacks contrast against black foreground font
* For some reason the table view doesn't update properly after the test string button is clicked nor when the input field is edited * The table rows only get repainted the rows properly after receiving a mouse click event * This doesn't happen on Linux
