Description:

This PR adds in a new Iterable subclass, FilteredITerable. Its goal is to allow for easy filtering of objects without a significant memory footprint, including on repeat iterations. To do this, it keeps a set of Identifiers for the objects in the Iterable. The identifier is defined as whatever the Identifier function returns for an object in the Iterable-- examples of this include the object itself (simple, but likely a heavy memory footprint), a data member (such as an AtlasEntity's identifier), or the result of a hash function. The iterator will skip over any element for which the Identifier function's return value is in the filtered set. Elements to be skipped on future iterations can be added to the FilteredIterator using the addToFilteredSet() method.

Example use cases are available in the unit tests for the class.

Potential Impact:

Nothing at the moment, given that it's not instantiated anywhere

Unit Test Approach:

There are two unit tests. The first makes a new Iterable from the numbers 0-9, then filters them in various ways to ensure filtering logic is correct. This check includes corner cases such as filtering twice, filtering first and last elements, and filtering all elements.

The second test uses a set of URLs in order to test a more complex identifier function, as well as integration with the StreamIterable class.

Test Results:

2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:45 - [https://github.com, http://github.com, https://test.com, http://test.com]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:52 - [https://github.com, https://test.com]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:60 - []
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:82 - [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:91 - [0, 1, 2, 3, 4, 7, 8, 9]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:99 - [0, 1, 2, 3, 4, 7, 8, 9]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:107 - [0, 1, 2, 3, 4, 7, 8, 9]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:120 - [8]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:127 - []
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:136 - [1, 2, 3, 4, 5, 6, 7, 8, 9]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:145 - [0, 1, 2, 3, 4, 5, 6, 7, 8]
2018-10-01 11:10:43 INFO  [main] FilteredIterableTest:155 - []

Describe other (non-unit) test results here.

In doubt: Contributing Guidelines