Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Way Section Exceeding Limit Bug #203

Merged
merged 11 commits into from
Aug 27, 2018
Merged

Conversation

MikeGost
Copy link
Contributor

Description:

Couple of changes here:

  1. A test case which reproduces the issue. The issue here is in the rare corner case that way-sectioning may run out of identifiers when creating Edges from an OSM Way. This can happen in a couple of cases - when a really (emphasis on really) long way has more than 1000 intersections or (more-likely) in a case of bad data, when a really long way (with more than 1000 shape points) is duplicated and overlayed on top of itself - resulting in sectioning at each shape point - causing us to run out of identifiers. The latter case is what the test case is based off of.
  2. The fix in the section logic to address the case. The basic premise is we are going to way-section as usual up to Edge 998. As soon as we see that we only have a single identifier left, we will create an Edge 999 with the rest of the un-sectioned way and add a SyntheticInvalidWaySectionTag to that Edge to indicate it isn't properly way-sectioned. This is done in case we want to add an atlas-check to catch these types of issues in the future.
  3. Minor update to way-sectioning logging to give context for fetch and sub-atlas operations when building a DynamicAtlas. Previously, we would only log the shard we were starting from in all logging statements, which led to duplicated logging when processing multiple shards. After the change, we have more granularity and know which shard we're starting from and what other shards we're working with.

Potential Impact:

This will no longer break way-sectioning for bad data. However, it will produce un-way-sectioned Edges for any downstream consumers.

Unit Test Approach:

Added an explicit unit test that broke prior to the fix.

Test Results:

Ran a full suite of integration tests and local testing for edge case analysis - all passed.


In doubt: Contributing Guidelines

Copy link
Collaborator

@matthieun matthieun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MikeGost! This looks great. One single change I would like to see is with the text atlas for testing. It is currently 10k lines, pretty big. Is there a way to make it smaller, by pruning all the lines that are unnecessary for the test?

@MikeGost
Copy link
Contributor Author

Good point, let me see if I can trim that down to only the bare minimum!

@matthieun matthieun merged commit a0c995b into osmlab:dev Aug 27, 2018
@matthieun matthieun added this to the 5.1.9 milestone Aug 27, 2018
@MikeGost MikeGost deleted the waySectionFix branch August 31, 2018 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants