Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WaySectionProcessor performance improvements #210

Merged
merged 13 commits into from
Aug 31, 2018
Merged

Conversation

matthieun
Copy link
Collaborator

@matthieun matthieun commented Aug 31, 2018

Description:

  • When the WaySectionProcessor expands to neighboring shards, using subAtlas was being slow for a minor gain in memory improvement. This PR disables it.
  • Fixes an issue when after way sectioning, the data was not always cut along the shard borders, sometimes resulting extra load time
  • Changed the dynamic atlas policy's expansion filter to remove a spatial index query on every point on the slicedRawAtlas files.

Potential Impact:

  • Slightly larger memory usage with WaySectionProcessor
  • Faster way sectioning

Unit Test Approach:

Added a new unit test for ensuring of the cut at the border.

image

All other tests pass.

Test Results:

The proper number of edges are excluded in the new unit test.

image


In doubt: Contributing Guidelines

@@ -742,9 +749,16 @@ private boolean isAtlasNode(final WaySectionChangeSet changeSet, final Point poi
private boolean isAtlasPoint(final WaySectionChangeSet changeSet, final Point point)
{
final boolean hasExplicitOsmTags = pointHasExplicitOsmTags(point);

// We use the presence of explicit OSM tagging to determine if it's a point
if (hasExplicitOsmTags)
Copy link
Contributor

@MikeGost MikeGost Aug 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup, this makes more sense logically and likely a tiny performance boost.

@matthieun matthieun changed the title DNM: WaySectionProcessor performance improvements WaySectionProcessor performance improvements Aug 31, 2018
Copy link
Contributor

@MikeGost MikeGost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great set of changes and tests, thanks @matthieun!

@MikeGost MikeGost merged commit b1a0e34 into osmlab:dev Aug 31, 2018
@matthieun matthieun deleted the subatlas branch August 31, 2018 22:04
@matthieun matthieun added this to the 5.1.10 milestone Aug 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants