Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Floating-point precision fix in meili::Project() fixes map-match issue #2946

Merged
merged 15 commits into from
Mar 18, 2021

Conversation

ktatterso
Copy link
Contributor

@ktatterso ktatterso commented Mar 17, 2021

Issue

Map-match segfault.

Investigation revealed that a very particular geometry led to gps-point projections to land very near to nodes which created a situation that downstream code was not prepared to handle (downstream code being: meili::CandidateCollector::WithinSquaredDistance(), meili::cut_segments()).

How to reproduce

These are the two gps points: 13.2626372011, 38.159659599999999;13.262609100000001, 38.159699600000003

Resolution

To detect that a gps-point has projected exactly onto a node, code in meili::CandidateCollector::WithinSquaredDistance() looks at the percentage_along value returned from meili::helpers::Project(). If the percentage_along is exactly 0.0 or exactly 1.0 - it projected onto one side of the segment or the other.

However, if the edge is not forward, the percentage_along is inverted, e.g., 1.f - percentage_along. When percentage_along is very small (e.g., less than 1e-8), the computation of 1.f - percentage_along will be exactly 1.0. When we project the same point along the opposing edge, it lands at 0.99998, which when subtracted from 1.f yields a somewhat small but non-zero result. This combination of results is unexpected: that the projection onto the edge percentage_along is 1.0 and onto the the opp-edge is small but non-zero.

There are a number of ways to fix this, but I decided to make the fix in meili::helpers::Project(). I simply check if the percentage_along is quite small (less than 1e-7) and if so snap it to 0.f. To be consistent/symmetric, I also check if percentage_along is within 1e-7 of 1.f and if so snap it to 1.f. Consequently, the projections will also snap to the respective segment end.

Thoughts

  • This isn't float's fault. This could also happen with double. It is simply the nature of floating-point.
  • I don't like exact comparison's of floats/doubles - like the ones in meili::CandidateCollector::WithinSquaredDistance() that check for exact node projection by checking if percentage_along is exactly 0.0 or 1.0. The snapping fix I made makes me feel a bit better about these particular comparisons, but as a rule I still don't like exact comparisons of floats/doubles. Float/double's are inexact things - and geometric/mathematical code frequently leads to results that are very close to a value but rarely exact.
  • Percentages along segments aren't geometrically rigorous (e.g., 10% of a 10 m segment is quite different than 10% of a 1 m segment). I'd rather employ distance tolerances. In fact, in some ways I like my previous fix better. For those who recall, I passed in a snap_distance=0.001 to meili::helpers::Project().
  • Hey @kevinkreiser, I know I indicated this might have something to do with interpolation but I was mistaken. No interpolation is occurring. Sorry about that.

Tasklist

  • Add tests
  • Add #fixes with the issue number that this PR addresses
  • Update the docs with any new request parameters or changes to behavior described
  • Update the changelog

@kevinkreiser
Copy link
Member

i forget, is the opposing edge projected separately, ie do we call this method twice with each direction of the edge and for one direction we get a 1 where as the other direction we get a 0.00001 but not 0.0 as expected? it seems incredibly wasteful to project the same geometry twice so i dont suspect we are doing that but i didnt go check to refresh my memory. is it possible that there are more than one pair of edges at the node in question and that we get a discrepancy there about percent along as you discribed, not with an edge and is opposing edge, but with two edges that share a node but arent opposing of each other?

@ktatterso ktatterso requested a review from mookerji March 17, 2021 21:44
… gurka and into

valhalla/tests as geometry.cc. Changed meili::Project() to use doubles as input/output
and within. Consequently, from using doubles, an existing map-match unit test started
failing - on my newly added "throw std::logic_error()" in cut_segments(). A quick
examination revealed that two projection percentage_along's were within 1e-10 of one
another but in the unexpected order. I added some tolerance into cut_segments() to
address the issue. I also modified the unit test to use decode7() and length()
instead of my homegrown stuff.
@@ -141,15 +141,22 @@ void cut_segments(const std::vector<MatchResult>& match_results,
if (!curr_match.is_break_point && curr_idx != last_idx) {
continue;
}

// Allow for some fp-fuzz in this gt comparison
bool prev_gt_curr = prev_match.distance_along > curr_match.distance_along + 1e-3;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i still feel that the precondition for calling this function is that things are in order. ie we shouldnt have to account for this because upstream callers were already made to do so. that said its not a huge deal but also kind of contradicts the throw on line 158.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, not sure if you saw my commit comment. After switching helpers::Project() to double's the unit test MapMatch::interpolation() was hitting the throw::logic_error(). Here are the values:

prev_match.distance_along:  0.85524916907388349
curr_match.distance_along:  0.85524916648864746

The difference starts on the 9th decimal. Float's only have 7.2 decimal digits of precision so, in a sense, we were already rounding to 7 decimal digits by using float. Given that our projection logic employs both flat-earth approximation for projecting and haversine for distances - and then we combine those computations to calculate percentage_along - I think we have to anticipate we will see wobbles like this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah my point was only that the idea was that we would fix the wobbliness upstream from this function by fixing the distance along in the match objects before this function ever gets called. but its fine to not widen the scope of this PR (which is what i meant by "not a huge deal").

why do you say the projection logic uses flat-earth approximation (https://github.com/valhalla/valhalla/blob/master/valhalla/midgard/util.h#L593)? the distance approximator certainly does do that, but its used just for the check of whether one point is closer than another. its implementation, while inaccurate for absolute distance, should produce distances that sort properly when comparing whether one pair of points is closer or further than another pair of points.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, we use projector_t::operator() for projection, which is a flattened-earth approach. However, inside helpers::Project() we use PointLL::Distance() which uses law-of-cosines (I was mistaken when I said haversine). However, the point remains that PointLL::Distance() attempts to take into account earth curvature, whereas projector_t::operator() does not.

Point being, the percentage_along is based on a projected point (using flattened earth) as a percentage of a distance along a segment's length (using curved earth). I don't have a problem with the approach, just noting that results derived by combining these two systems will likely have slight variances against ideal expectations (e.g., a pure cartesian 2D approach) depending on where we are on the globe, segment lengths, etc. Not to mention law-of-cosines known weakness for shorter distances.

Hence the need for tolerances.

Copy link
Member

@kevinkreiser kevinkreiser Mar 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe i misunderstood what you meant by flat earth, but i had assumed you meant cartesian or strictly 2d. you'll notice we do at least scale delta X based on the latitude to account for longitunial geodesics converging at the poles. i wouldnt consider this "flat earth" or "cartesian", but i agree its not a proper projection function that would intersect a perpendicular geodesic with the geodesic between two points.

perhaps your larger point though is actually that because the projection doesnt perfectly (it cant anyway, but even less perfectly) place the projected point along the geodesicl, using law of cosines to measure distance will give different results based on how off the geodesic the projected point is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct.

@kevinkreiser
Copy link
Member

other than a few nits and formatting/linting the code this looks pretty much ready

@ktatterso ktatterso merged commit 5fda148 into master Mar 18, 2021
@ktatterso ktatterso deleted the fp-prec branch March 18, 2021 17:06
abhi2039 pushed a commit to abhi2039/valhalla that referenced this pull request Mar 30, 2021
…halla into pedestrian_crossing

* 'pedestrian_crossing' of https://github.com/abhi2039/valhalla:
  build: Bail with error if non-existant build-type specified (valhalla#2965)
  nit: Enables compiler warnings in part of mjolnir module (valhalla#2922)
  fix missing comma scripts/valhalla_build_config (valhalla#2963)
  Remove duplicate names being used across different gurka tests (valhalla#2962)
  Dont abort bidirectional a* search if only one direction is exhausted (valhalla#2936)
  penalize uturns at pencil points and short internal edges (valhalla#2944)
  Remove unused IsNextEdgeInternal function (valhalla#2958)
  restore compatibility with gcc 6.3.0, libprotobuf 3.0.0, boost v1.62.0 (valhalla#2953)
  Add ability to build Valhalla modules as STATIC libraries (valhalla#2957)
  Allow disabling Werror (valhalla#2937)
  Enhanced logic for IsTurnChannelManeuverCombinable (valhalla#2952)
  Allow config object to be passed-in to path algorithms (valhalla#2949)
  Floating-point precision fix in meili::Project() fixes map-match issue (valhalla#2946)
  Use kServiceRoad edges while reclassifying ferry connections (valhalla#2933)
  ci: Update cache key to trigger clearing it (valhalla#2947)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants