Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace avoid_* with exclude_* #3093

Merged
merged 10 commits into from
Jun 21, 2021

Conversation

RhnSharma
Copy link
Contributor

Fixes #3092
Hi @nilsnolde, how does this look?
Let me know if this needs any changes.
Thanks

danpat
danpat previously requested changes May 24, 2021
Copy link
Member

@danpat danpat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with this PR is that it's a breaking API change - folks who are using the existing avoid_ feature will need to change their code in order to update to any build of Valhalla that includes this PR.

What I think this PR should do is add exclude_ that triggers the existing code. Ideally, the avoid_ parameter would trigger proper weighted avoid behaviour, but think we could live with avoid and exclude doing the same thing for the interim until somebody gets around to implementing proper avoid behaviour. After all, avoid is currently actually excluding, and has been for some time.

I do not think this PR can be merged as-is.

@nilsnolde
Copy link
Member

Nice you want to take this on @RhnSharma ! I should've left some more notes on the issue, @danpat is right: the important point of the issue was actually to alias avoid_polygons with exclude_polygons.

It's usually ok to rename things in the config JSON, of course it's also ok to rename the variables. Where it gets tricky:

@kevinkreiser
Copy link
Member

@RhnSharma @nilsnolde yes it is ok to rename the proto objects the protobuffers dont care about the names they just change the name in the generated code, the binary format stays the same so its not a breakign change in protobuf.

As @danpat and @nilsnolde pointed out you need to allow the code in src/worker.cc to parse either avoid_polygons or exclude_polygons. this way existing code that calls this API will still work.

proto/options.proto Outdated Show resolved Hide resolved
src/worker.cc Outdated Show resolved Hide resolved
src/worker.cc Outdated Show resolved Hide resolved
src/worker.cc Outdated Show resolved Hide resolved
@RhnSharma
Copy link
Contributor Author

@kevinkreiser Thanks for suggesting the changes. I made the changes, please take a look at it and let me know if you need any changes.

@nilsnolde
Copy link
Member

they just change the name in the generated code

@kevinkreiser that's sorta what I mean though: if someone wrote code using valhalla's public PBF interface to create avoid locations/polygons, changing the names would break their code right? Not a huge deal but a bigger deal than breaking the config JSON IMHO

@danpat
Copy link
Member

danpat commented May 24, 2021

@nilsnolde changing field names in protobuf is considered "binary compatible" - as long as the IDs remained the same and the meaning of the fields is unchanged, then client code generated with old protobuf interfaces will still be able to generate/parse protobuf binary objects successfully without anything breaking.

However these changes are not source compatible - clients calling old generated interface functions will not compile against interfaces generated with these new protobuf changes. It is not well defined what Valhalla considers its compatibility interface to be, but it is not unusual to exclude source-level compatibility for these kinds of reasons.

@nilsnolde
Copy link
Member

@nilsnolde changing field names in protobuf is considered "binary compatible" - as long as the IDs remained the same and the meaning of the fields is unchanged, then client code generated with old protobuf interfaces will still be able to generate/parse protobuf binary objects successfully without anything breaking.

@danpat ah didn't know that about protobuf, thanks! that's super considerate behaivor :)

@kevinkreiser
Copy link
Member

im not sure if we've documented this but this is the general conventions we follow are:

major version change: data incompatibilty, changes to the tiles that makes them not work with previous version soft the software
minor version change: this one is very poorly defined.. sometimes we've done this when we had either larger data changes (eg adding live traffic tiles) or when we add new top level APIs (eg isochrone)
patch version change: pretty much every thing else

We've never maintained source compatibility between any versions of the software as its just impractical and would require way way way too much planning and foresight to be feasible.

@kevinkreiser
Copy link
Member

@RhnSharma there hsould be some docs changes, specifically you should rename avoid_locations and avoid_polygons here: https://github.com/valhalla/valhalla/blob/master/docs/api/turn-by-turn/api-reference.md#other-request-options to encourage people to use the new more accurate names

@kevinkreiser
Copy link
Member

also i was thinking about this a bit more and beginning to wonder if maybe it should stay avoid_ to allow us to add a penalty factor down the road. So that a person can soft avoid edges all the way up to hard avoidance. for that we'd add a new parameter to go with the avoids, the locations can easily be updated for that but the polygons might be harder. anyway renaming in this way doesnt technically preclude it but it would make the api more confusing if we ever did decide to do soft user provided avoids. just a thought to ask ourselves here

@danpat
Copy link
Member

danpat commented May 24, 2021

maybe it should stay avoid_ to allow us to add a penalty factor down the road

There are use-cases for both strict "exclude" (e.g. hazmat, and you want a Noroute response if constraints can't be met) and soft "avoid" (avoid tolls if possible, but take them if costing says it's the best), so ultimately, I think we want both (and this applies to all features of this nature, including use_highways and use_tolls).

My vote here would be to do what @nilsnolde originally suggested - add exclude_ as an alias for the same feature so no APIs break, then later on, fix the avoid_ implementation so that it's a soft avoidance, rather than the current strict one.

@kevinkreiser
Copy link
Member

kevinkreiser commented May 24, 2021

@danpat i favored the alias too (it was my suggestion) but i just wanted to throw out the question here and talk myself in a circle one more time ;)

anyway this pr is looking close, whats missing i believe is the other change to the costing options proto didnt get reflected in costing where the avoid edges are used. @RhnSharma you probably need to make some renames in the DynamicCost.h/cc

@nilsnolde
Copy link
Member

Makes me think: what do you think about a warnings string array in the response generally? Warnings could be like deprecated things, e.g. using avoid_polygons instead of exclude_polygons or maybe smth like private roads considered when destination roads have to be enabled in a second pass. Anything that's not worth throwing an error but may be worth notifying the user about. Pelias has that since a few years and I find it pretty handy for an HTTP API.

@kevinkreiser
Copy link
Member

@nilsnolde yeah ive thought about it too. its nice also for those errors in the request that we smooth over without aborting the request too. eg if you set radius on a location above the max allowed we clip it and continue with the request. it would be good to tell the user we arent perfectly adhering to their request

@RhnSharma
Copy link
Contributor Author

@kevinkreiser @nilsnolde I made the changes you suggested. Please take a look at it and let me know if you need any changes.
Thanks

Copy link
Member

@nilsnolde nilsnolde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RhnSharma you're missing a few spots here where you didn't change the name to exclude_*, e.g. look at the CI logs. Before you push next time, can you pls make sure:

  • you can build the project successfully with default cmake options
  • you can build run-gurka_avoids, e.g. make -C build run-gurka_avoids

When CI is happier (i.e. at least the build step runs through) we can review further.

bench/meili/config.json Outdated Show resolved Hide resolved
src/loki/worker.cc Outdated Show resolved Hide resolved
@nilsnolde
Copy link
Member

@RhnSharma are you still planning to finish or should I carry this over the review line? In any case, thanks for the effort already put in!

@nilsnolde
Copy link
Member

I quickly finished this. thanks a lot @RhnSharma , was just tiny stuff left

@nilsnolde nilsnolde dismissed danpat’s stale review June 21, 2021 17:44

change requests were fixed

@nilsnolde nilsnolde merged commit b01bbd0 into valhalla:master Jun 21, 2021
@nilsnolde
Copy link
Member

btw @danduk82 &@ismailsunni, this should be relevant for you too.

@ismailsunni
Copy link
Contributor

Thanks @nilsnolde

@nilsnolde nilsnolde mentioned this pull request Jun 30, 2021
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace avoid_* with exclude_*
5 participants