Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pronunciation for names and destinations #3132

Merged
merged 176 commits into from
Oct 7, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
176 commits
Select commit Hold shift + click to select a range
a1b5882
added GetPronunciationsMap() and only_pronunciations to GetTaggedName…
gknisely May 21, 2021
42b6925
updated getsign logic for pronunciations
gknisely May 21, 2021
0431b98
updated call to addedgeinfo
gknisely May 21, 2021
380342a
added logic to process pronunciation_file and pronunciations.
gknisely May 21, 2021
88287ff
updated call to .GetNamesAndTypes
gknisely May 21, 2021
e616ec2
updated call to addedgeinfo
gknisely May 21, 2021
f3db0c8
added logic to write out pronunciations. Using tagged bit on signs now.
gknisely May 21, 2021
bbdb6b5
updated call to addedgeinfo
gknisely May 21, 2021
8d8add1
added logic to process pronunciations for ways.
gknisely May 21, 2021
56b8fc8
added logic to parse pronunciations. Nodes data saved to existing no…
gknisely May 21, 2021
994a6e4
updated call to addedgeinfo
gknisely May 21, 2021
9e2e8c6
updated call to addedgeinfo
gknisely May 21, 2021
f8d4894
just temp printing out pronunciations
gknisely May 21, 2021
c5c78b8
updated call to addedgeinfo
gknisely May 21, 2021
498f2dc
added new pronunciation file
gknisely May 21, 2021
575a8c7
added new kVerbal type.
gknisely May 21, 2021
af3796c
tagged bit is now in use.
gknisely May 21, 2021
bf4fd4b
added is_tagged_, has_phoneme_, phoneme_start_index_, and phoneme_cou…
gknisely May 21, 2021
e7d4109
updated comment
gknisely May 21, 2021
618161f
added pronunciations to AddEdgeInfo
gknisely May 21, 2021
22a6ac6
added pronunciation indexes.
gknisely May 21, 2021
cf6f5e7
added pronunciation_file
gknisely May 21, 2021
2191bf4
init copy
gknisely May 21, 2021
d357d62
init copy
gknisely May 21, 2021
acac388
lint
gknisely May 21, 2021
13cbedc
Merge remote-tracking branch 'origin/master' into phonemes
gknisely May 21, 2021
2f333af
Added verbal type and value to street name and sign element
dgearhart May 21, 2021
147ef5e
clean up temp files
gknisely Jun 11, 2021
ca55608
Merge remote-tracking branch 'origin' into phonemes
gknisely Jun 15, 2021
987c671
added Jeita.
gknisely Jun 28, 2021
dffa0ca
updated GetPronunciationsMap(). Data is now separated by a #. Updat…
gknisely Jun 28, 2021
8d439b1
updated to kJeita.
gknisely Jun 28, 2021
e423c1a
getsigns should of returned a unordered_multimap. delimeter is now #.
gknisely Jun 28, 2021
26b60e3
delimeter is now #.
gknisely Jun 28, 2021
27469c2
added name and ref set/get for jeita.
gknisely Jun 28, 2021
771d768
added logic for jeita.
gknisely Jun 28, 2021
f3035f7
added jeita indexes and accessor functions.
gknisely Jun 28, 2021
df1430b
added logic to set the jeita information.
gknisely Jun 28, 2021
34e1343
added types
gknisely Jun 28, 2021
e9e830c
added logic to parse jeita information.
gknisely Jun 28, 2021
4f046e2
clean up
gknisely Jun 28, 2021
adbc513
added logic to reference tagged names from the pronunciation map
gknisely Jun 28, 2021
dc400c4
updated to use new function signatures.
gknisely Jun 28, 2021
8d294c5
allow name nodes for netherlands only for testing
gknisely Jun 28, 2021
9a39136
updated to use new function signatures.
gknisely Jun 28, 2021
d3afe07
new tests.
gknisely Jun 28, 2021
a31016c
removed verbal info from sign
dgearhart Jun 30, 2021
6836c38
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Jun 30, 2021
b005172
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Jul 2, 2021
dbd6fe6
Added new Pronunciation message
dgearhart Jul 2, 2021
6f56132
refactored naming to match
dgearhart Jul 2, 2021
4d9f991
format fix
dgearhart Jul 3, 2021
2ca8c3e
refactrored for name consistency
dgearhart Jul 6, 2021
4975582
name refactor
dgearhart Jul 6, 2021
0196360
rename GetPronunciationsMulitMap
dgearhart Jul 6, 2021
c8b9d77
missed one
dgearhart Jul 6, 2021
35af424
added GetPronunciationsMap
gknisely Jul 6, 2021
0d5e160
updated to use edgeinfo.GetPronunciationsMap
gknisely Jul 6, 2021
69e0d59
reordered.
gknisely Jul 6, 2021
3eb2c67
refactor
dgearhart Jul 7, 2021
372ba66
Assign the street names pronunciations if they exist
dgearhart Jul 7, 2021
b8ee5fc
Updated StreetNamesToString and StreetNamesToParameterString
dgearhart Jul 7, 2021
a0430a3
updated test routes
dgearhart Jul 8, 2021
0ae7635
Added the GraphTile::GetSigns method using unordered_map for the inde…
dgearhart Jul 8, 2021
2711fb1
Changed the name of the internal GetSigns method to ProcessSigns
dgearhart Jul 8, 2021
6e1918d
update comment
dgearhart Jul 14, 2021
1de7412
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Jul 14, 2021
268d220
fix conflict
dgearhart Jul 14, 2021
f181beb
Added the PopulateSignElement method which will populate sign pronunc…
dgearhart Jul 15, 2021
b151d87
Added debug output for sign_element pronunciation
dgearhart Jul 16, 2021
ad1c6ce
GetPronunciationsMultiMap() no longer needed. emplace does not allow…
gknisely Jul 20, 2021
5780bab
added logic for signs_on_node check.
gknisely Jul 20, 2021
33a5173
fixed bug where destination_ref_to were not getting added with phonemes
gknisely Jul 20, 2021
9e7bb11
updated and added more tests.
gknisely Jul 20, 2021
d6335b1
updated comment
gknisely Jul 20, 2021
7bf7b4c
route num = true means phoneme is for the node.
gknisely Jul 20, 2021
fbe6f2e
fixed xsampa bug.
gknisely Jul 20, 2021
12bb7ff
lint
gknisely Jul 21, 2021
336d699
changed to nt-sampa
gknisely Jul 22, 2021
f29b170
lint
gknisely Jul 22, 2021
82bac45
Added pronunciation class
dgearhart Aug 2, 2021
0c9e24f
added maneuver street name debug output
dgearhart Aug 3, 2021
be2ae25
Assigned pronunciation to the signs
dgearhart Aug 4, 2021
328e2e2
update for street names debug
dgearhart Aug 4, 2021
58a2566
Assigned pronunciation to street names in the maneuvers builder
dgearhart Aug 4, 2021
053bde4
clan-tidy updates.
gknisely Aug 9, 2021
2f00de3
clang tidy updates.
gknisely Aug 9, 2021
643f04b
clang tidy updates.
gknisely Aug 9, 2021
e75195e
Updated the DirectionsBuilder to assign the street name and sign pron…
dgearhart Aug 10, 2021
8912922
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Aug 11, 2021
0c73673
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Aug 12, 2021
b067db4
Added another RAD test
dgearhart Aug 12, 2021
8ea351b
Merge remote-tracking branch 'origin/master' into phonemes_update
gknisely Aug 26, 2021
391539c
lint
gknisely Aug 26, 2021
dce938b
Merge remote-tracking branch 'origin' into phonemes_update
gknisely Aug 26, 2021
ed8780a
Update graphtilebuilder.cc
gknisely Aug 26, 2021
e5a58e1
Update graphtilebuilder.cc
gknisely Aug 26, 2021
fb90085
Merge remote-tracking branch 'origin/phonemes_update' into phonemes
gknisely Aug 26, 2021
d6ad7ee
jeita --> x-jeita and katakana --> x-katakana
gknisely Sep 2, 2021
f12ff7c
Added MarkupFormatter class
dgearhart Sep 2, 2021
65990f8
fixed format
dgearhart Sep 2, 2021
23aaefe
lint
gknisely Sep 2, 2021
3d4602e
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Sep 2, 2021
0ac1996
added config options for markup_formatter
gknisely Sep 3, 2021
2dc1997
Added markup support for signs
dgearhart Sep 8, 2021
3630395
Verify toward sign pronunciation instructions
dgearhart Sep 8, 2021
d75e635
Test exit number, exit onto street, and street name pronunciation ins…
dgearhart Sep 8, 2021
f3c7a33
Test ref and destination:ref pronunciation
dgearhart Sep 8, 2021
654b4cf
Added tests for destination:street:to and destination:ref:to
dgearhart Sep 8, 2021
1cdfb0f
added the CheckGuideSigns test
dgearhart Sep 8, 2021
05451bc
Added the CheckJunctionName test
dgearhart Sep 8, 2021
6c05aba
fixed typo
dgearhart Sep 10, 2021
574cdd6
cleanup refactor
dgearhart Sep 16, 2021
f74745e
Updated the quotes format and gurka tests
dgearhart Sep 16, 2021
2a5a053
refactor for multicue
dgearhart Sep 20, 2021
0287fd1
refactor markup formatter
dgearhart Sep 21, 2021
91b3383
refactor FormatPhonemeElement
dgearhart Sep 21, 2021
3d7576c
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Sep 21, 2021
217cfd0
Added 3132 to changelog
dgearhart Sep 21, 2021
1d1e702
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Sep 21, 2021
7590fb0
update submodules
dgearhart Sep 22, 2021
aa6534b
example for how to handle nuls in strings by computing the width of e…
kevinkreiser Sep 24, 2021
3d7010e
eagle eye duane
kevinkreiser Sep 24, 2021
c712afd
so much simpler
kevinkreiser Sep 24, 2021
2270541
debugging wrong data in hierarchybuilder
kevinkreiser Sep 24, 2021
10577a3
added logic to get the pronunciations
gknisely Sep 25, 2021
747a37e
updated init of the struct
gknisely Sep 25, 2021
b610dc7
updated output.
gknisely Sep 25, 2021
b00c7c4
added pronunciation to the osrm serializer
dgearhart Sep 27, 2021
ed358e6
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
dgearhart Sep 27, 2021
1335538
fixed issue where we were adding extra null chars
gknisely Sep 27, 2021
494e57c
added kNone.
gknisely Sep 27, 2021
4176cc4
moved language to the top as kNone is set to 1.
gknisely Sep 27, 2021
bdf8fe6
added kNone for PronunciationAlphabet Language
gknisely Sep 27, 2021
1aedf48
format
gknisely Sep 27, 2021
75d4574
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Sep 27, 2021
eaddc9f
fixed debug output
dgearhart Sep 27, 2021
52d0631
removed kNone from proto pronunciation alphabet
dgearhart Sep 27, 2021
c0c1a3e
only read to end of the pronunciation using the header.length_
gknisely Sep 28, 2021
df449ae
refactored to use new structure for phonemes
gknisely Sep 28, 2021
fa928e6
save data out in new phoneme structure.
gknisely Sep 28, 2021
e2737cc
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Sep 28, 2021
5f940f1
updated to support new structure
gknisely Sep 28, 2021
9d6354f
removed name.size() > 1 logic and clean up
gknisely Sep 28, 2021
5022e71
added comment
gknisely Sep 28, 2021
eb94aac
reduced some code. moved logic to AddPronunciation
gknisely Sep 28, 2021
a760014
cleanup
gknisely Sep 28, 2021
6638061
added AddPronunciation and renamed BuildPronunciations
gknisely Sep 28, 2021
979b34b
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Sep 28, 2021
0f3afa5
lint
gknisely Sep 28, 2021
651ff2c
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Sep 28, 2021
2a8d429
lint
gknisely Sep 28, 2021
d92a493
should start at IPA
gknisely Sep 29, 2021
a14a076
refactor PronunciationAlphabetToString
dgearhart Sep 29, 2021
484d5b1
Refactor Pronunciation_Alphabet_Name
dgearhart Sep 30, 2021
cd3654f
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Sep 30, 2021
b849ec4
account for empty phonemes
gknisely Sep 30, 2021
5c17e1b
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Sep 30, 2021
7d7f459
cp paste fail
gknisely Oct 1, 2021
e5f3c30
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Oct 1, 2021
c860d7b
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
dgearhart Oct 1, 2021
9218b0e
counts and begin indexes were off for pronunciations for signs when t…
gknisely Oct 4, 2021
9233d7a
added count
gknisely Oct 4, 2021
60b29f0
added tests for missing pronunciations for signs.
gknisely Oct 4, 2021
caaee19
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Oct 4, 2021
d750a7e
removed unused code.
gknisely Oct 4, 2021
80ad5e7
adding more name check for blank names
gknisely Oct 4, 2021
1701e6b
added TODO and addressed index issue.
gknisely Oct 6, 2021
19465d9
removed debug statements.
gknisely Oct 6, 2021
c6e27a8
added blank name testing.
gknisely Oct 6, 2021
c49180c
updated to include OSM pronunciation tags
gknisely Oct 6, 2021
4d640bd
Merge branch 'master' into phonemes
gknisely Oct 6, 2021
7b92a2d
Merge branch 'master' of https://github.com/valhalla/valhalla into ph…
dgearhart Oct 7, 2021
d2b06e2
updated.
gknisely Oct 7, 2021
f108c60
cleanup
gknisely Oct 7, 2021
2fea8e6
Merge branch 'phonemes' of https://github.com/valhalla/valhalla into …
gknisely Oct 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Changed the name of the internal GetSigns method to ProcessSigns
  • Loading branch information
dgearhart committed Jul 8, 2021
commit 2711fb18178e3a5779703d79e396aef250b11e6a
4 changes: 2 additions & 2 deletions src/baldr/graphtile.cc
Original file line number Diff line number Diff line change
Expand Up @@ -707,9 +707,9 @@ std::string GraphTile::GetName(const uint32_t textlist_offset) const {
}
}

// Convenience method to get the signs for an edge given the
// Convenience method to process the signs for an edge given the
// directed edge index.
std::vector<SignInfo> GraphTile::GetSigns(const uint32_t idx, bool signs_on_node) const {
std::vector<SignInfo> GraphTile::ProcessSigns(const uint32_t idx, bool signs_on_node) const {
uint32_t count = header_->signcount();
std::vector<SignInfo> signs;
if (count == 0) {
Expand Down
4 changes: 2 additions & 2 deletions src/mjolnir/graphfilter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ void FilterTiles(GraphReader& reader,

// Get signs from the base directed edge
if (directededge->sign()) {
std::vector<SignInfo> signs = tile->GetSigns(edgeid.id());
std::vector<SignInfo> signs = tile->ProcessSigns(edgeid.id());
if (signs.size() == 0) {
LOG_ERROR("Base edge should have signs, but none found");
}
Expand Down Expand Up @@ -174,7 +174,7 @@ void FilterTiles(GraphReader& reader,

// Get named signs from the base node
if (nodeinfo->named_intersection()) {
std::vector<SignInfo> signs = tile->GetSigns(nodeid.id(), true);
std::vector<SignInfo> signs = tile->ProcessSigns(nodeid.id(), true);
if (signs.size() == 0) {
LOG_ERROR("Base node should have signs, but none found");
}
Expand Down
4 changes: 2 additions & 2 deletions src/mjolnir/graphvalidator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ void validate(

// Validate signs
if (ni->named_intersection()) {
if (tile->GetSigns(i, true).size() == 0) {
if (tile->ProcessSigns(i, true).size() == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: but i liked the previous name way better

LOG_ERROR("Node marked as having signs but none found");
}
}
Expand All @@ -326,7 +326,7 @@ void validate(

// Validate signs
if (de->sign()) {
if (tile->GetSigns(idx).size() == 0) {
if (tile->ProcessSigns(idx).size() == 0) {
LOG_ERROR("Directed edge marked as having signs but none found");
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/mjolnir/hierarchybuilder.cc
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ void FormTilesInNewLevel(GraphReader& reader,

// Get signs from the base directed edge
if (directededge->sign()) {
std::vector<SignInfo> signs = tile->GetSigns(base_edge_id.id());
std::vector<SignInfo> signs = tile->ProcessSigns(base_edge_id.id());
if (signs.size() == 0) {
LOG_ERROR("Base edge should have signs, but none found");
}
Expand Down Expand Up @@ -358,7 +358,7 @@ void FormTilesInNewLevel(GraphReader& reader,

// Get named signs from the base node
if (baseni.named_intersection()) {
std::vector<SignInfo> signs = tile->GetSigns(base_node.id(), true);
std::vector<SignInfo> signs = tile->ProcessSigns(base_node.id(), true);
if (signs.size() == 0) {
LOG_ERROR("Base node should have signs, but none found");
}
Expand Down
4 changes: 2 additions & 2 deletions src/mjolnir/shortcutbuilder.cc
Original file line number Diff line number Diff line change
Expand Up @@ -637,7 +637,7 @@ uint32_t FormShortcuts(GraphReader& reader, const TileLevel& level) {

// Get signs from the base directed edge
if (directededge->sign()) {
std::vector<SignInfo> signs = tile->GetSigns(edgeid.id());
std::vector<SignInfo> signs = tile->ProcessSigns(edgeid.id());
if (signs.size() == 0) {
LOG_ERROR("Base edge should have signs, but none found");
}
Expand Down Expand Up @@ -706,7 +706,7 @@ uint32_t FormShortcuts(GraphReader& reader, const TileLevel& level) {
// Get named signs from the base node
if (nodeinfo.named_intersection()) {

std::vector<SignInfo> signs = tile->GetSigns(n, true);
std::vector<SignInfo> signs = tile->ProcessSigns(n, true);
if (signs.size() == 0) {
LOG_ERROR("Base node should have signs, but none found");
}
Expand Down
9 changes: 5 additions & 4 deletions src/thor/triplegbuilder.cc
Original file line number Diff line number Diff line change
Expand Up @@ -662,7 +662,8 @@ void AddTripIntersectingEdge(const AttributesController& controller,
GraphId beginnode = graphreader.GetBeginNodeId(intersecting_de, graphtile);
valhalla::baldr::graph_tile_ptr t2 = graphreader.GetGraphTile(beginnode);
size_t edge_idx = intersecting_de - t2->directededge(0);
std::vector<SignInfo> edge_signs = t2->GetSigns(edge_idx);
std::unordered_map<uint32_t, std::pair<uint8_t, std::string>> pronunciations;
std::vector<SignInfo> edge_signs = t2->GetSigns(edge_idx, pronunciations);
if (!edge_signs.empty()) {
valhalla::TripSign* sign = intersecting_edge->mutable_sign();
AddSignInfo(controller, edge_signs, sign);
Expand Down Expand Up @@ -865,8 +866,7 @@ TripLeg_Edge* AddTripEdge(const AttributesController& controller,
// Set the signs (if the directed edge has sign information) and if requested
if (directededge->sign()) {
// Add the edge signs
std::unordered_multimap<uint32_t, std::pair<uint8_t, std::string>> pronunciations;

std::unordered_map<uint32_t, std::pair<uint8_t, std::string>> pronunciations;
std::vector<SignInfo> edge_signs = graphtile->GetSigns(idx, pronunciations);

// TODO - debug output for testing - remove in the future
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gknisely this should also be removed now

Expand All @@ -886,7 +886,8 @@ TripLeg_Edge* AddTripEdge(const AttributesController& controller,
// Process the named junctions at nodes
if (has_junction_name && start_tile) {
// Add the node signs
std::vector<SignInfo> node_signs = start_tile->GetSigns(start_node_idx, true);
std::unordered_map<uint32_t, std::pair<uint8_t, std::string>> pronunciations;
std::vector<SignInfo> node_signs = start_tile->GetSigns(start_node_idx, pronunciations, true);
if (!node_signs.empty()) {
valhalla::TripSign* trip_sign = trip_edge->mutable_sign();
for (const auto& sign : node_signs) {
Expand Down
2 changes: 1 addition & 1 deletion test/gurka/test_build_admin.cc
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ TEST(AdminTest, TestBuildAdminFromPBF) {
const NodeInfo* X_node = X_tile->node(X_node_id);
EXPECT_EQ(X_node->drive_on_right(), false);
EXPECT_EQ(X_node->named_intersection(), true);
EXPECT_EQ(X_tile->GetSigns(X_node_id.id(), true).at(0).text(), "namae wa nan desu ka");
EXPECT_EQ(X_tile->ProcessSigns(X_node_id.id(), true).at(0).text(), "namae wa nan desu ka");
AdminInfo X_admin = X_tile->admininfo(X_node->admin_index());
EXPECT_EQ(X_admin.state_text(), "Hyogo");
EXPECT_EQ(X_admin.country_text(), "Japan");
Expand Down
2 changes: 1 addition & 1 deletion valhalla/baldr/graphtile.h
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,7 @@ class GraphTile {
* intersection names.
* @return Returns a list (vector) of signs.
*/
std::vector<SignInfo> GetSigns(const uint32_t idx, bool signs_on_node = false) const;
std::vector<SignInfo> ProcessSigns(const uint32_t idx, bool signs_on_node = false) const;

/**
* TODO: Remove after updating the tests
Expand Down