Community lexicons #62
Replies: 71 comments 55 replies
-
|
https://github.com/karashiiro/TextToTalk/wiki/Community-lexicons |
Beta Was this translation helpful? Give feedback.
-
|
Main Character names British Voice Lexicon.zip Works only with British voices. Uses aliases because pronemes aren't currently working with the standard voice setting. |
Beta Was this translation helpful? Give feedback.
-
|
FFXIVCharacters&LocationsEN.zip I've done all the characters I've noticed the most when going through MSQ. As well as Mispronounced Location names. These use pronemes so pronunciation should be more consistent though different regions of English. |
Beta Was this translation helpful? Give feedback.
-
|
Is there an app or something easy to make these lexicons? |
Beta Was this translation helpful? Give feedback.
-
|
I am not sure if this should be a separate issue or not, but I tried using the FFXIVCharacters&LocationsEN.zip lexicon (both through Amazon Polly and directly uploaded to the plugin) and it said "Maximum lexicons size has been exceeded". |
Beta Was this translation helpful? Give feedback.
-
https://docs.aws.amazon.com/polly/latest/dg/gs-put-lexicon.html I don't know of any apps for this, but this article has some lexicons used in its examples that might explain the concept.
I've never heard of this happening, but I assume that means Amazon Polly has some sort of size limit on lexicons. You can try splitting the lexicon in half, maybe? Pulling out half of the lexemes and putting them into a new lexicon file and uploading the resulting two smaller ones. |
Beta Was this translation helpful? Give feedback.
-
Looks like you nailed it, as per Amazon Polly’s site, “Each lexicon can be up to 4,000 characters in size. ” |
Beta Was this translation helpful? Give feedback.
-
Wow 4000 characters is quite a small limit, in the future I'll have to make split files for Amazon Polly. For now though, I removed a lexeme that can't be used right now with the way the plugin currently works, and the character count is now 3999! If there is any issue let me know! |
Beta Was this translation helpful? Give feedback.
-
|
Oh damn, thank you so much! Hah, karashiiro saw the future and knew it needed to get under 4000 characters LOL. But seriously, thank you so much, I'll try it in a few, hopefully, after a few morning jobs. Also, do you recommend uploading it to Amazon directly or just uploading it to the addon? |
Beta Was this translation helpful? Give feedback.
-
|
You'll need to upload it through the TextToTalk plugin. |
Beta Was this translation helpful? Give feedback.
-
|
FFXIVCharacters&LocationsEN.zip
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
That zip looks empty 👀 |
Beta Was this translation helpful? Give feedback.
-
|
FFXIVCharacters&Locations.zip |
Beta Was this translation helpful? Give feedback.
-
|
--------Update: Added plurals to the new additions-------- Fixed pronunciation for the word Aetheryte and one of the expansions location names. Also I've split the polly zip version into two lexicon files to respect the 4000 character limit. |
Beta Was this translation helpful? Give feedback.
-
|
I've been sitting on an idea for the lexicon repository for a while now, and I think it would be pretty neat to have a main repository for all the non-"controversial" pronunciations for lack of a better word. And then, on the side, for people like me who prefer a more western style pronunciation for some character names and who use UK voices with British accents, I could create an alternative lexicon for words so that R's sound like R's and L's sound like L's in the context of a British accent, which many voiced characters in FFXIV actually do have. For example, the character name Yugiri has two alternative ways of pronouncing it, depending on which character in FFXIV is speaking the name. |
Beta Was this translation helpful? Give feedback.
-
|
Should we consider removing "Eld" and "eld" from the lexicon? It seems to be messing with the world "battlefield", pronouncing it as "battle-fee eld" instead of "battle-feeld". Is that a bug that can be fixed in C# code or is that just how lexicons work and that we should just remove "eld" from the lexicon? Below is a screenshot of the Dalamud Log for this text to speech:
|
Beta Was this translation helpful? Give feedback.
-
|
Tagging #104 as a related issue. If, however, we force all graphemes to match only when surrounded by spaces or punctuation, that could also break the graphemes' that make |
Beta Was this translation helpful? Give feedback.
-
|
Yeah I'd just put a space or punctuation in the lexicon file on a case by case basis. Until we sort out a better solution that is. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I'm mentioned the same issue in that community post I made Here . Just was trying to figure out if it's possible to regex those shorter words or something. The work around for eld would be putting a space before it, so prevent it from interrupting mid word. |
Beta Was this translation helpful? Give feedback.
-
|
I just finished adding Uberduck support as of v1.15, no need to debug lexicons on it though (since it doesn't seem to support SSML). |
Beta Was this translation helpful? Give feedback.
-
|
Is there any reason why we shouldn't just append new lexemes to the bottom of the main lexicon? I grew tired of manually finding the right category and alphabetical place to place each lexeme when I can just ctrl+f find to check if a word has been added yet. I'm inclined to just add new lexemes at the bottom of the lexicon and then push the commit, but maybe there's a reason I haven't thought of for the way you have it organized @johnysandels. I've just pushed a batch of new lexemes that I've added to my local copy as I've been playing through Shadowbringers since it's been working fine for me for a couple weeks now. |
Beta Was this translation helpful? Give feedback.
-
|
No real point to make it organized tbh. I just like it! If we're just doing them as we think of them, I'll remove the headings on the sections. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, do you think there is a way to fix the mammets's speech with a Lexicon? At least using Polly, it's unintelligible. Example:
|
Beta Was this translation helpful? Give feedback.
-
|
Ahahah! I could imagine what that sounds like. It would be possible to fix, but it would be at a plugin level. and it would cause issues with lexicons afterwards depending on how it's implemented. Capitalization is important to how the lexicon is written, so if it was all forced into lowercase we would have to rewrite some of our lexicons. And we would run into issues as well. Only option I could think of, is to have it change the text to lower case conditionally. Maybe it could parse the string and only set it all to lowercase if the text is =>40% capitals |
Beta Was this translation helpful? Give feedback.
-
|
Would it also cause issues if the graphemes were made to be lowercase, too? I don't know if there are any lexemes that are different depending on their capitalization. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah the only real issue would be short words at the beginning of the sentence, where the word could be part of other word. Like "Oh". There are many words that have oh in them and in the beginning of the text the only way to discern it from other words that have oh in them is the capital. There was an idea about using regex to fix that issue, but it opens another can of worms. |
Beta Was this translation helpful? Give feedback.
-
|
I just saw this, and this gave me and idea, but I don't know how feasible it is yet. What if literally all text was converted to lowercase before feeding it to text-to-speech? That might solve both this problem and not make us have to keep various versions of every non-proper noun word in the lexicons. |
Beta Was this translation helpful? Give feedback.
-
|
That would run into the issue @johnysandels just mentioned, unless we changed SSML rendering significantly (which we could do, it'd just be a lot of work). |
Beta Was this translation helpful? Give feedback.
-
|
Small notice: I added Azure support in v1.18, and it has lexicons enabled. Not sure if it'll create any new compatibility issues, but from my minimal testing it looks like it works fine with Polly lexicons. |
Beta Was this translation helpful? Give feedback.
-
|
Hey I have installed the plugin and unfortunately every time I enter the API Credential key from Amazon Polly it disappears, I thought I'd inform you on this bug, it maybe due to the recent patch. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
I'm aware, I just haven't time to deal with this, yet. Going to fix it tonight when I'm back at my hotel. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, honestly your Text To Talk plugin is amazing, it's helped me so much when was playing Final Fantasy. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Was doing some scholar/summoner quests and noticed these needed fixing: Also I got tired of hearing HM for all those "Hm..." so I had to extend the current ones. Can anyone else test these? I don't' know if they'll interfere with other common words. A weird one I came across (the word was paladin-ing): Some tweaks I use and propose to add:
|
Beta Was this translation helpful? Give feedback.
-
|
Added more: For some reason I can't get it to pronounce the "E" here: Does anyone have any ideas on how to get it to work? I tried separating it into different words but is still doesn't work for some reason. I tried System, Amazon, and Azure. |
Beta Was this translation helpful? Give feedback.
-
|
Seems like this issue is caused by having "Attempt to remove stutter from NPC dialogue on. I'm adding your entries, But i need to comb through it to fix an issue I'm coming across before posting. Also thank you for the updated city names! I made those when I first started on the list, so I wasn't as good back then 👍 |
Beta Was this translation helpful? Give feedback.
-
|
I see. It might work if I made an alias for it maybe? I'll test it. EDIT: Hm, didn't seem to work. I'm open for ideas.
Oh no problem, as a matter of fact if anyone has an improvement over my stuff I welcome it. Not saying they're all perfect. I'm still tweaking them as I go along. |
Beta Was this translation helpful? Give feedback.
-
|
Just realized PvP might work easier as
|
Beta Was this translation helpful? Give feedback.
-
|
I've updated the system lexicons but I will work on the polly one later today. Also unfortunately the stutter fix removes the E entirely, so it wont work for E-Sumi-Yan, it will just have to be turned off. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, it seems like it. I mean, at least the pronunciation is correct, so there's not much more I can do. |
Beta Was this translation helpful? Give feedback.
-
|
Good Morning! I see that this thread has been quiet for most of this current year (2024), but I am gonna try asking my question here to see if any of you have been successful with making a lexicon for something like this: I have repeated issues with punctuation being read aloud whenever it is followed by a music note symbol: ( ♪ ). This happens whenever a character is singing, such as The Wandering Minstrel, Jehantel (from the Bard questline), and even Tataru when she is humming a tune. So what I have gotten is:
I think these are the four most common occurrences in which the music note causes the punctuation or other symbol to be read aloud, but there may be more. What I cannot figure out, yet, is how to silence those punctuations from being verbalized when directly followed by a music note symbol without any spaces, as generally the ♪ always comes after punctuation without any spaces. It can be quite annoying, especially when the song or ditty has multiple lines to it. |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, that's an issue with how the text is parsed. I've created an issue for this here, #197 . |
Beta Was this translation helpful? Give feedback.
-
|
Hey, just a heads up. I'm back in FFXIV again. And... I just pushed a fix for my bad commit that I just realized that I made that completely messed up the Names and locations Amazon Poly lexicon about 2 years ago. I guess I'm the only one who uses that if no one noticed or fixed it since then. But it's fixed now. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @johnysandels, I pushed a commit (d27de5e) for the "Polly" lexicon to merge a bunch of your contributions in with mine by using the built-in git diff tool in VS Code after overwriting the Polly file with the System file and then manually adjusting it from there bringing over the additions I have made back into the Polly file. The above is relevant for this next part. I also wrote a Powershell and Windows Batch script set that will automatically sort our each element by its first value. It's only half working right now, so it needs about 10 seconds of manual editing afterward. That's why I'm not committing my scripts as utilities right now until it just works. My sorted lexicon commit is here (f87112a). I used the So my question for you is, do you mind if we standardize on a 4-space indent for the XML formatting, if not for readability, then at least to make it easier to keep contributions between the two major lexicons in sync a bit easier? If so, I'll do a sort and format for the System lexicon too. Then, periodically, when I sync lexicons together, I can just sort, format, and then merge them, in that order so I don't accidentally merge in duplicates. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I'm into the 4 space formatting. Although I do personally prefer having sub-categories for my lexicons, all alphabetically listed. if you just wanted to make everything alpabetical and delete the categories that is also fine. I only know basic coding and coding language so if you want to overhaul it, I really don't mind! |
Beta Was this translation helpful? Give feedback.
-
|
Ok, I'm done making changes. I've got the System lexicon all sorted and formatted, and then I merged a few additions into it without merging in any conflicting differences since the system voices might not work as well with the phonemes I use with Amazon Polly. Let me know if you have any trouble if merging any over your local changes with the new format. You can send your local file to me, and I'll figure out how to merge them. |
Beta Was this translation helpful? Give feedback.
-
|
Oh, I've also pushed my completed scripts for auto-sorting to the base lexicon directory. It has a README file for instructions. No coding necessary. I've included an option for drag and drop, and then double-clicking a file to run it in Windows Explorer (your file browser). I ended up completely rewriting my first set of scripts in Python mainly because I know Python. I don't know PowerShell as well. The only downside is that it requires you to install Python first. |
Beta Was this translation helpful? Give feedback.
-
|
@johnysandels Which System voices do you test your phonemes with? I'm updating the "Community Lexicon" wiki page with recommendations for voices since, at least with Amazon Polly, some voices are objectively worse than others when handling custom phonemes. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, sorry for the delay, I use Microsoft George and Microsoft Catherine |
Beta Was this translation helpful? Give feedback.
-
(Continued from #43)
If anyone has lexicons they're willing to share, I'd appreciate it if they could drop a link so I can provide them to anyone who wants them and doesn't know how to make them themselves 😄 Alternatively, feel free to post them in the #preset-sharing channel in the goat place Discord, and we can relink them.
A list of community lexicons is maintained on the wiki, and we have a checked-in collection in the repo.
Beta Was this translation helpful? Give feedback.
All reactions