Meaningless sounds get produced randomly if there are continuous spaces(newlines, tabs) regardless of prompts or input content #765
Open
Description
Self Checks
- This template is only for bug reports. For questions, please visit Discussions.
- I have thoroughly reviewed the project documentation (installation, training, inference) but couldn't find information to solve my problem. English 中文 日本語 Portuguese (Brazil)
- I have searched for existing issues, including closed ones. Search issues
- I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
- Please do not modify this template and fill in all required fields.
Cloud or Self Hosted
Self Hosted (Source)
Environment Details
Ubuntu 22.04.3 LTS (Google Colab), Python 3.10, torch==2.4.1
Steps to Reproduce
Run tools/llama/generate.py with any Chinese text that has continuous whitespaces, such as indentations, and with any prompts.
For example:
要转换的文本。
要转换的文本？
要转换的文本。
“要转换的文本！”
要转换的文本。
Note that if the indents at the beginning each line get removed, the random meaningless sounds also disappear. Also the meaningless sounds don't always appear on every line. It happens periodically and has no specific patterns.
Right now my workaround is to add an additional clean-up step here inside clean_text
fish-speech/fish_speech/text/clean.py
Line 24 in f54c50f
to replace any continuous whitespace characters with the first occurrence of the matched characters (including newlines or tabs).
✔️ Expected Behavior
No random meaningless sounds.
❌ Actual Behavior
As described above.