Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permission Denied fairseq \ examples #3

Open
sriprad opened this issue Jan 28, 2021 · 19 comments
Open

Permission Denied fairseq \ examples #3

sriprad opened this issue Jan 28, 2021 · 19 comments

Comments

@sriprad
Copy link

sriprad commented Jan 28, 2021

Hi,
Thank you for the great library. Very useful. I tried to install in my machine and it landed up in error. Can you please let me know if i am missing anything,?
ERROR: Command errored out with exit status 1:
command: 'C:\Program Files\Anaconda\python.exe' 'C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py' get_requires_for_build_wheel 'C:\Users\Public\Documents\Wondershare\CreatorTemp\tmp90q37bss'
cwd: C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-zxfcubdl\fairseq
Complete output (31 lines):
Traceback (most recent call last):
File "setup.py", line 214, in
do_setup(package_data)
File "setup.py", line 136, in do_setup
setup(
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools_init_.py", line 152, in setup
install_setup_requires(attrs)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools_init
.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 60, in fetch_build_eggs
raise SetupRequirementsError(specifier_list)
setuptools.build_meta.SetupRequirementsError: ['cython', 'numpy', 'setuptools>=18.0']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 280, in
main()
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py", line 114, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 149, in get_requires_for_build_wheel
return self._get_build_requires(
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 130, in _get_build_requires
self.run_setup()
File "C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-build-env-hxrh3fi7\overlay\Lib\site-packages\setuptools\build_meta.py", line 145, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 217, in
os.unlink(fairseq_examples)
PermissionError: [WinError 5] Access is denied: 'fairseq\examples'

ERROR: Command errored out with exit status 1: 'C:\Program Files\Anaconda\python.exe' 'C:\Program Files\Anaconda\lib\site-packages\pip_vendor\pep517_in_process.py' get_requires_for_build_wheel 'C:\Users\Public\Documents\Wondershare\CreatorTemp\tmp90q37bss' Check the logs for full command output.

@nreimers
Copy link
Member

nreimers commented Jan 28, 2021

@sriprad
The issue is with fairseq, it is sadly not compatible with windows due to a badly written setup script.

Fairseq is requried for the mBART and m2m model.

If you use opus-mt, fairseq is not required. You can install it then like this:

pip install --no-deps easynmt
pip install tqdm transformers numpy nltk sentencepiece  

You also need pytorch:
https://pytorch.org/get-started/locally/

If you need automatic language detection, you also need fastText, which can be installed like this:

pip install fasttext 

or when you use Anaconda: https://anaconda.org/conda-forge/fasttext

In that case you can use the opus-mt model.

@sriprad
Copy link
Author

sriprad commented Jan 28, 2021

Thank you so much @nreimers . Really helpful. However i landed up in another issue may be not related to library but got this one if you can help. I always landed up in the wrong place to download this. if you can help me here.
building 'fasttext_pybind' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

@sriprad
Copy link
Author

sriprad commented Jan 28, 2021

Also i am running the code like this hope this is correct.
from easynmt import EasyNMT
model = EasyNMT('opus-mt')

#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', target_lang='de'))

#Translate several sentences to German
sentences = ['You can define a list with sentences.',
'All sentences are translated to your target language.',
'Note, you could also mix the languages of the sentences.']
print(model.translate(sentences, target_lang='de'))

@nreimers
Copy link
Member

Thank you so much @nreimers . Really helpful. However i landed up in another issue may be not related to library but got this one if you can help. I always landed up in the wrong place to download this. if you can help me here.
building 'fasttext_pybind' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

fastText is used for the automatic language detection.

When on Windows and using Anaconda, you can install it like this:
https://anaconda.org/conda-forge/fasttext

Yes, the code you showed is correct.

@sriprad
Copy link
Author

sriprad commented Jan 29, 2021 via email

@nreimers
Copy link
Member

If you provide the source language (source_lang=), fastText is not needed. In that case, you don't have to install it.

Translation of documents (like word, powerpoint, excel) is sadly not yet supported (it is on the agenda, help appreciated).

But you could implement it yourself for excel:

  • Store it as CSV
  • Load it with Python
  • Translate all entries
  • Write it back to CSV
  • Load with excel

@sriprad
Copy link
Author

sriprad commented Jan 29, 2021

Thank you @nreimers if i provide (source_lang=) then how will be the structure of the code like what changes i need to make below to include

  1. source_lang =
  2. pass the csv file.

from easynmt import EasyNMT
model = EasyNMT('opus-mt')

#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', target_lang='de'))

#Translate several sentences to German
sentences = ['You can define a list with sentences.',
'All sentences are translated to your target language.',
'Note, you could also mix the languages of the sentences.']
print(model.translate(sentences, target_lang='de'))

Sure happy to support in development

@nreimers
Copy link
Member

You code must then look like this:

from easynmt import EasyNMT
model = EasyNMT('opus-mt')

#Translate a single sentence to German
print(model.translate('This is a sentence we want to translate to German', source_lang='en', target_lang='de'))

Note, with version 1.0.2:
https://github.com/UKPLab/EasyNMT/releases/tag/v1.0.2

I added two more options for language detection that are compatible with Windows:

pip install langid

If fastText is not installed, it will fall back to either langid or to langdetect.

@sriprad
Copy link
Author

sriprad commented Jan 29, 2021

Thank you @nreimers . I could run it with your examples.
Please find below i am trying to upload a csv with a column needs to be translated.
how should i pass the dataframe?

thanks

df = pd.read_csv("C:/xx/trans.csv",encoding = 'unicode_escape')
#how to pass the dataframe?
print(model.translate(, source_lang='en', target_lang='de'))

@nreimers
Copy link
Member

Hi @sriprad
You could try:

df['translated_text'] = model.translate(df['source_text'], source_lang='en', target_lang='de')

Not sure how the pandas DF looks for your csv. You might have to update the column names in the Python code.

@sriprad
Copy link
Author

sriprad commented Jan 29, 2021

Thank you @nreimers . very helpful.
Do you need GPU to run it? I am running the translation on 10k rows of data. With each row has varying line length of 5 to 8. It's been running for last 30 mins and still running.

It has finally run with 1 hour 25 mins. But brilliant . The translation is amazing. Great work @nreimers

@nreimers
Copy link
Member

Happy to hear that :)

You can pass the show_progress_bar=True to the fit method.

But yes, these models are quite slow on a CPU (see Readme). I can recommend to use Google Colab, there, you get a GPU for free which significantly speeds up the processing.

@sriprad
Copy link
Author

sriprad commented Feb 2, 2021

Thank you @nreimers :). sure i will check in with Google Colab. But not sure if we are allowed to use Google Colab for official purpose.

@nreimers
Copy link
Member

nreimers commented Feb 2, 2021

@sriprad
A docker container will soon be published. This will make it easy to run it (as long as you have docker installed).

@sriprad
Copy link
Author

sriprad commented Feb 4, 2021

@nreimers thank you . Is there a possibility of extending this to convert word doc? or lengthy contracts please?

@nreimers
Copy link
Member

nreimers commented Feb 4, 2021

@sriprad
Translating word docs is quite difficult, as docx is quite a complex format. Further, the content is mixed together with style & format commands. So extracting the text, translating it, and putting it back to a valid a nicely formatted word document is non-trivial.

@jonbesga
Copy link

jonbesga commented Feb 5, 2021

This is also happening to me in Arch Linux.

Edit: Fixed it using Python 3.7 instead of Python 3.9

@mzeidhassan
Copy link

mzeidhassan commented Feb 10, 2021

@nreimers If you can somehow use Okapi filters, you would be able to use most formats.
https://okapiframework.org/wiki/index.php/Filters

Or maybe, give higher priority to xliff files, since this is the main format in translation industry.

@IsaacDC
Copy link

IsaacDC commented Apr 17, 2024

Im having trouble trying to download easyNMT,, when i do 'pip install easynmt' i get this error:

Building wheels for collected packages: fasttext
Building wheel for fasttext (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [75 lines of output]
...

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for fasttext
Running setup.py clean for fasttext
Failed to build fasttext
ERROR: Could not build wheels for fasttext, which is required to install pyproject.toml-based projects.

It also says that fasttext is required for easynmt 2.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants