This repository has been archived by the owner on Nov 22, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 802
Specify CFLAGS to install fairseq in MacOS #1175
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
Do not delete this pull request or issue due to inactivity.
fb-exported
labels
Nov 30, 2019
This pull request was exported from Phabricator. Differential Revision: D18753339 |
…ed-bert Differential Revision: D18753252 fbshipit-source-id: 18bfe1d38d1afca4eb48a2e2e33ddc1b137aa047
Summary: Pull Request resolved: facebookresearch#1175 Currently, we installs fairseq by requirements.txt which is equivalent to ```pip install fairseq``` According to https://github.com/pytorch/fairseq, to install fairseq in MacOS, we need to run ```CFLAGS="-stdlib=libc++" pip install fairseq``` Before this diff: Can't install PyText in MacOS by $ ./install_deps Installing collected packages: click, pycparser, cffi, fastBPE, numpy, regex, portalocker, typing, sacrebleu, torch, tqdm, fairseq, attrs, hypothesis, joblib, typing-extensions, six, setuptools, protobuf, onnx, python-dateutil, certifi, urllib3, idna, chardet, requests, jmespath, docutils, botocore, s3transfer, boto3, pytorch-pretrained-bert, pytz, pandas, scipy, torchtext, wheel, grpcio, werkzeug, markdown, absl-py, tensorboard, sentencepiece, pytext-nlp Running setup.py install for pycparser ... done Running setup.py install for fastBPE ... error ERROR: Command errored out with exit status 1: command: /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"'; __file__='"'"'/private/var/folders/ sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-record-1kds02_a/install-record.txt --single-version-externally-managed --compile --install-headers /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/include/site/python3.7/fastBPE cwd: /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/ Complete output (16 lines): running install running build running build_py package init file 'fastBPE/__init__.py' not found (or not a regular file) running build_ext building 'fastBPE' extension creating build creating build/temp.macosx-10.7-x86_64-3.7 creating build/temp.macosx-10.7-x86_64-3.7/fastBPE gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/stevenliu/ anaconda3/include -arch x86_64 -I/Users/stevenliu/anaconda3/include -arch x86_64 -I/Users/stevenliu/workspace/sphinx/pytext/pytext_venv/ include -I/Users/stevenliu/anaconda3/include/python3.7m -c fastBPE/fastBPE.cpp -o build/temp.macosx-10.7-x86_64-3.7/fastBPE/fastBPE.o -std=c++11 -Ofast -pthread warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found] fastBPE/fastBPE.cpp:603:10: fatal error: 'ios' file not found #include "ios" ^~~~~ 1 warning and 1 error generated. error: command 'gcc' failed with exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"'; __file__='"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-record-1kds02_a/install-record.txt --single-version-externally-managed --compile --install-headers /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/include/site/python3.7/fastBPE Check the logs for full command output. BTW, made activation_venv executable as install_deps Differential Revision: D18753339 fbshipit-source-id: 9507101610de0f01fef5c766a315cf3269068565
hudeven
force-pushed
the
export-D18753339
branch
from
December 2, 2019 18:40
b90f4fa
to
bd89765
Compare
This pull request was exported from Phabricator. Differential Revision: D18753339 |
This pull request has been merged in 5cb79b7. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Currently, we installs fairseq by requirements.txt which is equivalent to
pip install fairseq
According to https://github.com/pytorch/fairseq, to install fairseq in MacOS, we need to run
CFLAGS="-stdlib=libc++" pip install fairseq
Before this diff:
Can't install PyText in MacOS by
$ ./install_deps
Installing collected packages: click, pycparser, cffi, fastBPE, numpy, regex, portalocker, typing, sacrebleu, torch, tqdm, fairseq, attrs, hypothesis, joblib, typing-extensions, six, setuptools, protobuf, onnx, python-dateutil, certifi, urllib3, idna, chardet, requests, jmespath, docutils, botocore, s3transfer, boto3, pytorch-pretrained-bert, pytz, pandas, scipy, torchtext, wheel, grpcio, werkzeug, markdown, absl-py, tensorboard, sentencepiece, pytext-nlp
Running setup.py install for pycparser ... done
Running setup.py install for fastBPE ... error
ERROR: Command errored out with exit status 1:
command: /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"'; file='"'"'/private/var/folders/ sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-record-1kds02_a/install-record.txt --single-version-externally-managed --compile --install-headers /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/include/site/python3.7/fastBPE
cwd: /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/
Complete output (16 lines):
running install
running build
running build_py
package init file 'fastBPE/init.py' not found (or not a regular file)
running build_ext
building 'fastBPE' extension
creating build
creating build/temp.macosx-10.7-x86_64-3.7
creating build/temp.macosx-10.7-x86_64-3.7/fastBPE
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/stevenliu/ anaconda3/include -arch x86_64 -I/Users/stevenliu/anaconda3/include -arch x86_64 -I/Users/stevenliu/workspace/sphinx/pytext/pytext_venv/ include -I/Users/stevenliu/anaconda3/include/python3.7m -c fastBPE/fastBPE.cpp -o build/temp.macosx-10.7-x86_64-3.7/fastBPE/fastBPE.o -std=c++11 -Ofast -pthread
warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
fastBPE/fastBPE.cpp:603:10: fatal error: 'ios' file not found
#include "ios"
^~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"'; file='"'"'/private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-install-x0x9inoi/fastBPE/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/sg/jcdjt2vn6pndx_d0vmf0l2n15y5jp7/T/pip-record-1kds02_a/install-record.txt --single-version-externally-managed --compile --install-headers /Users/stevenliu/workspace/sphinx/pytext/pytext_venv/include/site/python3.7/fastBPE Check the logs for full command output.
BTW, made activation_venv executable as install_deps
Differential Revision: D18753339