Make pip changes additive

google-deepmind · Dec 5, 2019 · 025a480 · 025a480
1 parent b691354
commit 025a480
Show file tree

Hide file tree

Showing 9 changed files with 273 additions and 103 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -30,5 +30,5 @@ matrix:
 
 script:
   - ./install.sh
-  - python3 --version
+  - python --version
   - ./open_spiel/scripts/travis_script.sh
diff --git a/docs/install.md b/docs/install.md
@@ -1,99 +1,7 @@
 # Installation
 
 The instructions here are for Linux and MacOS. For installation on Windows, see
-[these separate installation instructions](windows.md).
-
-## Summary
-
-1.  Install system packages and download some dependencies. Only needs to be
-    run once.
-
-    ```bash
-    ./install.sh
-    ```
-
-2.  Install your Python dependencies, e.g. in Python 3 using
-    [`virtualenv`](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/):
-
-    ```bash
-    virtualenv -p python3 venv
-    source venv/bin/activate
-    pip3 install --upgrade .
-    ```
-
-    Use `deactivate` to quit the virtual environment.
-
-3.  Build and run tests to check everything works:
-
-    ```bash
-    ./open_spiel/scripts/build_and_run_tests.sh
-    ```
-
-To make sure OpenSpiel works on the default configurations, we do use the
-`python3` command and not `python` (which still defaults to Python 2 on modern
-Linux versions).
-
-## Detailed steps
-
-### Installing system-wide dependencies
-
-See `install.sh` for the required packages and cloned repositories.
-
-### Installing Python dependencies
-
-Using a `virtualenv` to install python dependencies is highly recommended. For
-more information see:
-[https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/)
-
-Install dependencies (Python 3):
-
-```bash
-virtualenv -p python3 venv
-source venv/bin/activate
-pip3 install --upgrade .
-```
-
-Alternatively, although not recommended, you can install the Python dependencies
-system-wide with:
-
-```bash
-pip3 install --upgrade .
-```
-
-### Building and running tests
-
-Make sure that the virtual environment is still activated.
-
-Build and run tests (Python 3):
-
-```bash
-mkdir build
-cd build
-CXX=g++ cmake -DPython_TARGET_VERSION=3.6 -DCMAKE_CXX_COMPILER=${CXX} ../open_spiel
-make -j$(nproc)
-ctest -j$(nproc)
-```
-
-The CMake variable `Python_TARGET_VERSION` is used to specify a Python version.
-Any Python library found with CMake modules FindPython2/FindPython3 that agrees
-with the major version and is at least as high for minor version and patch
-number is accepted. If the variable is not set, the FindPython module is used:
-it builds for Python 3 if both Python 2 and Python 3 are available. In the two
-examples above, CMake will search for Python 2 and accept any version >= 2.7 or
-search for Python 3 and accept any version >= 3.6.
-
-One can run an example of a game running (in the `build/` folder):
-
-```bash
-./examples/example --game=tic_tac_toe
-```
-
-# Running the first example
-
-In the `build` directory, running `examples/example` will prints out a list of
-registered games and the usage. Now, let’s play game of Tic-Tac-Toe with uniform
-random players:
-
-```bash
-examples/example --game=tic_tac_toe
-```
+[these separate installation instructions](windows.md). Currently there are two
+installation methods:
+1. [editing `PYTHONPATH`](install_pythonpath.md) or
+2. [using `pip`](install_pip.md).
diff --git a/docs/install_pip.md b/docs/install_pip.md
@@ -0,0 +1,48 @@
+# Installation using `pip`
+
+This covers installation using the Python package manager `pip`.
+
+## Summary
+
+1.  Install system packages and download some dependencies. Only needs to be
+    run once.
+
+    ```bash
+    ./install.sh
+    ```
+
+2.  Install your Python dependencies, e.g. in Python 3 using
+    [`virtualenv`](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/):
+
+    ```bash
+    virtualenv -p python3 venv
+    source venv/bin/activate
+    python3 -m pip install .
+    ```
+
+    Optionally, add `-e` to the last command to install in
+    [editable mode](https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs),
+    which will allow you to skip this step if you edit any Python source files.
+    If you edit any C++ files, you will have to rerun the install command.
+
+    Use `deactivate` to quit the virtual environment.
+
+3.  Run tests to check everything works:
+
+    ```bash
+    python setup.py test
+    ```
+
+To make sure OpenSpiel works on the default configurations, we do use the
+`python3` command and not `python` (which still defaults to Python 2 on modern
+Linux versions).
+
+# Running the first example
+
+In the `build` directory, running `examples/example` will prints out a list of
+registered games and the usage. Now, let’s play game of Tic-Tac-Toe with uniform
+random players:
+
+```bash
+examples/example --game=tic_tac_toe
+```
diff --git a/docs/install_pythonpath.md b/docs/install_pythonpath.md
@@ -0,0 +1,127 @@
+# Installation using `PYTHONPATH`
+
+This covers installation by editing the `PYTHONPATH` environment variable.
+
+## Summary
+
+1.  Install system packages and download some dependencies. Only needs to be
+    run once.
+
+    ```bash
+    ./install.sh
+    ```
+
+2.  Install your Python dependencies, e.g. in Python 3 using
+    [`virtualenv`](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/):
+
+    ```bash
+    virtualenv -p python3 venv
+    source venv/bin/activate
+    pip3 install -r requirements.txt
+    ```
+
+    Use `deactivate` to quit the virtual environment.
+
+3.  Build and run tests to check everything works:
+
+    ```bash
+    ./open_spiel/scripts/build_and_run_tests.sh
+    ```
+
+4.  Add
+
+    ```bash
+    # For the python modules in open_spiel.
+    export PYTHONPATH=$PYTHONPATH:/<path_to_open_spiel>
+    # For the Python bindings of Pyspiel
+    export PYTHONPATH=$PYTHONPATH:/<path_to_open_spiel>/build/python
+    ```
+
+    to `./venv/bin/activate` or your `~/.bashrc` to be able to import OpenSpiel
+    from anywhere.
+
+To make sure OpenSpiel works on the default configurations, we do use the
+`python3` command and not `python` (which still defaults to Python 2 on modern
+Linux versions).
+
+## Detailed steps
+
+### Installing system-wide dependencies
+
+See `install.sh` for the required packages and cloned repositories.
+
+### Installing Python dependencies
+
+Using a `virtualenv` to install python dependencies is highly recommended. For
+more information see:
+[https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/)
+
+Install dependencies (Python 3):
+
+```bash
+virtualenv -p python3 venv
+source venv/bin/activate
+pip3 install -r requirements.txt
+```
+
+Alternatively, although not recommended, you can install the Python dependencies
+system-wide with:
+
+```bash
+pip3 install --upgrade -r requirements.txt
+```
+
+### Building and running tests
+
+Make sure that the virtual environment is still activated.
+
+Build and run tests (Python 3):
+
+```bash
+mkdir build
+cd build
+CXX=g++ cmake -DPython_TARGET_VERSION=3.6 -DCMAKE_CXX_COMPILER=${CXX} ../open_spiel
+make -j$(nproc)
+ctest -j$(nproc)
+```
+
+The CMake variable `Python_TARGET_VERSION` is used to specify a Python version.
+Any Python library found with CMake modules FindPython2/FindPython3 that agrees
+with the major version and is at least as high for minor version and patch
+number is accepted. If the variable is not set, the FindPython module is used:
+it builds for Python 3 if both Python 2 and Python 3 are available. In the two
+examples above, CMake will search for Python 2 and accept any version >= 2.7 or
+search for Python 3 and accept any version >= 3.6.
+
+One can run an example of a game running (in the `build/` folder):
+
+```bash
+./examples/example --game=tic_tac_toe
+```
+
+### Setting Your PYTHONPATH environment variable
+
+To be able to import the Python code (both the C++ binding `pyspiel` and the
+rest) from any location, you will need to add to your PYTHONPATH the root
+directory and the `open_spiel` directory.
+
+When using a virtualenv, the following should be added to
+`<virtualenv>/bin/activate`. For a system-wide install, ddd it in your `.bashrc`
+or `.profile`.
+
+```bash
+# For the python modules in open_spiel.
+export PYTHONPATH=$PYTHONPATH:/<path_to_open_spiel>
+# For the Python bindings of Pyspiel
+export PYTHONPATH=$PYTHONPATH:/<path_to_open_spiel>/build/python
+```
+
+# Running the first example
+
+In the `build` directory, running `examples/example` will prints out a list of
+registered games and the usage. Now, let’s play game of Tic-Tac-Toe with uniform
+random players:
+
+```bash
+examples/example --game=tic_tac_toe
+```
diff --git a/install.sh b/install.sh
@@ -58,7 +58,7 @@ fi
 
 # 2. Install other required system-wide dependencies
 if [[ "$OSTYPE" == "linux-gnu" ]]; then
-  EXT_DEPS="cmake python3 python3-venv python3-dev python3-pip python3-setuptools python3-wheel"
+  EXT_DEPS="virtualenv cmake python3 python3-dev python3-pip python3-setuptools python3-wheel"
   APT_GET=`which apt-get`
   if [ "$APT_GET" = "" ]
   then

diff --git a/open_spiel/scripts/build_and_run_tests.sh b/open_spiel/scripts/build_and_run_tests.sh
@@ -20,20 +20,47 @@
 # It assumes:
 #   - we are at the root of the project
 #   - `install.sh` has been run
+#   - the Python dependencies has been installed, e.g. using
+#     `pip install --upgrade -r ../requirements.txt`.
 #
 # Using a virtualenv is recommended but not mandatory.
 #
 set -e  # exit when any command fails
 set -x
 
+CXX=g++
+NPROC=nproc
+if [[ "$OSTYPE" == "darwin"* ]]; then  # Mac OSX
+  NPROC="sysctl -n hw.physicalcpu"
+  CXX=/usr/local/bin/g++-7
+fi
+
+MAKE_NUM_PROCS=$(${NPROC})
+let TEST_NUM_PROCS=4*${MAKE_NUM_PROCS}
+
 PYVERSION=$(python3 -c 'import sys; print(sys.version.split(" ")[0])')
 
+BUILD_DIR="build"
+mkdir -p $BUILD_DIR
+cd $BUILD_DIR
+
 echo "Building and testing in $PWD using 'python' (version $PYVERSION)."
 
-python3 -m pip install .
-if python3 setup.py test; then
+cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=${CXX} ../open_spiel
+make -j$MAKE_NUM_PROCS
+
+export PYTHONPATH=$PYTHONPATH:`pwd`/../open_spiel
+export PYTHONPATH=$PYTHONPATH:`pwd`/python  # For the Python bindings of Pyspiel
+
+if ctest -j$TEST_NUM_PROCS --output-on-failure ../open_spiel; then
   echo -e "\033[32mAll tests passed. Nicely done!\e[0m"
 else
   echo -e "\033[31mAt least one test failed.\e[0m"
+  echo "If this is the first time you have run these tests, try:"
+  echo "pip install -r requirements.txt"
+  echo "Note that outside a virtualenv, you will need to install the system "
+  echo "wide matplotlib: sudo apt-get install python-matplotlib"
   exit 1
 fi
+
+cd ..
diff --git a/open_spiel/scripts/pip_install_and_run_tests.sh b/open_spiel/scripts/pip_install_and_run_tests.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+# Copyright 2019 DeepMind Technologies Ltd. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# The following builds open_spiel and executes the tests using the `python`
+# command. The version under 'python' is automatically detected.
+#
+# It assumes:
+#   - we are at the root of the project
+#   - `install.sh` has been run
+#
+# Using a virtualenv is recommended but not mandatory.
+#
+set -e  # exit when any command fails
+set -x
+
+PYVERSION=$(python3 -c 'import sys; print(sys.version.split(" ")[0])')
+
+echo "Building and testing in $PWD using 'python' (version $PYVERSION)."
+
+python3 -m pip install .
+if python3 setup.py test; then
+  echo -e "\033[32mAll tests passed. Nicely done!\e[0m"
+else
+  echo -e "\033[31mAt least one test failed.\e[0m"
+  exit 1
+fi