Skip to content

Commit

Permalink
Documentation updates for Mac and Linux
Browse files Browse the repository at this point in the history
  • Loading branch information
Natalie Perlin authored and Natalie Perlin committed Nov 28, 2022
1 parent 2e3742a commit b7a520d
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 25 deletions.
40 changes: 32 additions & 8 deletions docs/source/hpc-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,9 @@ The example for Ubuntu (tested for 20.04 and 22.04) is for a user with sudo priv
sudo apt install gcc-10 g++-10
sudo apt install gfortran-10
which gcc
which gcc-10
The location of default compilers is likely be in ``/usr/bin/`` (e.g. /usr/bin/gcc), and other versions could be installed with the same location with the version tag. Check all the versions installed and configure the alternatives to specify which version is to be default. Below is the example when two versions are available, e.g., gcc-9 and gcc-10.
Newly installed compiler versions and default compilers likely are located under ``/usr/bin/`` directory. Verify the location of a default compiler using ``which gcc`` command (expected ``/usr/bin/gcc``). Several versions of gcc, g++ and gfortran compilers may be present at the same location with the version tag, e.g. gcc-9, gcc-10, g++-9, g++-10. Check all the versions installed and configure the alternatives to specify a default version for each compier, as well further priorities. Below is the example on how to set alternatives and a default compiler for ``gcc`` when two versions are available, e.g., gcc-9 and gcc-10.

.. code-block:: console
Expand All @@ -156,16 +156,28 @@ The location of default compilers is likely be in ``/usr/bin/`` (e.g. /usr/bin/g
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 2
sudo update-alternatives --config gcc
Make selection (1) for gcc-10 to be the first priority (default).
Make a selection (1) for gcc-10 to be the first priority (default).

Repeat similar steps to configure the alternatives for g++ and gfortran. No need to configure for gfortran if you have only a single version.
Repeat similar steps to configure the alternatives for g++ and gfortran, if several versions are present.

.. code-block:: console
ls -la /usr/bin/g++-*
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 1
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 2
sudo update-alternatives --config g++
ls -la /usr/bin/gfortran-*
sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-10 1
sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-9 2
sudo update-alternatives --config gfortran
If there are no multiple versions, a symbolic link could be created instead of setting up alternatives. The example below shows how to create a link when only gfortran-10 is present:

.. code-block:: console
ls -la /usr/bin/gfortran*
ln -s /usr/bin/gfortran-10 /usr/bin/gfortran
Install ``lua``, ``luac`` and ``tcl`` needed to support the ``Lmod`` module management system. Verify the installation location.

Expand All @@ -177,11 +189,12 @@ Install ``lua``, ``luac`` and ``tcl`` needed to support the ``Lmod`` module mana
which luac
Standard installation paths for lua/luac are ``/usr/bin/lua`` and ``/usr/bin/luac``.
Download and install Lmod, the module management environment, if not installed in the system. When installed, the environmental variable $LMOD_ROOT is usually defined:
Download and install Lmod, the module management environment, if not installed in the system. When installed, the environmental variable $LMOD_ROOT or $LMOD_PKG are usually defined.

.. code-block:: console
echo $LMOD_ROOT
echo $LMOD_PKG
git clone https://github.com/TACC/Lmod.git
cd Lmod
Expand All @@ -200,7 +213,7 @@ i.e. to $HOME/.bash_profile (login bash session) or $HOME/.bashrc (non-login bas
export BASH_ENV=$HOME/apps/lmod/lmod/init/profile
source $BASH_ENV
Install wget, git, make and cmake; cmake could also be built bundled with the hpc-stack.
Install wget, git, make and cmake; cmake could also be built bundled with the hpc-stack as an alternative..

.. code-block:: console
Expand All @@ -223,7 +236,18 @@ Install make and cmake:
sudo apt install make
sudo apt install cmake
Both ``python2`` and ``python3`` are needed; python2 version higher than 2.7.x; python3 version higher than 3.6. Usually, ``python`` and ``python2`` are available with the Linux distribution. You could update ``python3`` if higher versions are available (python3.9 in the example below at the time of writing), and set the alternatives when multiple versions exist. The highest version among the existing python3.8 and python3.9 is set as a default in the example below (choose selection 1 for python3.9 when prompt).
Both ``python`` and ``python3`` commands need to be defined. The ``python`` could be set to python2 version higher than 2.7.x or to python3. python3 version needs to be higher than 3.6. Verify they are present and their versions:

.. code-block:: console
which python
which python3
python --version
python3 --version
apt list | grep python
Usually, ``python`` and ``python2`` are available with the Linux distribution. If no ``python`` is found, a symbolic link to ``python3`` could be created, or alternatives to use python3 could be set instead. Python3 could be installed or updated if higher versions than existing are available. The example below shows how to verify the current installed version (e.g. python3.8), update to a higher one (e.g, python3.9), and set the alternatives to use python3.9 as a default version of ``python3``. Choose selection 1 for python3.9 when prompt:

.. code-block:: console
Expand All @@ -236,7 +260,7 @@ Both ``python2`` and ``python3`` are needed; python2 version higher than 2.7.x;
sudo update-alternatives --config python3
python3 --version
Verify the version (python 3.9) set as default after the configuration.
Verify the version (python 3.9 in the above example) set as default using ``python3 --version``.


.. _NonConConfigure:
Expand Down
47 changes: 30 additions & 17 deletions docs/source/mac-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,24 +21,36 @@ Prerequisites for Building HPC-Stack
Install Homebrew and Xcode Command-Line Tools (CLT)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Open Terminal.app and a web browser. Go to https://brew.sh, copy the command-line installation directive, and run it in a new Terminal window. Terminal will request a ``sudo`` access password. The installation command will look similar to the following:
Open Terminal.app and a web browser. Go to https://brew.sh, copy the command-line installation directive, and run it in a new Terminal window. The installation command will look similar to the example below. A ``sudo`` access password will be promted to proceed with the installation.

.. code-block:: console
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
This will install Homebrew, Xcode CLT, and Ruby.
This will install Homebrew, and on some systems, also Xcode CLT, and Ruby.

An alternative way to install the Xcode command-line tools (CLT) is as follows:

.. code-block:: console
xcode-select --install
Note the messages at the end of the installation. Users may need to update the environment variable ``$PATH`` and add it to the shell initialization. Standard installation path could be queued using ``brew --prefix``. The instructions below assume a variable ``$BREW`` is set as following: ``BREW=$(brew --prefix)``.
Note the messages at the end of the installation. Users may need to update the environment variable ``$PATH`` and add it to the shell initialization, such as $HOME/.bash_profile (login shell), and $HOME/.bashrc (non-login interactive shell).

When XCode >= 14.x.x is installed on higher versions of MacOS (Ventura OS 13.x.x), some issues with linking executables in the end of building the hpc-stack have been reported. A suggested workaround was to downgrade the XCode to 13.x.x version. Verify the version of the XCode CLT:

.. code-block:: console
pkgutil --pkgs
# There likely be a package named 'com.apple.pkg.CLTools_macOS_SDK'
pkgutil --pkg-info com.apple.pkg.CLTools_macOS_SDK
Homebrew installs packages in their own independent directories, and subsequently creates links to package locations from a standard installation path. It is usually ``/home/homebrew/`` on systems with M1 (arm64), or ``/usr/local/`` on Intel (x86_64) systems. The Standard installation path could be queued using ``brew --prefix``. The instructions below set an environmentl variable ``$BREW`` for architecture-independent path substitutions:

.. code-block:: console
BREW=$(brew --prefix)
export PATH=$BREW/bin:$PATH
echo 'export PATH="$BREW/bin:$PATH"' >> ~/.bashrc
Expand All @@ -51,8 +63,7 @@ Install GNU compiler suite (version 11) with gfortran:
brew install gcc@11
Create symbolic links from the version-specific binaries to gcc and g++. A ``sudo`` password may be requested. Standard installation paths are ``/opt/homebrew/bin/`` (M1/arm64), or ``/usr/local/bin/`` (x86_64).

Create symbolic links from the version-specific binaries to ``gcc``, ``g++``, and ``gfortran``. You will likely be prompted for a ``sudo`` password. If previous versions of gcc, g++ or gfortran exist, it is recommended to rename them. For example, if existing gcc is version 9 ('gcc --version'

.. code-block:: console
Expand Down Expand Up @@ -182,13 +193,13 @@ Python (python2.7.x) is no longer provided with the MacOS version 12.3 (Monterey

.. code-block:: console
/bin/bash -c "$(curl -fsSL https://github.com/alfredapp/dependency-scripts/raw/main/scripts/install-python2.sh)"
/bin/bash -c "$(curl -fsSL https://github.com/alfredapp/dependency-scripts/raw/main/scripts/install-python2.sh)"
Python3 could be installed using the similar approach:

.. code-block:: console
/bin/bash -c "$(curl -fsSL https://github.com/alfredapp/dependency-scripts/raw/main/scripts/install-python3.sh)"
/bin/bash -c "$(curl -fsSL https://github.com/alfredapp/dependency-scripts/raw/main/scripts/install-python3.sh)"
For other Python3 installation ways and updates on MacOS refer to https://www.python.org/downloads. A 64-bit universal2 installer package is recommended (``python-3.9.11-macos11.pkg`` latest at the time of writing). Double-click on the installer package, and accept the license terms. An administrative level password will be requested for the installation. At the end of the installation, run ``Install Certificates.command`` by double-clicking on the shell script in Finder.app that opens and runs it.

Expand All @@ -200,14 +211,15 @@ Start a new bash session (type ``bash`` in the existing terminal), and verify th
The output should now correspond to the Python version you installed.

Install Git
^^^^^^^^^^^^^^^
Install Git and Git-lfs
^^^^^^^^^^^^^^^^^^^^^^^^^^

Install git and dependencies:
Install git, dependencies, and git-lfs:

.. code-block:: console
brew install git
brew install git-lfs
Building HPC-Stack
Expand All @@ -223,6 +235,8 @@ Download HPC-Stack code from `GitHub <github.com>`__:
git clone https://github.com/NOAA-EMC/hpc-stack.git
cd hpc-stack
An alternative and more updated location for the hpc-stack is on NOAA-EPIC repository: https://github.com/NOAA-EPIC/hpc-stack.git

The example of a configuration file is ``./config/config_macos_gnu.sh``.

The ``./stack/stack_macos.yaml`` file lists the libraries that will be built as part of HPC-Stack, in addition to library-specific options. These can be altered based on user preferences and particular application for which the HPC-stack is being built.
Expand Down Expand Up @@ -281,22 +295,21 @@ Set the environmental variables for compiler paths in ``./config/config_macos_gn
Specify MPI Libraries
^^^^^^^^^^^^^^^^^^^^^^^^

Specify the MPI libraries to be built within the HPC-Stack in ``./stack/stack_macos.yaml``. The ``openmpi/4.1.2`` has been built successfully on all the systems, and ``mpich/3.3.2`` on some.
Specify the MPI libraries to be built within the HPC-Stack in ``./stack/stack_macos.yaml``. When using GNU compilers installed with Homebrew, specify _NOT_ to build ``gnu`` compilers, and to build ``mpi`` libraries. The ``openmpi/4.1.2`` has been built successfully on all the systems, and ``mpich/3.3.2`` on some.

.. code-block:: console
gnu:
build: YES
version: 11.3.0
mpi:
build: YES
flavor: openmpi
version: 4.1.2
You could leave the defaults for other libraries and versions in ``./stack/stack_macos.yaml``.

.. code-block:: console
libpng:
build: NO

Set Up the Modules and Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -307,7 +320,7 @@ Set up the modules and environment:
./setup_modules.sh -c config/config_macos_gnu.sh -p $HPC_INSTALL_DIR | tee setup_modules.log
where the ``$HPC_INSTALL_DIR`` is the *absolute* path of the HPC-stack installation directory. When asked whether to use "native" Python or compilers, choose "YES" if using those already installed on your system, or "NO" if they will be built during the HPC-stack installation. The likely response is to answer "YES" to python, "YES" to compilers, and "NO" for MPI/openmpi.
where the ``$HPC_INSTALL_DIR`` is the *absolute* path of the HPC-stack installation directory. The $HPC_INSTALL_DIR needs to be different from the source directory, where you build and compile the software stack. When asked whether to use "native" Python or compilers, choose "YES" if using those already installed on your system, or "NO" if they will be built during the HPC-stack installation. The likely response is to answer "YES" to python, "YES" to compilers, and "NO" for MPI/openmpi.

Building HPC-Stack
^^^^^^^^^^^^^^^^^^^^^
Expand Down

0 comments on commit b7a520d

Please sign in to comment.