3. Install and Build HPC-Stack on MacOS

HPC-Stack can be installed and built on MacOS systems. The following two options have been tested:

  • Option 1: MacBookAir 2020, M1 chip (arm64, running natively), 4+4 cores, Big Sur 11.6.4, GNU compiler suite v.11.2.0 (gcc, gfortran, g++); no MPI pre-installed

  • Option 2: MacBook Pro 2015, 2.8 GHz Quad-Core Intel Core i7 (x86_64), Catalina OS X 10.15.7, GNU compiler suite v.11.2.0 (gcc, gfortran, g++); no MPI pre-installed

Note

Examples throughout this chapter presume that the user is running Terminal.app with a bash shell environment. If this is not the case, users will need to adjust commands to fit their command line application and shell environment.

3.1. Prerequisites for Building HPC-Stack

3.1.1. Install Homebrew and Xcode Command-Line Tools (CLT)

Open Terminal.app and a web browser. Go to https://brew.sh, copy the command-line installation directive, and run it in a new Terminal window. Terminal will request a sudo access password. The installation command will look similar to the following:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

This will install Homebrew, Xcode CLT, and Ruby.

An alternative way to install the Xcode command-line tools (CLT) is as follows:

xcode-select --install

Note the messages at the end of the installation. Users may need to update the environment variable $PATH and add it to the shell initialization:

export PATH=/opt/homebrew/bin:$PATH
echo 'export PATH="/opt/homebrew/bin:$PATH"' >> ~/.bashrc

3.1.2. Install Compilers

Install GNU compiler suite (version 11) with gfortran:

brew install gcc@11

Create symbolic links from the version-specific binaries to gcc and g++. A sudo password may be requested. The path will likely be /opt/homebrew/bin/gcc-11 (Option 1), or /usr/local/bin/gcc-11 (Option 2).

which gcc-11
cd /opt/homebrew/bin/    (Version 2: cd /usr/local/bin/)
ln -s gcc-11 gcc
ln -s g++-11 g++

Verify that compiler path installed using homebrew takes precedence over /usr/bin path with system compilers: echo $PATH.

Check if a previous version of gfortran exists; rename it in that case (e.g., to “gfortran-X”) and create a link to a newer binary:

which gfortran
mv gfortran gfortran-X
ln -s gfortran-11 gfortran

Verify the paths for the compiler binaries:

which gcc
which g++
which gfortran

Verify that they show the correct version of GNU installed:

gcc --version
g++ --version
gfortran --version

3.1.3. Install CMake

Install the cmake utility via homebrew:

brew install cmake

3.1.4. Install/Upgrade Make

To install the make utility via homebrew:

brew install make

To upgrade the make utility via homebrew:

brew upgrade make

3.1.5. Install Openssl@3

To install the openssl@3 package, run:

brew install openssl@3

Note the messages at the end of the installation. Depending on what they say, users may need to add the location of the openssl@3 binaries to the environment variable $PATH. To add it to the PATH, run:

echo 'export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.bashrc

Users may also need to set certain flags so that the compilers can find the openssl@3 package:

export LDFLAGS="-L/opt/homebrew/opt/openssl@3/lib"
export CPPFLAGS="-I/opt/homebrew/opt/openssl@3/include"

3.1.6. Install Lmod

Install Lmod, which is the module management environment, run:

brew install lmod

You may need to add the Lmod environment initialization to your shell profile, e.g., to $HOME/.bashrc.

For the Option 1 installation, add:

export BASH_ENV="/opt/homebrew/opt/lmod/init/profile"
source $BASH_ENV

For the Option 2 installation, add:

export BASH_ENV="/usr/local/opt/lmod/init/profile"
source $BASH_ENV

3.1.7. Install libpng

The libpng library has issues when building on MacOS during the HPC-Stack bundle build. Install it separately using homebrew:

brew install libpng

3.1.8. Install wget

Install the Wget software package:

brew install wget

3.1.9. Install or Update Python3

First, verify that Python3 is installed, and check the current version:

which python3
python3 --version

The first command should return /usr/bin/python3 and the second should return Python 3.8.2 or similar (the exact version is unimportant).

For Python3 installation and updates on MacOS refer to https://www.python.org/downloads. A 64-bit universal2 installer package is recommended (python-3.9.11-macos11.pkg latest at the time of writing). Double-click on the installer package, and accept the license terms. An administrative level password will be requested for the installation. At the end of the installation, run Install Certificates.command by double-clicking on the shell script in Finder.app that opens and runs it.

Start a new bash session (type bash in the existing terminal), and verify the installed version:

python3 --version

The output should now correspond to the Python version you installed.

3.1.10. Install Git

Install git and dependencies:

brew install git

3.2. Building HPC-Stack

3.2.1. Clone HPC-Stack

Download HPC-Stack code from GitHub:

git clone https://github.com/NOAA-EMC/hpc-stack.git
cd hpc-stack

The configuration files are ./config/config_<machine>.sh. For Option 1, <machine> is mac_m1_gnu and for Option 2, <machine> is mac_gnu.

The ./stack/stack_<machine>.yaml file lists the libraries that will be built as part of HPC-Stack, in addition to library-specific options. These can be altered based on user preferences.

3.2.2. Lmod Environment

Verify the initialization of Lmod environment, or add it to the configuration file ./config/config_<machine>.sh, as in Step 3.1.6.

For Option 1:

export BASH_ENV="/opt/homebrew/opt/lmod/init/profile"
source $BASH_ENV

For Option 2:

export BASH_ENV="/usr/local/opt/lmod/init/profile"
source $BASH_ENV

3.2.3. Specify Compiler, Python, and MPI

Specify the combination of compilers, python libraries, and MPI libraries in the configuration file ./config/config_<machine>.sh.

export HPC_COMPILER="gnu/11.2.0"
export HPC_MPI="openmpi/4.1.2"
export HPC_PYTHON="python/3.10.2"

Comment out any export statements not relevant to the system, and make sure that version numbers reflect the versions installed on the system (which may differ from the versions listed here).

3.2.4. Set Appropriate Flags

When using gfortran version 10 or higher, verify that the following flags are set in config_<machine>.sh:

For Option 1:

export STACK_FFLAGS="-fallow-argument-mismatch -fallow-invalid-boz"

For Option 2:

export STACK_FFLAGS=“-fallow-argument-mismatch -fallow-invalid-boz”
export STACK_CXXFLAGS="-march=native"

3.2.5. Set Environment Variables

Set the environmental variables for compiler paths in ./config/config_<machine>.sh. The variable GNU below refers to the directory where the compiler binaries are located. For example, with Option 1, GNU=/opt/homebrew/bin, and with Option 2: GNU=/usr/local/bin.

export GNU="path/to/compiler/binaries"
export CC=$GNU/gcc
export FC=$GNU/gfortran
export CXX=$GNU/g++
export SERIAL_CC=$GNU/gcc
export SERIAL_FC=$GNU/gfortran
export SERIAL_CXX=$GNU/g++

3.2.6. Specify MPI Libraries

Specify the MPI libraries to be built within the HPC-Stack in ./stack/stack_<machine>.yaml. The openmpi/4.1.2 (Option 1 and Option 2) and mpich/3.3.2 (Option 2) have been built successfully.

mpi:
build: YES
flavor: openmpi
version: 4.1.2

3.2.7. Libpng

Set build libpng library to NO in ./stack/stack_<machine>.yaml to avoid problems during the HPC-Stack build. Leave the defaults for other libraries and versions in ./stack/stack_<machine>.yaml.

libpng:
build: NO

3.2.8. Set Up the Modules and Environment

Set up the modules and environment:

./setup_modules.sh -c config/config_<machine>.sh -p $HPC_INSTALL_DIR | tee setup_modules.log

where <machine> is mac_m1_gnu (Option 1), or mac_gnu (Option 2), and $HPC_INSTALL_DIR is the absolute path of the HPC-stack installation directory. When asked whether to use “native” Python or compilers, choose “YES” if using those already installed on your system, or “NO” if they will be built during the HPC-stack installation. The likely response is to answer “YES” to python, “YES” to compilers, and “NO” for MPI/mpich.

3.2.9. Building HPC-Stack

Build the modules:

./build_stack.sh -c config/config_<machine>.sh -p $HPC_INSTALL_DIR  -y stack/stack_<machine>.yaml -m 2>&1 | tee build_stack.log

Attention

  • The option -p requires an absolute path (full path) of the installation directory!

  • The -m option is needed to build separate modules for each library package.