7. HPC-Stack Additional Notes
7.1. Setting Compiler Flags and Other Options
Often it is necessary to specify compiler flags (e.g., gfortran-10 requires
-fallow-argument-mismatch``to be used with ``FFLAGS. There are 2 ways this can be achieved:
For all packages: One can define variable e.g.,
STACK_FFLAGS=-fallow-argument-mismatchin the config file
config_custom.sh. This will append
FFLAGSin every build script under ./libs/ directory.
Package-specific flags: To compile only the specific package under
libswith the above compiler flag, one can define variable
<package>section of the YAML file
stack_custom.yaml. This will append
FFLAGSin the build script for that package only.
7.2. Adding a New Library or Package
If you want to add a new library to the stack you need to follow these steps:
Write a new build script in
libs, using existing scripts as a template.
Define a new section in the
yamlfile for that library/package in
If the package is a python virtual environment, add a
environment.ymlfile listing the python packages required to install the package. These files should be named and placed in
VENVTYPE=pyvenvwill use the
Add a call to the new build script in
Create a new module template at the appropriate place in the modulefiles directory, using exising files as a template.
Update the HPC Components file to include the name of the new library or package.
7.3. Configuring for a new HPC
If you want to port this to a new HPC, you need to follow these steps:
Write a new config file
config/config_<hpc>.sh, using existing config files as a template. Also create a new yaml file
config/stack_<hpc>.yaml, using existing yaml files as a template.
Add/remove basic modules for that HPC.
Choose the appropriate Compiler/MPI combination.
If a template modulefile does not exist for that Compiler/MPI combinattion, create module templates at the appropriate place in the
modulefilesdirectory, using existing files as a template (e.g.,
If the new HPC system provides some basic modules for e.g., Git, CMake, etc., they can be loaded in
7.4. Using the DOWNLOAD_ONLY Option
If an HPC (e.g., NOAA RDHPCS Hera) does not allow access to online software via
git clone, you will have to download all the packages using the
DOWNLOAD_ONLY option in the
build_stack.sh as you would on a machine that does allow access to online software with
DOWNLOAD_ONLY=YES and all the packages will be downloaded in the
pkg directory. Transfer the contents of the
pkg directory to the machine where you wish to install the HPC-Stack, and execute
build_stack.sh script will detect the already-downloaded packages and use them rather than fetching them.
7.5. Using the HPC-Stack
If Lmod is used to manage the software stack, you will need to activate the HPC-Stack in order to use it. This is done by loading the
module use $PREFIX/modulefiles/stack module load hpc/1.0.0
This will put the
hpc-<compilerName> module in your
MODULEPATH, which can be loaded as:
module load hpc-<compilerName>/<compilerVersion>
If the HPC-Stack is not managed via modules, you need to add
$PREFIXto the PATH as follows:
export PATH="$PREFIX/bin:$PATH" export LD_LIBRARY_PATH="$PREFIX/lib:$LD_LIBRARY_PATH" export CMAKE_PREFIX_PATH="$PREFIX"
7.6. Known Workaround for Certain Installations of Lmod
On some machines (e.g., WCOSS_DELL_P3), LMod is built to disable loading of default modulefiles and requires the user to load the module with an explicit version of the module (e.g.,
module load netcdf/4.7.4instead of
module load netcdf). The latter looks for the
defaultmodule which is either the latest version or a version that is marked as default. To circumvent this, it is necessary to place the following lines in
modulefiles/stack/hpc/hpc.luaprior to executing
setenv("LMOD_EXACT_MATCH", "no") setenv("LMOD_EXTENDED_DEFAULT", "yes")
See more on the Lmod website.
7.7. Known Issues
NetCDF-C++ does not build with LLVM Clang. It can be disabled by setting
disable_cxx: YESin the stack file under the NetCDF section.
Json-schema-validator does not build with LLVM Clang. It can be disabled in the stack file in the json-schema-validator-section.