Skip to content

Using EasyBuild to produce dependecies as modules

Jump to bottom
mvdebolskiy edited this page Dec 3, 2024 · 1 revision

F.e. you want to modify an existing ESMF module that is referenced in ccs_config: You can see that on betzy, with intel compiler and openmpi mpilib we do the following: module use /cluster/shared/noresm/eb_mods_2023b/modules/all module load ESMF/8.6.1-iomkl-2023b-ParallelIO-2.6.3

If you did module use, you can see where does the module's .lua file points to for library paths: module show ESMF/8.6.1-iomkl-2023b-ParallelIO-2.6.3 For user-built software it is usually located in software folder: /cluster/shared/noresm/eb_mods_2023b/software/ESMF/8.6.1-iomkl-2023b-ParallelIO-2.6.3/ Inside the folder above, you can find an easybuild folder that contains log files and .eb config file as well as reprod folder. reprod has .env file environment variables and modules used during the build, .eb config file, as well as a folder with easyblocks that were used.

You can use these to reproduce a build of the software you want. F.e., if you want to rebuild ESMF:

cd ~
mkdir ebconf
mkdir ebblocks
cp -f /cluster/shared/noresm/eb_mods_2023b/software/ESMF/8.6.1-iomkl-2023b-ParallelIO-2.6.3/easybuild/reprod/*.eb ./ebconf
cp -f /cluster/shared/noresm/eb_mods_2023b/software/ESMF/8.6.1-iomkl-2023b-ParallelIO-2.6.3/easybuild/reprod/easyblocks/esmf.py ./ebblocks
module purge
module load EasyBuild/4.9.4
module use /cluster/shared/noresm/eb_mods_2023b/modules/all
export EASYBUILD_INCLUDE_EASYBLOCKS=$HOME/ebblocks/\*.py
cd ./ebconf
eb ./ESMF-8.6.1-iomkl-2023b-ParallelIO-2.6.3.eb --robot --robot-paths :$HOME/ebconfs/  --installpath=<where you want to install things>  --use-existing-modules --tmp-logdir=$USERWORK/eb_log_dir --buildpath=$USERWORK/eb_bld_dir

Note, that --installpath is the prefix where your modules and software folders will be located. Since we specified --use-existing-modules and module use /cluster/shared/noresm/eb_mods_2023b/modules/all Easybuild will load the dependecies for ESMF that as modules from that location if they are not already in the $MODULEPATH. If you are changing dependency versions/toolschains you will need to build them beforehand and module use, so they are loaded as modules. You can build the whole stack at ones if you have .eb files for dependencies in your robotpaths.

Clone this wiki locally