diff --git a/docs/source/370_jupyter_lmod.rst b/docs/source/370_jupyter_lmod.rst new file mode 100644 index 000000000..7dc95ef62 --- /dev/null +++ b/docs/source/370_jupyter_lmod.rst @@ -0,0 +1,130 @@ +Jupyter and Lmod +================ + +It is possible to use Lmod inside of Jupyter with the terminal or jupyter-lmod. + +jupyter-lmod is a frontend and server extension for Jupyter (notebook and jupyterlab). +The extension graphically provides most of Lmod's functions: + - Browser and filter available modules + - Load and unload modules + - Display a module content + - Create and load collections + +The extension is located in the 'Softwares' tab of the main interface in Jupyter Notebook, +while in JupyterLab, it can be accessed as a sidepanel by clicking on the Lmod logo icon +located in the sidebar. + +Installation +~~~~~~~~~~~~ + +To install jupyter-lmod extension: :: + + pip install jupyterlmod + +This enables the extension for both Jupyter Notebook and JupyterLab. + +How does it work? +~~~~~~~~~~~~~~~~~ + +jupyter-lmod uses lmod python interface to generate Python code +that modifies the environment variables of the Python process running Jupyter. All child +processes of Jupyter (including kernels) created after Lmod calls inherit +the environment variables as defined by the set of loaded modules. + +In concrete terms, kernels in use by active notebooks will not be aware of changes +made to the environment (e.g. **$PATH**, **$PYTHONPATH**, **$LD_LIBRARY_PATH**) by modules +loaded after they were spawned. Depending on the Jupyter version, a manual kernel restart +or a complete kernel shutdown/start cycle is needed to make an active notebook aware of changes +to the environment variables. + +Lmod environment variable configuration for jupyter-lmod +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The plugin requires the environment variable **$LMOD_CMD** to be defined when Jupyter starts, +plus all environment variables required by Lmod to work properly (i.e.: **$MODULEPATH**). + +Some JupyterHub spawners do not setup the complete user's environment, leaving **$LMOD_CMD** +and **$MODULEPATH** undefined. To address this issue, the environment variables have to be +defined in **jupyterhub_config.py** using +`c.Spawner.environment `_. + +If the environment variables are defined before JupyterHub launch, you can add their +names only to `c.Spawner.env_keep `_ +instead. + +Controlling launcher items with jupyter-lmod +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +jupyter-lmod can add a kernel or a server proxy to the Jupyter launcher +following module loading if the module matches some specifications. + +kernels ++++++++ + +If a module adds one or more absolute paths to **$JUPYTER_PATH**, Jupyter's kernel spec manager +will be notified of the changes. If the added paths contain kernel spec files (e.g: **kernel.json**), +the kernels will be added to the available choices in Jupyter interface. + +The following example demonstrates this feature with Julia. Here, we have an **ijulia-kernel/1.8** module +that prepends a path to **$JUPYTER_PATH**. :: + + whatis("Description: IJulia kernelspec for Julia 1.8") + whatis("Homepage: https://julialang.org/") + whatis("URL: https://julialang.org/") + conflict("ijulia-kernel") + depends_on("julia/1.8.1") + depends_on("ijulia/.1.24.0") + prepend_path("JUPYTER_PATH","/software/Core/ijulia-kernel/1.8/share/jupyter") + + +In **/software/Core/ijulia-kernel/1.8/share/jupyter/kernels/julia-1.8**, there are four files :: + + kernel.json logo-32x32.png logo-64x64.png logo-svg.svg + +and **kernel.json** looks like this :: + + { + "display_name": "Julia 1.8.1", + "argv": [ + "/software/Core/julia/1.8.1/bin/julia", + "-i", + "--color=yes", + "/software/Core/ijulia/1.24.0/packages/IJulia/6TIq1/src/kernel.jl", + "{connection_file}" + ], + "language": "julia", + "env": {}, + "interrupt_mode": "signal" + } + +The folders and files under **share/jupyter/kernels** are typically generated by a kernelspec installer. + + +server proxies +++++++++++++++ + +Jupyter Server Proxy is an extension for Jupyter and JupyterLab that let users run arbitrary +external processes (such as RStudio, Shiny Server, Syncthing, PostgreSQL, Code Server, etc) +alongside their notebook server and provide authenticated web access to them using a path +like /rstudio next to others like /lab. It also provides buttons in the launcher to launch +these external processes. + +When jupyter-lmod and jupyter-server-proxy are both installed, it is possible for the launcher +button of an external process to appear only when its corresponding module is loaded. +In order to do so, the frontend extension of jupyter-server-proxy must be deactivated as +the control of the launcher will be entrusted to jupyter-lmod. :: + + jupyter nbextension disable --py jupyter_server_proxy --sys-prefix + jupyter labextension disable @jupyterlab/server-proxy + jupyter labextension disable jupyterlab-server-proxy + +Then, the name of each `server proxy ` +must match the name of module. When a module with a name corresponding to a server +proxy name is loaded, jupyter-lmod will add a button to the launcher to start that process. + +References +~~~~~~~~~~ + +For more information, refer to jupyter-lmod github repo: +https://www.github.com/cmd-ntrf/jupyter-lmod + diff --git a/docs/source/index.rst b/docs/source/index.rst index f99a5d75d..fd04e3e81 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -146,6 +146,7 @@ Advanced Topics 340_inherit 350_community 360_check_syntax + 370_jupyter_lmod Internal Structure of Lmod ^^^^^^^^^^^^^^^^^^^^^^^^^^