Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apptainer definition file for PySR #687

Merged
merged 28 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fa70d28
Initial working Apptainer definition
wardharold Jul 31, 2024
b62c58e
Final definition
wardharold Jul 31, 2024
0afb3c4
Revert to pip3 installation
wardharold Jul 31, 2024
3b85476
Use bullseye julia and python images mimic Dockerfile better
wardharold Aug 1, 2024
85b5275
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 1, 2024
75905b7
Merge branch 'MilesCranmer:master' into apptainer
wkharold Aug 1, 2024
1bf697b
Update Apptainer.def
wkharold Aug 1, 2024
70e1167
Add CI for apptainer
wardharold Aug 5, 2024
0781561
Use container Julia dependencies
wardharold Aug 7, 2024
d6df661
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 7, 2024
3056cfb
Merge branch 'MilesCranmer:master' into apptainer
wkharold Aug 7, 2024
69b86c1
Use PYTHONPATH to find pysr.version
wardharold Aug 7, 2024
dfe1652
Merge branch 'apptainer' of https://github.com/wkharold/PySR into app…
wardharold Aug 7, 2024
5a3f6c0
Update .github/workflows/CI_apptainer.yml
wkharold Aug 7, 2024
52965a8
Update Apptainer.def
wkharold Aug 7, 2024
385eb7f
Update Apptainer.def
wkharold Aug 7, 2024
1868051
Drop arguments sections for 1.1.x compatibility. Lose unnecessary JUL…
wardharold Aug 7, 2024
1a5c5ac
Skip notebook test on read-only file system
wardharold Aug 8, 2024
95f7e7e
Update CI_apptainer.yml
wkharold Aug 8, 2024
6f4c202
Update CI_apptainer.yml
wkharold Aug 9, 2024
f9d7bd6
Build tests into container; run tests in a separate directory
wardharold Aug 10, 2024
afc18cc
Update CI_apptainer.yml
wkharold Aug 11, 2024
e7013f2
Merge branch 'master' into apptainer
wkharold Aug 11, 2024
dc20020
Update .github/workflows/CI_apptainer.yml
MilesCranmer Aug 12, 2024
91274c0
fix: clean up apptainer env
MilesCranmer Aug 12, 2024
af4e093
ci: clean up CI
MilesCranmer Aug 12, 2024
24c516f
Merge branch 'master' into apptainer
MilesCranmer Aug 12, 2024
9ca474a
test: more portable way of checking writeable
MilesCranmer Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/CI_apptainer.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Apptainer

on:
push:
branches:
- 'master'
paths:
- '**'
tags:
- 'v*.*.*'
pull_request:
branches:
- 'master'
paths:
- '**'

jobs:
test:
runs-on: ${{ matrix.os }}
timeout-minutes: 60
defaults:
run:
shell: bash
strategy:
matrix:
os: [ubuntu-latest]

steps:
- uses: actions/checkout@v4
- uses: eWaterCycle/setup-apptainer@v2
with:
apptainer-version: 1.3.0
- name: Build apptainer
run: apptainer build --notest pysr.sif Apptainer.def
- name: Test apptainer
run: |
TMPDIR=$(mktemp -d)
cp pysr.sif $TMPDIR
cd $TMPDIR
apptainer test ./pysr.sif
55 changes: 55 additions & 0 deletions Apptainer.def
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Build an Apptainer SIF file containing a working copy of PySR and its prereqs
Bootstrap: docker
From: julia:1.10.4-bullseye
Stage: jl

Bootstrap: docker
From: python:3.12-bullseye
Stage: runtime

%environment
# Use the container Julia binary
export PATH="/usr/local/julia/bin:$PATH"

# Create a stacked environment for additional Julia packages
export JULIA_DEPOT_PATH="$HOME/.pysr:/pysr/depot:$JULIA_DEPOT_PATH"
export JULIA_LOAD_PATH="$HOME/.pysr:/pysr:$JULIA_LOAD_PATH"

%files from jl
/usr/local/julia /usr/local/julia

%files
./requirements.txt /pysr/requirements.txt
./pyproject.toml /pysr/pyproject.toml
./setup.py /pysr/setup.py
./pysr /pysr/pysr

%post
# Ensure we don't use the local pysr commands:
cd $(mktemp -d)

export PATH="/usr/local/julia/bin:$PATH"

# Install IPython and other useful libraries:
pip3 install --no-cache-dir ipython matplotlib pytest nbval
# Install PySR and requirements:
pip3 install --no-cache-dir /pysr

# Put the Julia dependencies in /pysr/depot
mkdir /pysr/depot
export JULIA_DEPOT_PATH="/pysr/depot"

# And set a specific environment for Julia dependencies
mkdir /pysr/env
export PYTHON_JULIAPKG_PROJECT="/pysr/env"

# Pull in all the Julia dependencies
python3 -c 'import pysr; pysr.load_all_packages()'

%test
python3 -m pysr test main,cli,startup

%runscript
# Start ipython when the container is executed
[ ! -d $HOME/.pysr ] && mkdir $HOME/.pysr
PYTHONPATH=/pysr ipython
2 changes: 2 additions & 0 deletions pysr/test/test_startup.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ def test_notebook(self):
self.skipTest("Julia version too old")
if platform.system() == "Windows":
self.skipTest("Notebook test incompatible with Windows")
if not os.access(Path(__file__).parent, os.W_OK):
self.skipTest("Read-only file system")

notebook_file = Path(__file__).parent / "test_nb.ipynb"
sanitize_file = Path(__file__).parent / "nb_sanitize.cfg"
Expand Down
Loading