Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Eddie w/ --runmode=qsub & --platform=eddie #136

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
37802c0
refactor: simplify cluster profile names, convert cluster from yaml t…
kelly-sovacool Dec 17, 2024
06f663c
feat: adapt grid engine profile from https://github.com/Snakemake-Pro…
kelly-sovacool Dec 17, 2024
9ab541b
style: fix yaml quotes
kelly-sovacool Dec 17, 2024
d883661
feat: add --platform & -m=runqsub to support eddie
kelly-sovacool Dec 17, 2024
dc80ccc
docs: note new platform & runmode options
kelly-sovacool Dec 17, 2024
8f794b2
docs: note sge config sources
kelly-sovacool Dec 17, 2024
b05f54b
fix: create sing_cache_dir during init
kelly-sovacool Dec 27, 2024
ac602af
chore: snakemake must already be in user path for eddie
kelly-sovacool Dec 27, 2024
1d8670a
fix(eddie): do not specify python version
kelly-sovacool Dec 27, 2024
442b6cb
fix: only load singularity module for eddie
kelly-sovacool Dec 27, 2024
2a8bc5b
docs: add details to warning message for unknown platform
kelly-sovacool Dec 31, 2024
a89b1ff
fix(eddie): add Taka's venv to path
kelly-sovacool Dec 31, 2024
191b11e
fix(runlocal): only check slurm job id on biowulf and frce
kelly-sovacool Jan 2, 2025
ab5ca2e
fix: do not submit batch job; eddie workers do not have qsub
kelly-sovacool Jan 9, 2025
8de9454
feat: add back custom submit & jobscripts from sge profile
kelly-sovacool Jan 17, 2025
e0778f1
chore: make scripts executable
kelly-sovacool Jan 21, 2025
1740bc2
fix: use qsub -V & mem_mib
kelly-sovacool Jan 21, 2025
440cee9
fix: set singularity cachedir & tmpdir in submission script
kelly-sovacool Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
## CHARLIE development version

- Support Eddie (The University of Edinburgh's HPC cluster) with `--platform=eddie` and `--runmode=qsub`. (#136, @kelly-sovacool)

## CHARLIE 0.11.1

- CHARLIE was falsely throwing a file permissions error for tempdir values containing bash variables. (#118, @kelly-sovacool)
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,14 +164,16 @@ Required Arguments:
2. RUNMODE : [Type: String] Valid options:
* init : initialize workdir
* dryrun : dry run snakemake to generate DAG
* run : run with slurm
* run : run by submitting the job with slurm
* qsub : run by submitting the job with qsub
* runlocal : run without submitting to sbatch
ADVANCED RUNMODES (use with caution!!)
* unlock : unlock WORKDIR if locked by snakemake NEVER UNLOCK WORKDIR WHERE PIPELINE IS CURRENTLY RUNNING!
* reconfig : recreate config file in WORKDIR (debugging option) EDITS TO config.yaml WILL BE LOST!
* reset : DELETE workdir dir and re-init it (debugging option) EDITS TO ALL FILES IN WORKDIR WILL BE LOST!
* printbinds: print singularity binds (paths)
* local : same as runlocal
* slurm : same as run (run with slurm)

Optional Arguments:

Expand All @@ -181,6 +183,7 @@ Optional Arguments:
--viruses|-v : supply comma-separated list of viruses at command line (--runmode=init only)
--manifest|-s : absolute path to samples.tsv. This will be copied to output folder (--runmode=init only)
--changegrp|-z : change group to "Ziegelbauer_lab" before running anything. Biowulf-only. Useful for correctly setting permissions.
--platform : set the HPC platform (biowulf, fnlcr, eddie). If not set, CHARLIE will try to detect the platform with `scontrol`.
--help|-h : print this help


Expand Down
Loading
Loading