Skip to content

Latest commit

 

History

History
68 lines (43 loc) · 1.49 KB

README.md

File metadata and controls

68 lines (43 loc) · 1.49 KB

https://www.singularity-hub.org/static/img/hosted-singularity--hub-%23e32929.svg

[ebb &] Flow

Experiment pipeline

TODO

Tools

Deploy code to singularity-hub

Include it in .travis.yml files for automatic deployment after successful tests.

flow-deploy ...

Submit jobs on clusters

Using SmartDispatch as a backend

flow-submit [SCHEDULER OPTIONS] [COMMAND] ...

Can be combined with flow-execute

flow-submit [SCHEDULER OPTIONS] flow-execute shub://bouthilx/my-experiment:version [COMMAND] ...

Execute jobs using singularity containers

flow-execute shub://bouthilx/my-experiment:version [COMMAND] ...

Install

TODO

Configuration

In .bashrc or similar,

SINGULARITY_DIR=/some/path/to/singularity
export SREGISTRY_STORAGE=$SINGULARITY_DIR
export SINGULARITY_CACHEDIR=$SINGULARITY_DIR/cache
export SREGISTRY_DATABASE=$SINGULARITY_DIR
export SREGISTRY_NVIDIA_TOKEN=<SECRET>

export DATA_FOLDER=/some/path/to/data

If using mongodb secured with a certificate,

export CERTIFICATE_FOLDER=/some/path/to/certs

TODO

Containers

https://www.singularity-hub.org/static/img/hosted-singularity--hub-%23e32929.svg

Pytorch 0.3.1