From 37dc09c42438fd8b5a2be773647b7f1c61425322 Mon Sep 17 00:00:00 2001 From: Luca Visentin Date: Tue, 16 Apr 2024 11:06:04 +0200 Subject: [PATCH 1/2] feat: start using callouts in the book --- .github/workflows/deploy_book.yml | 11 ++++++----- docs/book.toml | 2 ++ docs/src/introduction.md | 1 - docs/src/kerblam.toml.md | 1 + docs/src/philosophy.md | 4 ++++ docs/src/quickstart.md | 3 +++ docs/src/tutorial/fetch_data.md | 18 ++++++++++-------- docs/src/tutorial/new.md | 1 + docs/src/tutorial/package_data.md | 1 + docs/src/tutorial/run.md | 3 +++ docs/src/tutorial/run_containers.md | 1 + 11 files changed, 32 insertions(+), 14 deletions(-) delete mode 100644 docs/src/introduction.md diff --git a/.github/workflows/deploy_book.yml b/.github/workflows/deploy_book.yml index 3d7229a..ed0dc60 100644 --- a/.github/workflows/deploy_book.yml +++ b/.github/workflows/deploy_book.yml @@ -16,13 +16,14 @@ jobs: - uses: actions/checkout@v4 with: fetch-depth: 0 + - uses: dtolnay/rust-toolchain@stable + - uses: Swatinem/rust-cache@v2 - name: Install latest mdbook run: | - tag=$(curl 'https://api.github.com/repos/rust-lang/mdbook/releases/latest' | jq -r '.tag_name') - url="https://github.com/rust-lang/mdbook/releases/download/${tag}/mdbook-${tag}-x86_64-unknown-linux-gnu.tar.gz" - mkdir mdbook - curl -sSL $url | tar -xz --directory=./mdbook - echo `pwd`/mdbook >> $GITHUB_PATH + cargo install mdbook + - name: Install preprocessors + run: | + cargo install mdbook-callouts - name: Build Book run: | # This assumes your book is in the root of your repository. diff --git a/docs/book.toml b/docs/book.toml index c6e35ee..b7490b6 100644 --- a/docs/book.toml +++ b/docs/book.toml @@ -8,3 +8,5 @@ title = "Kerblam Docs" [output.html] default-theme="dark" preferred-dark-theme = "ayu" + +[preprocessor.callouts] diff --git a/docs/src/introduction.md b/docs/src/introduction.md deleted file mode 100644 index e10b99d..0000000 --- a/docs/src/introduction.md +++ /dev/null @@ -1 +0,0 @@ -# Introduction diff --git a/docs/src/kerblam.toml.md b/docs/src/kerblam.toml.md index 5ed4f5f..f811354 100644 --- a/docs/src/kerblam.toml.md +++ b/docs/src/kerblam.toml.md @@ -4,6 +4,7 @@ The `kerblam.toml` file is the control center of kerblam! All of its configuration is found there. Here is what fields are available, and what they do. +> [!WARNING] > Extra fields not found here are silently ignored. > This means that you must be careful of typos! diff --git a/docs/src/philosophy.md b/docs/src/philosophy.md index 701eae5..aa90b5c 100644 --- a/docs/src/philosophy.md +++ b/docs/src/philosophy.md @@ -48,6 +48,7 @@ you have a `pyproject.toml`/`setup.py`, a module directory[^pyweird]... Similarly, when you work on a Rust project, you use `cargo`, and therefore have a `cargo.toml` file, a `/src` directory... +> [!NOTE] > The topic of structuring the code itself is even deeper, with different ways to > think of your coding problem: object oriented vs functional vs procedural, > monolithic vs microservices, etcetera, but it's out of the scope of this piece. @@ -60,6 +61,7 @@ A ***project layout system (PLS)*** is a pre-established way to layout these fil Kerblam! is a tool that can help you with general tasks if you follow the Kerblam! project layout system. +> [!NOTE] > There are also project *management* systems, that are tasked with managing > what has to be done while writing code. > They are not the subject of this piece, however. @@ -95,6 +97,7 @@ binary files. [^proud]: I'm very proud of this pun. Please don't take it from me. +> [!QUOTE] > I assume that the reader knows how vital version control is when writing software. > In case that you do not, I want to briefly outline why you'd want to use a version > control system in your work: @@ -208,6 +211,7 @@ Other than input, output, temporary and intermediate data, kerblam! also conside little repercussion (i.e. you just re-download it or re-run the pipeline to obtain it again. +> [!NOTE] > Practically, data can be input/output/temp/intermediate, either fragile > or precious and either local or remote. diff --git a/docs/src/quickstart.md b/docs/src/quickstart.md index d277525..5ff5d0d 100644 --- a/docs/src/quickstart.md +++ b/docs/src/quickstart.md @@ -23,6 +23,7 @@ These folders, relative to where the `kerblam.toml` file is, are: They have to be written as if they were saved in `./`. - `./src/dockerfiles/`: Container build scripts should be saved here. +> [!TIP] > Any sub-folder of one of these specific folders (with the exception of > `src/pipes` and `src/dockerfiles`) contains the same type of files as the > parent directory. For instance, `data/in/fastq` is treated as if it contains @@ -33,6 +34,7 @@ This is mostly done for compatibility reasons with non-kerblam! projects. New projects that wish to use Kerblam! are strongly encouraged to follow the standard folder structure, however. +> [!IMPORTANT] > The rest of these docs are written as if you are using the standard > folder structure. If you are not, don't worry! All Kerblam! commands respect > your choices in the `kerblam.toml` file. @@ -66,5 +68,6 @@ The rest of this tutorial walks you through every feature. I hope you enjoy Kerblam! and that it makes your projects easier to understand, run and reproduce! +> [!INFO] > If you like Kerblam!, please consider [leaving a star on Github](https://github.com/MrHedmad/kerblam/stargazers). > Thank you for supporting Kerblam! diff --git a/docs/src/tutorial/fetch_data.md b/docs/src/tutorial/fetch_data.md index 2f29c68..bcd32d0 100644 --- a/docs/src/tutorial/fetch_data.md +++ b/docs/src/tutorial/fetch_data.md @@ -17,11 +17,13 @@ and therefore locally expendable for the sake of saving disk size** (see the You can specify any number of URLs and file names in `[data.remote]`, one for each file that you wish to be downloaded. -The download directory for all fetched data is your input directory, -so if you specify `some/nested/dir/file.txt`, kerblam! will save the file in -`./data/in/some/nested/dir/file.txt`. -This also means that if you write an absolute path (e.g. `/some_file.txt`), -Kerblam! will treat the path as it should treat it - by making `some_file.txt` -in the root of the filesystem (and most likely failing to do so). -It will, however, warn you before acting that it is about to do something -potentially unwanted, giving you the chance to abort. +> [!DANGER] +> The download directory for all fetched data is your input directory, +> so if you specify `some/nested/dir/file.txt`, kerblam! will save the file in +> `./data/in/some/nested/dir/file.txt`. +> This also means that if you write an absolute path (e.g. `/some_file.txt`), +> Kerblam! will treat the path as it should treat it - by making `some_file.txt` +> in the root of the filesystem (and most likely failing to do so). +> +> Kerblam! will, however, warn you before acting, telling you that it is about +> to do something potentially unwanted, and giving you the chance to abort. diff --git a/docs/src/tutorial/new.md b/docs/src/tutorial/new.md index 063ae2b..10571e0 100644 --- a/docs/src/tutorial/new.md +++ b/docs/src/tutorial/new.md @@ -20,6 +20,7 @@ Say 'yes' to all of these questions to follow along. Kerblam! will then: files (if you opted to use Python), - and setup the `.gitignore` file with appropriate ignores. +> [!IMPORTANT] > Kerblam! will **NOT** do an `Initial commit` for you! > You still need to do that manually once you've finished setting up. diff --git a/docs/src/tutorial/package_data.md b/docs/src/tutorial/package_data.md index 3bc2c22..2cf21d1 100644 --- a/docs/src/tutorial/package_data.md +++ b/docs/src/tutorial/package_data.md @@ -4,6 +4,7 @@ You can `tar -czvf exported_data.tar.gz ./data/` and send your whole data folder but you might want to only pick the output and non-remotely available inputs, and leave re-downloading the (potentially bulky) remote data to your colleague. +> [!FAILURE] > It is widely known that remembering `tar` commands is impossible. If you run `kerblam data pack` you can do just that. diff --git a/docs/src/tutorial/run.md b/docs/src/tutorial/run.md index 6fd96f9..556fca5 100644 --- a/docs/src/tutorial/run.md +++ b/docs/src/tutorial/run.md @@ -58,6 +58,7 @@ You can then run the same makefile with the new data with: kerblam run process_csv --profile alternate ``` +> [!IMPORTANT] > Paths under every profile section are relative to the input data directory, > by default `data/in`. @@ -71,6 +72,7 @@ Under the hood, Kerblam! will: This effectively causes the makefile to run with different input data. +> [!WARNING] > Careful that the *output* data will (most likely) be saved as the > same file names as a "normal" run! > @@ -101,6 +103,7 @@ Kerblam! tries its best to cleanup after itself (e.g. undo profiles, delete temporary files, etc...) when you use `kerblam run`, even if the pipe fails, and even if you kill your pipe with `CTRL-C`. +> [!TIP] > If your pipeline is unresponsive to a `CTRL-C`, pressing it twice (two > `SIGTERM` signals in a row) will kill Kerblam! instead, leaving the child > process to be cleaned up by the OS and the (eventual) profile not cleaned up. diff --git a/docs/src/tutorial/run_containers.md b/docs/src/tutorial/run_containers.md index b358480..5d5884d 100644 --- a/docs/src/tutorial/run_containers.md +++ b/docs/src/tutorial/run_containers.md @@ -123,6 +123,7 @@ workdir = "/app" ``` In this way, Kerblam! will run the containers with the proper paths. +> [!IMPORTANT] > **This option applies to *ALL* containers managed by Kerblam!** > > There is currently no way to configure a different working directory for every From 62ae866d35da972372aaf6882786c7f0c37843dc Mon Sep 17 00:00:00 2001 From: Luca Visentin Date: Tue, 16 Apr 2024 11:08:48 +0200 Subject: [PATCH 2/2] docs: add callout to block --- docs/src/tutorial/run.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/src/tutorial/run.md b/docs/src/tutorial/run.md index abf5541..404d8e3 100644 --- a/docs/src/tutorial/run.md +++ b/docs/src/tutorial/run.md @@ -117,6 +117,7 @@ In this way, you can detect from inside the pipeline if you are in a profile or This is useful if you want to keep the outputs of different profiles separate, for instance. +> [!TIP] > Profiles work on directories too! If you specify a directory as a target > of a profile, Kerblam! will move the whole directory to the new location.