Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pangolin v4 #329

Merged
merged 8 commits into from
Apr 2, 2022
Merged

Add pangolin v4 #329

merged 8 commits into from
Apr 2, 2022

Conversation

kapsakcj
Copy link
Collaborator

@kapsakcj kapsakcj commented Apr 1, 2022 •
edited
Loading

  • This comment contains a description of what is in the pull request.
    This PR adds:
  • dockerfile for Pangolin v4.0
  • GH actions workflow for testing
  • GH actions workflow for deployment
  • updated README.md for basic pangolin usage

Versions pinned in Dockerfile, used to create the conda environment

ARG PANGOLIN_VER="v4.0"
ARG PANGOLIN_DATA_VER="v1.2.133.2"
ARG SCORPIO_VER="v0.3.16"
ARG CONSTELLATIONS_VER="v0.1.4"
ARG USHER_VER="0.5.3"

Though this is not accurately reflected with pangolin --all-versions. It's a tad bit confusing but more info can be found here. I don't want to wait for a fix to be put in place so I'd like to deploy this asap and warn users about the discrepancy.

(pangolin) root@707c2228bd71:/data# pangolin --all-versions
****
Pangolin running in usher mode.
****
pangolin: 4.0
pangolin-data: 1.2.133
constellations: v0.1.4
scorpio: 0.3.16
pangolin-assignment: v1.2.133
  • Build your own docker image using a Dockerfile
    • Directory structure should be name of the tool in lower case with special characters removed with a subdirectory of the version number (i.e. spades/3.12.0/Dockerfile)
    • Includes the recommended LABELS
  • (Optional) Dockerfile is built with best practices and has been approved by a linter (such as https://hadolint.github.io/hadolint/)
  • Edit main README.md
  • Ensure tool is listed in Program_Licenses.md
  • Create a simple container-specific README.md in the same directory as the Dockerfile (i.e. spades/3.12.0/README.md)
  • Update GitHub actions workflow if needed
  • Any files required for building are located in the same directory as the Dockerfile (i.e. spades/3.12.0/my_spades_tests.sh)
  • Have successfully run the workflow "Test image" in your forked repository

@erinyoung
Copy link
Contributor

I'm so excited for this!

@kapsakcj
Update README.md
8b7e777 
added another list for pangolin v4.0 and pangolin-data versions
@kapsakcj
Copy link
Collaborator Author

kapsakcj commented Apr 1, 2022

Hmm looks like the older GH actions workflow kicked off in addition to the one I just added.

Guessing it's due to this line with a *: https://github.com/kapsakcj/docker-builds/blob/8b7e7770430cfb7b45e80ac556e40604715f600d/.github/workflows/test-pangolin3.1.20-slim.yml#L7

@kapsakcj
Copy link
Collaborator Author

kapsakcj commented Apr 2, 2022

I have a feeling that this error is related to RAM usage while testing pangolearn/fast mode. The tests work perfect on my VM with 8cpus and 30GB of RAM.

The 7GB on GH actions runners may not be enough https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources

cov-lineages/pangolin#395

@kapsakcj
Copy link
Collaborator Author

kapsakcj commented Apr 2, 2022

Yep, its the RAM usage.

When I ran $ docker build -m 7000000000 --target test -t kapsakcj/pangolin:4.0 pangolin/4.0/ which limits RAM to 7GB, the test failed with the same error when attempting pangolearn analysis-mode.

Going to remove pangolearn testing from the test layer

@erinyoung
Copy link
Contributor

Looks like the tests all worked. Can we merge and deploy this?

I tested it locally with some fastas, and it worked for me.

@erinyoung erinyoung self-requested a review April 2, 2022 01:19
erinyoung
erinyoung reviewed Apr 2, 2022
View reviewed changes
README.md
@@ -92,7 +92,7 @@ To learn more about the docker pull rate limits and the open source software pro
| [NanoPlot](https://hub.docker.com/r/staphb/nanoplot) <br/> [![docker pulls](https://img.shields.io/docker/pulls/staphb/nanoplot.svg?style=popout)](https://hub.docker.com/r/staphb/nanoplot) | <ul><li>1.27.0</li><li>1.29.0</li><li>1.30.1</li><li>1.32.0</li><li>1.33.0</li></ul> | https://github.com/wdecoster/NanoPlot |
| [NCBI AMRFinderPlus](https://hub.docker.com/r/staphb/ncbi-amrfinderplus) <br/> [![docker pulls](https://img.shields.io/docker/pulls/staphb/ncbi-amrfinderplus.svg?style=popout)](https://hub.docker.com/r/staphb/ncbi-amrfinderplus) | <ul><li>3.1.1b</li><li>3.8.4</li><li>3.8.28</li><li>3.9.3</li><li>3.9.8</li><li>3.10.1</li><li>3.10.5</li><li>3.10.16</li><li>3.10.20</li></ul> | [https://github.com/ncbi/amr](https://github.com/ncbi/amr) |
| [OrthoFinder](https://hub.docker.com/r/staphb/OrthoFinder) <br/> [![docker pulls](https://img.shields.io/docker/pulls/staphb/orthofinder.svg?style=popout)](https://hub.docker.com/r/staphb/orthofinder) | <ul><li>2.17</li></ul> | https://github.com/davidemms/OrthoFinder |
| [Pangolin](https://hub.docker.com/r/staphb/pangolin) <br/> [![docker pulls](https://img.shields.io/docker/pulls/staphb/pangolin.svg?style=popout)](https://hub.docker.com/r/staphb/pangolin)<br/> ![Pangolin](https://github.com/StaPH-B/docker-builds/actions/workflows/pangolin.yml/badge.svg) | **Pangolin version & pangoLEARN data release date** <ul><li>1.1.14</li><li>2.0.4 & 2020-07-20</li><li>2.0.5 & 2020-07-20</li><li>2.1.1 & 2020-12-17</li><li>2.1.3 & 2020-12-17</li><li>2.1.6 & 2021-01-06</li><li>2.1.7 & 2021-01-11</li><li>2.1.7 & 2021-01-20</li><li>2.1.8 & 2021-01-22</li><li>2.1.10 & 2021-02-01</li><li>2.1.11 & 2021-02-01</li><li>2.1.11 & 2021-02-05</li><li>2.2.1 & 2021-02-06</li><li>2.2.2 & 2021-02-06</li><li>2.2.2 & 2021-02-11</li><li>2.2.2 & 2021-02-12</li><li>2.3.0 & 2021-02-12</li><li>2.3.0 & 2021-02-18</li><li>2.3.0 & 2021-02-21</li><li>2.3.2 & 2021-02-21</li><li>2.3.3 & 2021-03-16</li><li>2.3.4 & 2021-03-16</li><li>2.3.5 & 2021-03-16</li><li>2.3.6 & 2021-03-16</li><li>2.3.6 & 2021-03-29</li><li>2.3.8 & 2021-04-01</li><li>2.3.8 & 2021-04-14</li><li>2.3.8 & 2021-04-21</li><li>2.3.8 & 2021-04-23</li><li>2.4 & 2021-04-28</li><li>2.4.1 & 2021-04-28</li><li>2.4.2 & 2021-04-28</li><li>2.4.2 & 2021-05-10</li><li>2.4.2 & 2021-05-11</li><li>2.4.2 & 2021-05-19</li><li>3.0.5 & 2021-06-05</li><li>3.1.3 & 2021-06-15</li><li>3.1.5 & 2021-06-15</li><li>3.1.5 & 2021-07-07-2</li><li>3.1.7 & 2021-07-09</li><li>3.1.8 & 2021-07-28</li><li>3.1.10 & 2021-07-28</li><li>3.1.11 & 2021-08-09</li><li>3.1.11 & 2021-08-24</li><li>3.1.11 & 2021-09-17</li><li>3.1.14 & 2021-09-28</li><li>3.1.14 & 2021-10-13</li><li>3.1.16 & 2021-10-18</li><li>3.1.16 & 2021-11-04</li><li>3.1.16 & 2021-11-09</li><li>3.1.16 & 2021-11-18</li><li>3.1.16 & 2021-11-25</li><li>3.1.17 & 2021-11-25</li><li>3.1.17 & 2021-12-06</li><li>3.1.17 & 2022-01-05</li><li>3.1.18 & 2022-01-20</li><li>3.1.19 & 2022-01-20</li><li>3.1.20 & 2022-02-02</li><li>3.1.20 & 2022-02-28</li></ul> | https://github.com/cov-lineages/pangolin<br/>https://github.com/cov-lineages/pangoLEARN<br/>https://github.com/cov-lineages/pango-designation<br/>https://github.com/cov-lineages/scorpio<br/>https://github.com/cov-lineages/constellations<br/>https://github.com/cov-lineages/lineages (archived)<br/>https://github.com/hCoV-2019/pangolin (archived) |
| [Pangolin](https://hub.docker.com/r/staphb/pangolin) <br/> [![docker pulls](https://img.shields.io/docker/pulls/staphb/pangolin.svg?style=popout)](https://hub.docker.com/r/staphb/pangolin)<br/> ![Pangolin](https://github.com/StaPH-B/docker-builds/actions/workflows/pangolin.yml/badge.svg) | **Pangolin version & pangoLEARN data release date** <ul><li>1.1.14</li><li>2.0.4 & 2020-07-20</li><li>2.0.5 & 2020-07-20</li><li>2.1.1 & 2020-12-17</li><li>2.1.3 & 2020-12-17</li><li>2.1.6 & 2021-01-06</li><li>2.1.7 & 2021-01-11</li><li>2.1.7 & 2021-01-20</li><li>2.1.8 & 2021-01-22</li><li>2.1.10 & 2021-02-01</li><li>2.1.11 & 2021-02-01</li><li>2.1.11 & 2021-02-05</li><li>2.2.1 & 2021-02-06</li><li>2.2.2 & 2021-02-06</li><li>2.2.2 & 2021-02-11</li><li>2.2.2 & 2021-02-12</li><li>2.3.0 & 2021-02-12</li><li>2.3.0 & 2021-02-18</li><li>2.3.0 & 2021-02-21</li><li>2.3.2 & 2021-02-21</li><li>2.3.3 & 2021-03-16</li><li>2.3.4 & 2021-03-16</li><li>2.3.5 & 2021-03-16</li><li>2.3.6 & 2021-03-16</li><li>2.3.6 & 2021-03-29</li><li>2.3.8 & 2021-04-01</li><li>2.3.8 & 2021-04-14</li><li>2.3.8 & 2021-04-21</li><li>2.3.8 & 2021-04-23</li><li>2.4 & 2021-04-28</li><li>2.4.1 & 2021-04-28</li><li>2.4.2 & 2021-04-28</li><li>2.4.2 & 2021-05-10</li><li>2.4.2 & 2021-05-11</li><li>2.4.2 & 2021-05-19</li><li>3.0.5 & 2021-06-05</li><li>3.1.3 & 2021-06-15</li><li>3.1.5 & 2021-06-15</li><li>3.1.5 & 2021-07-07-2</li><li>3.1.7 & 2021-07-09</li><li>3.1.8 & 2021-07-28</li><li>3.1.10 & 2021-07-28</li><li>3.1.11 & 2021-08-09</li><li>3.1.11 & 2021-08-24</li><li>3.1.11 & 2021-09-17</li><li>3.1.14 & 2021-09-28</li><li>3.1.14 & 2021-10-13</li><li>3.1.16 & 2021-10-18</li><li>3.1.16 & 2021-11-04</li><li>3.1.16 & 2021-11-09</li><li>3.1.16 & 2021-11-18</li><li>3.1.16 & 2021-11-25</li><li>3.1.17 & 2021-11-25</li><li>3.1.17 & 2021-12-06</li><li>3.1.17 & 2022-01-05</li><li>3.1.18 & 2022-01-20</li><li>3.1.19 & 2022-01-20</li><li>3.1.20 & 2022-02-02</li><li>3.1.20 & 2022-02-28</li></ul>**Pangolin version & pangolin-data version** </br><ul><li>4.0 & 1.2.133</li></ul> | https://github.com/cov-lineages/pangolin<br/>https://github.com/cov-lineages/pangoLEARN<br/>https://github.com/cov-lineages/pango-designation<br/>https://github.com/cov-lineages/scorpio<br/>https://github.com/cov-lineages/constellations<br/>https://github.com/cov-lineages/lineages (archived)<br/>https://github.com/hCoV-2019/pangolin (archived) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting so long

@kapsakcj
Copy link
Collaborator Author

kapsakcj commented Apr 2, 2022

Sure, go for it!

@erinyoung erinyoung merged commit fb08f1c into StaPH-B:master Apr 2, 2022
@kapsakcj kapsakcj deleted the cjk-pangolin-v4 branch April 2, 2022 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erinyoung erinyoung erinyoung approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kapsakcj @erinyoung