Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue when rerunning the pipeline for MACS2 narrow peak #402

Open
SuhasSrinivasan opened this issue Jan 12, 2025 · 3 comments
Open

Issue when rerunning the pipeline for MACS2 narrow peak #402

SuhasSrinivasan opened this issue Jan 12, 2025 · 3 comments
Labels
bug Something isn't working

Comments

@SuhasSrinivasan
Copy link

SuhasSrinivasan commented Jan 12, 2025 •
edited
Loading

Description of the bug

The pipeline completed in default broad peak mode, and only required rerunning the MACS2 task in narrow peak mode.
But the process resumes from BWA alignment task which is unexpected and consumes a lot of time.

Command used and terminal output

$ nextflow run nf-core/atacseq -profile docker -resume -with-trace --narrow_peak --read_length 150 --input ./exp2_sample_sheet.csv --outdir ./results --fasta ./results/genome/Homo_sapiens.GRCh38.dna.primary_assembly.fa --gtf ./results/genome/Homo_sapiens.GRCh38.111._ensembl.gtf --bwa_index ./results/genome/index/bwa/

[5c/fa411f] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:GTF2BED (Homo_sapiens.GRCh38.111._ensembl.gtf)                               [100%] 1 of 1 ✔
[14/5f793a] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:TSS_EXTRACT                                                                  [100%] 1 of 1 ✔
[78/677e42] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (Homo_sapiens.GRCh38.dna.primary_assembly.fa)           [100%] 1 of 1 ✔
[0d/b0fa56] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:GET_AUTOSOMES (Homo_sapiens.GRCh38.dna.primary_assembly.fa.fai)              [100%] 1 of 1 ✔
[b5/8ee142] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:GENOME_BLACKLIST_REGIONS (Homo_sapiens.GRCh38.dna.primary_assembly.fa.sizes) [100%] 1 of 1 ✔
[49/aeb1f4] process > NFCORE_ATACSEQ:ATACSEQ:PREPARE_GENOME:KHMER_UNIQUEKMERS (Homo_sapiens.GRCh38.dna.primary_assembly.fa)              [100%] 1 of 1 ✔
[c9/7a5968] process > NFCORE_ATACSEQ:ATACSEQ:INPUT_CHECK:SAMPLESHEET_CHECK (exp2_sample_sheet.csv)                                       [100%] 1 of 1, cached: 1 ✔
[61/e63750] process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:FASTQC (D6_MLN_REP2_T1)                                    [100%] 12 of 12, cached: 12 ✔
[e7/3a81d5] process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE (D3_DMSO_REP2_T1)                               [100%] 12 of 12, cached: 12 ✔
[11/6ac85d] process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BWA_MEM (D6_MLN_REP2_T1)                                                    [  0%] 0 of 12
[-        ] process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_SORT                                       -
[-        ] process > NFCORE_ATACSEQ:ATACSEQ:FASTQ_ALIGN_BWA:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_INDEX                                      -
.
.
.
Other tasks follow

Relevant files

No response

System information

This seems as a better order to list the system information, from lowest to highest level.

Version of nf-core/atacseq: 2.1.2
Nextflow version: v24.10.3
Container engine: Docker 27.4.1
OS: Ubuntu 22.04.5 LTS x86_64
Executor: Local
Hardware: Desktop
@SuhasSrinivasan SuhasSrinivasan added the bug Something isn't working label Jan 12, 2025
@SuhasSrinivasan
Copy link
Author

Final pipeline status, only 16.3% of cached output was used, i.e., from before alignment tasks.

[4c/cb0d8d] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BAM_TO_BIGWIG:BEDTOOLS_GENOMECOV (D3_MLN_REP1)                                        [100%] 12 of 12 ✔
[20/e2b712] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BAM_TO_BIGWIG:UCSC_BEDGRAPHTOBIGWIG (D3_MLN_REP1)                                     [100%] 12 of 12 ✔
[27/eae515] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BIGWIG_PLOT_DEEPTOOLS:DEEPTOOLS_COMPUTEMATRIX_SCALE_REGIONS (D3_MLN_REP1)             [100%] 12 of 12 ✔
[bf/04594c] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BIGWIG_PLOT_DEEPTOOLS:DEEPTOOLS_COMPUTEMATRIX_REFERENCE_POINT (D3_MLN_REP1)           [100%] 12 of 12 ✔
[aa/42a6ed] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BIGWIG_PLOT_DEEPTOOLS:DEEPTOOLS_PLOTPROFILE (D3_MLN_REP1)                             [100%] 12 of 12 ✔
[20/218435] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_BIGWIG_PLOT_DEEPTOOLS:DEEPTOOLS_PLOTHEATMAP (D3_MLN_REP1)                             [100%] 12 of 12 ✔
[c3/b97693] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_DEEPTOOLS_PLOTFINGERPRINT (D3_MLN_REP1)                                               [100%] 12 of 12 ✔
[90/2412d9] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:MACS2_CALLPEAK (D3_MLN_REP1)                                      [100%] 12 of 12 ✔
[b9/2223db] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:FRIP_SCORE (D3_MLN_REP1)                                          [100%] 12 of 12 ✔
[39/9c05b5] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:MULTIQC_CUSTOM_PEAKS (D3_MLN_REP1)                                [100%] 12 of 12 ✔
[71/ec3c8e] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:HOMER_ANNOTATEPEAKS (D3_MLN_REP1)                                 [100%] 12 of 12 ✔
[90/723f0d] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:PLOT_MACS2_QC                                                     [100%] 1 of 1 ✔
[e1/f85254] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CALL_ANNOTATE_PEAKS:PLOT_HOMER_ANNOTATEPEAKS                                          [100%] 1 of 1 ✔
[33/522876] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CONSENSUS_PEAKS:MACS2_CONSENSUS (consensus_peaks)                                     [100%] 1 of 1 ✔
[83/00a388] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CONSENSUS_PEAKS:HOMER_ANNOTATEPEAKS (consensus_peaks)                                 [100%] 1 of 1 ✔
[b7/36a759] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CONSENSUS_PEAKS:SUBREAD_FEATURECOUNTS (consensus_peaks)                               [100%] 1 of 1 ✔
[a9/b4d240] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_CONSENSUS_PEAKS:DESEQ2_QC (consensus_peaks)                                           [100%] 1 of 1 ✔
[42/47bc38] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_ATAQV_ATAQV (D3_MLN_REP1)                                                             [100%] 12 of 12 ✔
[b2/d39a6a] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_LIBRARY_ATAQV_MKARV                                                                           [100%] 1 of 1 ✔
[4f/0cd467] process > NFCORE_ATACSEQ:ATACSEQ:PICARD_MERGESAMFILES_REPLICATE (D6_MLN)                                                              [100%] 6 of 6 ✔
[c8/a7e384] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_MARKDUPLICATES_PICARD:PICARD_MARKDUPLICATES (D6_MLN)                                [100%] 6 of 6 ✔
[2e/0396f1] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_MARKDUPLICATES_PICARD:SAMTOOLS_INDEX (D6_MLN)                                       [100%] 6 of 6 ✔
[55/f9707b] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (D3_MLN)                    [100%] 6 of 6 ✔
[1e/8f0ce3] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT (D0_MLN)                 [100%] 6 of 6 ✔
[36/cdbaa1] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_MARKDUPLICATES_PICARD:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS (D6_DMSO)                [100%] 6 of 6 ✔
[25/4241c6] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_BAM_TO_BIGWIG:BEDTOOLS_GENOMECOV (D6_MLN)                                           [100%] 7 of 7, failed: 1, retries: 1 ✔
[f9/8c75f0] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_BAM_TO_BIGWIG:UCSC_BEDGRAPHTOBIGWIG (D6_MLN)                                        [100%] 6 of 6 ✔
[ea/af0978] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:MACS2_CALLPEAK (D6_MLN)                                         [100%] 6 of 6 ✔
[a3/1cf52a] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:FRIP_SCORE (D6_MLN)                                             [100%] 6 of 6 ✔
[d3/dd1172] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:MULTIQC_CUSTOM_PEAKS (D6_MLN)                                   [100%] 6 of 6 ✔
[f3/f2b86a] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:HOMER_ANNOTATEPEAKS (D6_MLN)                                    [100%] 6 of 6 ✔
[f1/32a0b0] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:PLOT_MACS2_QC                                                   [100%] 1 of 1 ✔
[ec/a1af67] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CALL_ANNOTATE_PEAKS:PLOT_HOMER_ANNOTATEPEAKS                                        [100%] 1 of 1 ✔
[e9/e12cb2] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CONSENSUS_PEAKS:MACS2_CONSENSUS (consensus_peaks)                                   [100%] 1 of 1 ✔
[80/e2638e] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CONSENSUS_PEAKS:HOMER_ANNOTATEPEAKS (consensus_peaks)                               [100%] 1 of 1 ✔
[e8/a6d9b2] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CONSENSUS_PEAKS:SUBREAD_FEATURECOUNTS (consensus_peaks)                             [100%] 1 of 1 ✔
[dd/8411a6] process > NFCORE_ATACSEQ:ATACSEQ:MERGED_REPLICATE_CONSENSUS_PEAKS:DESEQ2_QC (consensus_peaks)                                         [100%] 1 of 1 ✔
[a6/41531e] process > NFCORE_ATACSEQ:ATACSEQ:IGV                                                                                                  [100%] 1 of 1 ✔
[d9/ba9065] process > NFCORE_ATACSEQ:ATACSEQ:CUSTOM_DUMPSOFTWAREVERSIONS (1)                                                                      [100%] 1 of 1 ✔
[29/fc1f9f] process > NFCORE_ATACSEQ:ATACSEQ:MULTIQC (1)                                                                                          [100%] 1 of 1 ✔
-[nf-core/atacseq] Pipeline completed successfully-
WARN: Undocumented setting `docker.userEmulation` is not supported any more - please remove it from your config
Completed at: 12-Jan-2025 07:14:40
Duration    : 12h 2m 35s
CPU hours   : 205.5 (16.3% cached, 0.3% failed)
Succeeded   : 490
Cached      : 25
Failed      : 1

@JoseEspinosa
Copy link
Member

That is strange, never saw such behavior, are you sure that nothing has broken the Nextflow cache, see this blog post for more details.

@SuhasSrinivasan
Copy link
Author

Thank you for reviewing, there were no user made changes with the work directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JoseEspinosa @SuhasSrinivasan