Output channels from TOMTE workflow for integration in local Nextflow-pipeline

[Jakob37]

Description of feature

This is perhaps more of a question than a request.

We have discussed how to integrate Tomte locally, and landed in that we will need a post-processing tool to fit it into our local systems. For this purpose, we built a small Nextflow pipeline "Nisse" (https://github.com/SMD-Bioinformatics-Lund/Nisse) designed to work with the Tomte output.

Next question is how to best integrate this locally.

I am thinking that the most clean way would be to execute Tomte as a part of Nisse, as a subworkflow. Then we would have a single entitity we can run locally, and it would allow Nextflow to manage resources efficiently, parallellize parts and so on.

The alternative would be to run Tomte as a subworkflow and take the published results as input. Or to run them as two separate processes. I think full integration would be the cleanest.

I have played around a bit with this, and it works fine to execute Tomte from within Nisse.

The final puzzle piece would be a way to access the Tomte results through channels, similarly to how is done for the multiqc_report:

    MULTIQC (
        ch_multiqc_files.collect(),
        ch_multiqc_config.toList(),
        ch_multiqc_custom_config.toList(),
        ch_multiqc_logo.toList(),
        [],
        []
    )

    emit:multiqc_report = MULTIQC.out.report.toList() // channel: /path/to/multiqc_report.html
    versions       = ch_versions                 // channel: [ path(versions.yml) ]
}

Would this make sense to add more output channels here, or is there some principal issue I am missing here? I would be happy to PR!

[jemten]

Nisse looks cool. There are some modules there that we definitely want for tomte as well

[Jakob37]

Nisse looks cool. There are some modules there that we definitely want for tomte as well

Yes, if there are modules that fit in Tomte I think we should move them over. Nisse is more thought of as a place where we can do the Lund-specific things that does not make sense to place in Tomte.

Do you think it would make sense to add output channels to Tomte? Such that it can be interfaced with directly from other pipelines such as Nisse.

[jemten]

What do you say @Lucpen?
I don't think it will impact tomte much if we also specify output channels

[Lucpen]

No, I don't think so, the output files will be saved anyway and that's what we use, so it could be done on the main pipeline without any issues

[Jakob37]

No, I don't think so, the output files will be saved anyway and that's what we use, so it could be done on the main pipeline without any issues

OK great. I'll discuss this a bit more here locally, but I think this would be the cleanest solution for us. If so I can look into it.