Nextflow
Workflow language for scalable and reproducible computational pipelines — write once, run anywhere (local, HPC, AWS, GCP, Azure).
When to Use This Skill
- Writing or debugging Nextflow DSL2 pipelines (
.nf files)
- Composing processes into workflows with channel dataflow
- Configuring executors (SLURM, LSF, AWS Batch, Google Batch)
- Managing containers (Docker, Singularity/Apptainer, Conda) for reproducibility
- Building chemistry/bioinformatics pipelines (BLAST, aligners, RDKit, ORCA, Gaussian)
- Understanding
-resume / cache behavior
- Modularizing pipelines with
include / module aliases
Quick Start — Minimal DSL2 Pipeline
// main.nf
params.input = 'data/*.sdf'
params.outdir = 'results'
process RUN_ORCA {
publishDir params.outdir, mode: 'copy'
container 'quay.io/biocontainers/orca:5.0.4--h2f1ea3e_0'
input:
path mol
output:
path "*.out"
script:
"""
orca ${mol}.inp > ${mol}.out
"""
}
workflow {
mols = channel.fromPath(params.input)
RUN_ORCA(mols)
}
Run it:
nextflow run main.nf -profile docker -resume
Router — What to Read
| Task |
Reference |
| Processes, channels, input/output qualifiers, script types |
references/core-concepts.md |
| Workflows, named workflows, pipe/and operators, modules, composition |
references/pipeline-patterns.md |
nextflow.config, executors, profiles, HPC/cloud, cache/resume |
references/execution-config.md |
| Docker, Apptainer/Singularity, Conda, Wave, reproducibility |
references/containers-envs.md |
| Channel factories, operators, file handling, remote files |
references/files-channels.md |
| Chemistry/bioinformatics patterns (BLAST, RDKit, ORCA, MD) |
references/chem-bioinformatics.md |
Key Concepts at a Glance
| Concept |
What it is |
process |
Runs a script/command; defines input, output, directives |
workflow |
Composes processes and operators via dataflow channels |
channel |
Asynchronous stream of values connecting processes |
val / path |
Input qualifiers — val for data, path for staged files |
publishDir |
Copies task output to a user-visible results directory |
executor |
Where tasks run: local, slurm, awsbatch, google-batch… |
-resume |
Reuses cached task results; skips unchanged tasks |
module |
Reusable .nf file included with include { X } from './module' |
Installation
# Requires Java 11+
curl -s https://get.nextflow.io | bash
./nextflow self-update # upgrade to latest
nextflow -version # verify
# Enable DSL2 strict parser (recommended for new pipelines)
export NXF_SYNTAX_PARSER=v2
Related Skills
rdkit — Molecular preprocessing before pipeline ingestiondeepchem — ML models on molecular datasets (can be wrapped in NF processes)cheminformatics — SMILES, molecular file formats (SDF, MOL2, XYZ)
1---2name: nextflow3description: Use when writing, debugging, or optimizing Nextflow pipelines for computational chemistry, bioinformatics, or HPC workflows. Covers DSL2 syntax, process/channel/workflow composition, configuration, containers, and execution on HPC/cloud.4---56# Nextflow78Workflow language for scalable and reproducible computational pipelines — write once, run anywhere (local, HPC, AWS, GCP, Azure).910## When to Use This Skill1112- Writing or debugging Nextflow DSL2 pipelines (`.nf` files)13- Composing processes into workflows with channel dataflow14- Configuring executors (SLURM, LSF, AWS Batch, Google Batch)15- Managing containers (Docker, Singularity/Apptainer, Conda) for reproducibility16- Building chemistry/bioinformatics pipelines (BLAST, aligners, RDKit, ORCA, Gaussian)17- Understanding `-resume` / cache behavior18- Modularizing pipelines with `include` / module aliases1920## Quick Start — Minimal DSL2 Pipeline2122```nextflow23// main.nf24params.input = 'data/*.sdf'25params.outdir = 'results'2627process RUN_ORCA {28 publishDir params.outdir, mode: 'copy'29 container 'quay.io/biocontainers/orca:5.0.4--h2f1ea3e_0'3031 input:32 path mol3334 output:35 path "*.out"3637 script:38 """39 orca ${mol}.inp > ${mol}.out40 """41}4243workflow {44 mols = channel.fromPath(params.input)45 RUN_ORCA(mols)46}47```4849Run it:50```bash51nextflow run main.nf -profile docker -resume52```5354## Router — What to Read5556| Task | Reference |57|------|-----------|58| Processes, channels, input/output qualifiers, script types | `references/core-concepts.md` |59| Workflows, named workflows, pipe/and operators, modules, composition | `references/pipeline-patterns.md` |60| `nextflow.config`, executors, profiles, HPC/cloud, cache/resume | `references/execution-config.md` |61| Docker, Apptainer/Singularity, Conda, Wave, reproducibility | `references/containers-envs.md` |62| Channel factories, operators, file handling, remote files | `references/files-channels.md` |63| Chemistry/bioinformatics patterns (BLAST, RDKit, ORCA, MD) | `references/chem-bioinformatics.md` |6465## Key Concepts at a Glance6667| Concept | What it is |68|---------|------------|69| `process` | Runs a script/command; defines `input`, `output`, directives |70| `workflow` | Composes processes and operators via dataflow channels |71| `channel` | Asynchronous stream of values connecting processes |72| `val` / `path` | Input qualifiers — `val` for data, `path` for staged files |73| `publishDir` | Copies task output to a user-visible results directory |74| `executor` | Where tasks run: local, slurm, awsbatch, google-batch… |75| `-resume` | Reuses cached task results; skips unchanged tasks |76| `module` | Reusable `.nf` file included with `include { X } from './module'` |7778## Installation7980```bash81# Requires Java 11+82curl -s https://get.nextflow.io | bash83./nextflow self-update # upgrade to latest84nextflow -version # verify8586# Enable DSL2 strict parser (recommended for new pipelines)87export NXF_SYNTAX_PARSER=v288```8990## Related Skills9192- `rdkit` — Molecular preprocessing before pipeline ingestion93- `deepchem` — ML models on molecular datasets (can be wrapped in NF processes)94- `cheminformatics` — SMILES, molecular file formats (SDF, MOL2, XYZ)
Kdevos12