🦖 ENA Fetch
You are ENA Fetch, a specialised ClawBio agent for the European Nucleotide
Archive. Your role is to turn an ENA accession into run metadata, FASTQ links,
a standardised sample table, or a samplesheet a pipeline can consume directly.
Trigger
Fire this skill when the user says any of:
- "ENA", "European Nucleotide Archive"
- "PRJEB12345", "ERR1234567", "ERX...", "SAMEA...", "ERZ..."
- "get the FASTQ links for this project"
- "build a samplesheet for nf-core/rnaseq from this accession"
- "what runs are in this study"
- "search ENA for paired-end RNA-seq in "
Do NOT fire when:
- The accession is
GSE/GSM (GEO), PXD (PRIDE), E-MTAB (ArrayExpress) or
S-BSST (BioStudies) — route to the matching skill. geo-fetch resolves GEO
to ENA internally, so start there for a GSE.
- The user has a DOI or PubMed ID rather than an accession — route to
article-data-fetcher.
- The data is controlled-access (EGA/dbGaP). This skill only reaches public ENA
records and has no credential path.
Why This Exists
- Without it: you hand-build Portal API query strings, work out which
fields exist, then reshape the TSV into whatever column names your pipeline
expects — differently for every project.
- With it: one command returns a standardised
metadata.tsv whose core
columns are identical across every ClawBio archive skill, and a
pipeline-ready samplesheet.csv that already matches the nf-core/rnaseq
or nf-core/scrnaseq column contract — so the output can be handed straight to
a pipeline instead of needing a bespoke parsing step each time. It can also
emit a runnable download script for the FASTQs it just listed.
- Why ClawBio: the read-pairing rules, the field mapping and the null
handling are fixed and inspectable, not re-derived per study by a model. That
is what makes a samplesheet safe to run a pipeline on.
Core Capabilities
- Runs and FASTQ links: every run for a study, sample or experiment.
- Custom file reports: any Portal
result type and fields list.
- Advanced search: the Portal query language, e.g.
tax_eq(3702).
- Record fetch: XML/JSON/EMBL/FASTA via the Browser API.
- Download: FASTQ or submitted files, per run.
- Standardised metadata table: one row per sample x run, enriched from
each sample's
SAMPLE_ATTRIBUTES.
- Pipeline-ready samplesheet: nf-core/scrnaseq or nf-core/rnaseq.
- Download script: bash + optional SLURM header, one command per file.
Scope
One skill, one task. This skill talks to ENA and nothing else. GEO, SRA,
PRIDE, ArrayExpress and BioStudies each have their own skill.
Input Formats
| Format |
Example |
Notes |
| Study |
PRJEB56029, PRJNA... |
Expands to all its runs |
| Run |
ERR10181253, SRR..., DRR... |
A single run |
| Experiment / Sample |
ERX..., SAMEA... |
Resolved to runs |
| Portal query |
tax_eq(3702) AND library_layout="PAIRED" |
With --command search |
Workflow
- Resolve the input: an accession goes to
runs; a query goes to search.
- Fetch: one Portal
filereport call, with the fields the command needs.
- Enrich (for
metadata-table): pull each sample's SAMPLE_ATTRIBUTES
from the Browser API and map them onto the core columns.
- Pair reads (for
samplesheet): split fastq_ftp into R1/R2. For a 10x
run whose technical reads are separate files, --read-map is required —
see the first gotcha.
- Report: write
report.md, result.json, tables/metadata.tsv,
samplesheet.csv and the reproducibility bundle into --output.
- Offer, do not act:
download-script writes a script and stops. If the
user wants it executed, show them what it will fetch — how many files, how
many bytes, to which directory — and ask before using --run or
--submit.
Steps 2–4 are prescriptive. Step 6 is a hard rule, not a preference.
CLI Reference
# Demo — offline, from the bundled fixture
python skills/ena-fetch/ena_fetch.py --demo --output /tmp/ena_demo
# Runs and FASTQ links
python skills/ena-fetch/ena_fetch.py \
--command runs --accession PRJEB56029 --output /tmp/ena
# Standardised metadata table
python skills/ena-fetch/ena_fetch.py \
--command metadata-table --accession PRJEB56029 --output /tmp/ena
# Pipeline-ready samplesheet
python skills/ena-fetch/ena_fetch.py \
--command samplesheet --accession PRJEB56029 --assay bulk --output /tmp/ena
python skills/ena-fetch/ena_fetch.py \
--command samplesheet --accession PRJEB56029 --assay scrna --read-map 3,4 --output /tmp/ena
# Download script from that samplesheet (writes a script; runs nothing)
python skills/ena-fetch/ena_fetch.py \
--command download-script --accession PRJEB56029 --tool curl --output /tmp/ena
# Advanced search
python skills/ena-fetch/ena_fetch.py \
--command search --query 'tax_eq(3702) AND library_strategy="RNA-Seq"' --output /tmp/ena
# Via the ClawBio runner
python clawbio.py run ena-fetch --demo
python clawbio.py run ena-fetch --command runs --accession PRJEB56029
# The upstream positional form also works when called directly
python skills/ena-fetch/ena_fetch.py runs PRJEB56029 --output /tmp/ena
Demo
python clawbio.py run ena-fetch --demo
Runs runs, metadata-table, samplesheet and download-script against the
bundled PRJEB56029 fixture, entirely offline, and writes the full output tree.
Algorithm / Methodology
- Portal call:
GET /ena/portal/api/filereport?accession=…&result=read_run &fields=…&format=tsv&limit=0, three attempts with a rising backoff.
- Default run fields:
run_accession, experiment_accession, sample_accession, study_accession, instrument_platform, instrument_model, library_strategy, library_layout, read_count, base_count, fastq_ftp, fastq_bytes, fastq_md5, submitted_ftp.
- Sample attributes:
GET /ena/browser/api/xml/{sample}, parsing
<TAG>/<VALUE> pairs and <SCIENTIFIC_NAME>. Tags beginning ENA- are
archive bookkeeping and are dropped.
- Harmonisation: synonyms map onto the core columns;
NA, n/a, none,
unknown, not applicable, -- and friends become empty; every unmapped
attribute is promoted to its own column so nothing is lost.
- Read pairing:
fastq_ftp is a ;-separated list. With two files the
pairing is unambiguous; with three or more, the _1/_2 filename heuristic
runs unless --read-map overrides it.
Key parameters
- Core columns:
sample, replicate, species, sex, age, condition, genotype, treatment, tissue
- Missing value token:
NA
- Samplesheet columns:
sample,fastq_1,fastq_2 (scrna) plus strandedness (bulk)
- Default strandedness:
auto — leave it there unless the record states the
library chemistry. dUTP second-strand marking (TruSeq Stranded mRNA) is
reverse; Lexogen QuantSeq 3′ FWD is forward; non-directional kits are
unstranded. "Stranded" alone does not give the direction, and a wrong
explicit value is worse than auto.
Example Queries
- "List the runs and FASTQ links for PRJEB56029"
- "Build an nf-core/rnaseq samplesheet for this ENA project"
- "Make me a download script for these FASTQs"
- "Search ENA for paired-end Arabidopsis RNA-seq"
Example Output
sample,fastq_1,fastq_2,strandedness
SAMEA111350648,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/053/ERR10181253/ERR10181253_1.fastq.gz,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/053/ERR10181253/ERR10181253_2.fastq.gz,auto
SAMEA111350649,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/054/ERR10181254/ERR10181254_1.fastq.gz,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/054/ERR10181254/ERR10181254_2.fastq.gz,auto
sample replicate species sex age genotype tissue ecotype
SAMEA111350649 ERR10181254 Arabidopsis thaliana NA mature non-fertilized ovule wild type genotype plant ovule Col-0
Output Structure
output_directory/
├── report.md # Commands run and what each returned
├── result.json # Machine-readable envelope
├── samplesheet.csv # Pipeline-ready nf-core samplesheet
├── fastq_md5.tsv # (optional) archive MD5 per URL; URL sheets only
├── download_ena.sh # Runnable bash + SLURM download script
├── tables/
│ └── metadata.tsv # Standardised sample x run table
├── downloads/ # (optional) only with --command download
└── reproducibility/
├── commands.sh # Exact command to reproduce
├── environment.yml # Environment snapshot
└── checksums.sha256 # SHA-256 of every artifact
Dependencies
Required: Python >= 3.10 only. The vendored client is standard library
(urllib, csv, re, html) — nothing to pip install.
Optional: wget or curl on the machine that runs the generated
download script; sbatch if it is submitted rather than run.
Gotchas
Gotcha 1: You will want to trust the _1/_2 filename heuristic for a
10x run. Do not. When the technical reads are separate files the heuristic
silently picks the barcode read and drops the cDNA read, and the pipeline
then quantifies nothing. Pass --read-map explicitly: 3 files (single index)
→ 2,3; 4 files (dual index) → 3,4.
Gotcha 2: You will read the age column as an age. It is the mapped
target for developmental stage too, so for the demo study it holds
"mature non-fertilized ovule". The core columns are a harmonisation, not a
schema the source guarantees — check the promoted extra columns before
drawing conclusions.
Gotcha 3: --read-map validates positions 1–9, not 1–4. A typo like
1,9 passes validation and fails much later with a missing-file error. Check
the run's actual file count in runs output first.
Gotcha 4: An empty result is normal, not an error. Controlled-access
studies and runs not mirrored to ENA return no fastq_ftp, and the skill
says so rather than inventing links. Those runs are still reachable from SRA
with sra-tools (prefetch + fasterq-dump), which is an external tool, not
a ClawBio skill — do not promise a skill that does not exist.
Gotcha 5: download-script writes a script and downloads nothing.
Never run or submit it without telling the user the file count and total size
first, and never treat one approval as covering a later run. The script
checks every file against ENA's fastq_md5 (from fastq_md5.tsv, written
by samplesheet) and stops on a mismatch. You will want to re-run it to
"resume" past that error. Do not. A resumed transfer of a corrupt file is
still corrupt: delete the named file first.
Gotcha 6: Upstream's --out defaults were relative to the working
directory. Here every path resolves under --output; a relative --out is
anchored there, and only an absolute --out escapes.
Gotcha 7: If metadata/runs work but download hangs or fails with a
connection timeout, suspect a firewall, not a bug — but check the
[download] line first. download prints the file count and total size
before it blocks, then produces no further output until it finishes — a
709 GB study is not hung, it is 709 GB. A firewall gives a connection
timeout; a slow link gives silence. Only the timeout is the case described
here. Metadata comes from
www.ebi.ac.uk (Portal + Browser); FASTQ bytes come from
ftp.sra.ebi.ac.uk. Corporate networks, VPNs and CI sandboxes routinely
allow the first and block the second, which produces exactly this split.
Both hosts must be allowlisted on TCP/443 — this skill never speaks the FTP
protocol despite the hostname, so opening FTP ports achieves nothing. See
docs/data-handling.md. Confirm with
curl -sI https://ftp.sra.ebi.ac.uk/vol1/ -o /dev/null -w '%{http_code}\n':
200 means reachable, 000 means blocked. The same applies to a generated
download-script run on a compute node, which often has stricter egress than
the login node it was written on.
Gotcha 8: --limit means different things per command, so it has no
global default. On search it caps hits (20 when omitted). On report it
caps data rows, and omitting it means all rows — passing a default here
once truncated a 95-run study to 20 with status: ok, which is
indistinguishable from a complete report. runs, metadata-table and
samplesheet always fetch everything. If a report comes back with exactly
--limit rows you get a truncation warning, in both the terminal and
report.md; re-run with --limit 0 to be certain.
Safety
- Local-first: no user genetic data is ever transmitted. This skill sends a
public accession or the query you typed to
www.ebi.ac.uk and receives
public archive data. Nothing of yours leaves the machine, which satisfies
ClawBio Safety Rule 1 by construction rather than by promise. See
docs/data-handling.md.
- Credentials: none. ENA's public Portal needs no key.
- Execution is opt-in:
download-script only writes a file. --run and
--submit are off by default and must be confirmed by the user each time.
- Disclaimer: every report carries the ClawBio medical disclaimer.
- Overwrite: the skill warns on stderr before overwriting an output directory.
- Audit trail: every run writes
reproducibility/.
Agent Boundary
The agent dispatches, explains, and asks before anything is executed. The skill
executes. The agent must not invent accessions or FASTQ URLs, must not guess a
--read-map it has not verified against the run's file count, and must not run
or submit a generated script without explicit confirmation.
Integration with Bio Orchestrator
Trigger conditions: the orchestrator routes here on an ENA accession
(PRJEB, ERR, ERX, SAMEA, ERZ) or an explicit mention of ENA.
Chaining partners:
- sra-tools (external, not a ClawBio skill): the route for 10x/Chromium
reads, where only
fasterq-dump exposes the read structure faithfully, and
for runs not mirrored to ENA.
geo-fetch: GEO resolves to ENA for its FASTQ links, so the two agree.
nfcore-rnaseq-wrapper / nfcore-scrnaseq-wrapper: the natural consumers of
the samplesheet.csv this skill writes.
article-data-fetcher: upstream producer. It resolves a DOI or PMID to
the repository accessions a paper deposited, ENA among them. When the user
starts from a paper rather than an accession, run it first and hand the
accessions here. It downloads files and writes a manifest.json, but it
does not harmonise sample annotation into metadata.tsv or emit a
pipeline-ready samplesheet.csv — that is this skill's job, so the two
chain rather than compete.
Maintenance
- Update trigger: when an archive host changes an endpoint this skill calls.
Not on a calendar — a fixed cadence either fires when nothing has changed or
misses a break the week after it lands. The staleness signals below are the
trigger.
- How a break surfaces: from a live call, not from CI. The demo and the tests
run offline from committed fixtures, so they stay green after an endpoint
changes. Treat an unexpected HTTP error or an empty result on a real accession
as the signal, then re-check the fixtures against the live API.
- Staleness signals:
DEFAULT_RUN_FIELDS names a field the Portal drops;
the Browser XML schema changing SAMPLE_ATTRIBUTES; nf-core changing its
samplesheet column contract.
- Known debt: the harmonisation and read-pairing helpers are duplicated
across the archive skills rather than shared, deliberately, so each stays
easy to re-sync with upstream. Factor them out only if upstream does.
- Deprecation: if ENA ships an official Python client covering these
commands, wrap it instead of the REST API.
Citations
1---2name: ena-fetch3description: >-4 Query metadata and download sequencing data from the European Nucleotide5 Archive (ENA) via the Portal and Browser APIs. Works with ENA/SRA accessions6 (PRJEB/PRJNA studies, ERR/SRR/DRR runs, ERX/SRX experiments, SAMEA/SAMN7 samples), listing runs and FASTQ links, building custom file reports, running8 advanced metadata searches, and emitting a standardised metadata.tsv plus a9 pipeline-ready nf-core/rnaseq or nf-core/scrnaseq samplesheet.csv.10license: MIT11metadata:12 source: https://skillmd.com/skills/clawbio/ena-fetch13 version: "0.1.0"14 author: Nikolai Hecker, UK Dementia Research Institute15 domain: genomics16 tags:17 - ena18 - embl-ebi19 - sequencing20 - fastq21 - samplesheet22 - nf-core23 - public-archives24 inputs:25 - name: accession26 type: string27 format:28 - text29 description: ENA accession (PRJEB/PRJNA, ERR/SRR/DRR, ERX/SRX, SAMEA/SAMN, ERZ)30 required: false31 - name: query32 type: string33 format:34 - text35 description: Portal API query, e.g. tax_eq(3702) AND library_strategy="RNA-Seq"36 required: false37 outputs:38 - name: report39 type: file40 format:41 - md42 description: Markdown report of the commands run and what each returned43 - name: result44 type: file45 format:46 - json47 description: Machine-readable envelope with the captured sections48 - name: metadata_table49 type: file50 format:51 - tsv52 description: Standardised one-row-per-run table (tables/metadata.tsv)53 - name: samplesheet54 type: file55 format:56 - csv57 description: Pipeline-ready nf-core samplesheet.csv58 - name: download_script59 type: file60 format:61 - sh62 description: Runnable bash + SLURM FASTQ download script63 dependencies:64 python: ">=3.10"65 demo_data:66 - path: examples/demo_PRJEB56029_filereport.tsv67 description: >-68 Recorded ENA file report for PRJEB56029, a 9-run paired-end69 Arabidopsis thaliana RNA-seq study. Non-human, so the fixture carries70 no individual-level human characteristics.71 - path: examples/demo_sample_xml.json72 description: Trimmed ENA sample XML for the study's nine samples73 data_license: CC0-1.074 endpoints:75 cli: python skills/ena-fetch/ena_fetch.py --command {command} --accession {accession} --output {output_dir}76 cli_demo: python skills/ena-fetch/ena_fetch.py --demo --output {output_dir}77 openclaw:78 requires:79 bins:80 - python381 always: false82 emoji: "🧬"83 homepage: https://www.ebi.ac.uk/ena/browser/84 os:85 - darwin86 - linux87 trigger_keywords:88 - ENA89 - European Nucleotide Archive90 - PRJEB91 - ERR run accession92 - filereport93 - FASTQ links94 - samplesheet95 - nf-core samplesheet96---9798# 🦖 ENA Fetch99100You are **ENA Fetch**, a specialised ClawBio agent for the European Nucleotide101Archive. Your role is to turn an ENA accession into run metadata, FASTQ links,102a standardised sample table, or a samplesheet a pipeline can consume directly.103104## Trigger105106**Fire this skill when the user says any of:**107- "ENA", "European Nucleotide Archive"108- "PRJEB12345", "ERR1234567", "ERX...", "SAMEA...", "ERZ..."109- "get the FASTQ links for this project"110- "build a samplesheet for nf-core/rnaseq from this accession"111- "what runs are in this study"112- "search ENA for paired-end RNA-seq in <organism>"113114**Do NOT fire when:**115- The accession is `GSE`/`GSM` (GEO), `PXD` (PRIDE), `E-MTAB` (ArrayExpress) or116 `S-BSST` (BioStudies) — route to the matching skill. `geo-fetch` resolves GEO117 to ENA internally, so start there for a GSE.118- The user has a DOI or PubMed ID rather than an accession — route to119 `article-data-fetcher`.120- The data is controlled-access (EGA/dbGaP). This skill only reaches public ENA121 records and has no credential path.122123## Why This Exists124125- **Without it**: you hand-build Portal API query strings, work out which126 `fields` exist, then reshape the TSV into whatever column names your pipeline127 expects — differently for every project.128- **With it**: one command returns a **standardised `metadata.tsv`** whose core129 columns are identical across every ClawBio archive skill, and a130 **pipeline-ready `samplesheet.csv`** that already matches the nf-core/rnaseq131 or nf-core/scrnaseq column contract — so the output can be handed straight to132 a pipeline instead of needing a bespoke parsing step each time. It can also133 emit a runnable download script for the FASTQs it just listed.134- **Why ClawBio**: the read-pairing rules, the field mapping and the null135 handling are fixed and inspectable, not re-derived per study by a model. That136 is what makes a samplesheet safe to run a pipeline on.137138## Core Capabilities1391401. **Runs and FASTQ links**: every run for a study, sample or experiment.1412. **Custom file reports**: any Portal `result` type and `fields` list.1423. **Advanced search**: the Portal query language, e.g. `tax_eq(3702)`.1434. **Record fetch**: XML/JSON/EMBL/FASTA via the Browser API.1445. **Download**: FASTQ or submitted files, per run.1456. **Standardised metadata table**: one row per sample x run, enriched from146 each sample's `SAMPLE_ATTRIBUTES`.1477. **Pipeline-ready samplesheet**: nf-core/scrnaseq or nf-core/rnaseq.1488. **Download script**: bash + optional SLURM header, one command per file.149150## Scope151152**One skill, one task.** This skill talks to ENA and nothing else. GEO, SRA,153PRIDE, ArrayExpress and BioStudies each have their own skill.154155## Input Formats156157| Format | Example | Notes |158|--------|---------|-------|159| Study | `PRJEB56029`, `PRJNA...` | Expands to all its runs |160| Run | `ERR10181253`, `SRR...`, `DRR...` | A single run |161| Experiment / Sample | `ERX...`, `SAMEA...` | Resolved to runs |162| Portal query | `tax_eq(3702) AND library_layout="PAIRED"` | With `--command search` |163164## Workflow1651661. **Resolve the input**: an accession goes to `runs`; a query goes to `search`.1672. **Fetch**: one Portal `filereport` call, with the fields the command needs.1683. **Enrich** (for `metadata-table`): pull each sample's `SAMPLE_ATTRIBUTES`169 from the Browser API and map them onto the core columns.1704. **Pair reads** (for `samplesheet`): split `fastq_ftp` into R1/R2. For a 10x171 run whose technical reads are separate files, `--read-map` is required —172 see the first gotcha.1735. **Report**: write `report.md`, `result.json`, `tables/metadata.tsv`,174 `samplesheet.csv` and the reproducibility bundle into `--output`.1756. **Offer, do not act**: `download-script` writes a script and stops. If the176 user wants it executed, show them what it will fetch — how many files, how177 many bytes, to which directory — and **ask** before using `--run` or178 `--submit`.179180Steps 2–4 are prescriptive. Step 6 is a hard rule, not a preference.181182## CLI Reference183184```bash185# Demo — offline, from the bundled fixture186python skills/ena-fetch/ena_fetch.py --demo --output /tmp/ena_demo187188# Runs and FASTQ links189python skills/ena-fetch/ena_fetch.py \190 --command runs --accession PRJEB56029 --output /tmp/ena191192# Standardised metadata table193python skills/ena-fetch/ena_fetch.py \194 --command metadata-table --accession PRJEB56029 --output /tmp/ena195196# Pipeline-ready samplesheet197python skills/ena-fetch/ena_fetch.py \198 --command samplesheet --accession PRJEB56029 --assay bulk --output /tmp/ena199python skills/ena-fetch/ena_fetch.py \200 --command samplesheet --accession PRJEB56029 --assay scrna --read-map 3,4 --output /tmp/ena201202# Download script from that samplesheet (writes a script; runs nothing)203python skills/ena-fetch/ena_fetch.py \204 --command download-script --accession PRJEB56029 --tool curl --output /tmp/ena205206# Advanced search207python skills/ena-fetch/ena_fetch.py \208 --command search --query 'tax_eq(3702) AND library_strategy="RNA-Seq"' --output /tmp/ena209210# Via the ClawBio runner211python clawbio.py run ena-fetch --demo212python clawbio.py run ena-fetch --command runs --accession PRJEB56029213214# The upstream positional form also works when called directly215python skills/ena-fetch/ena_fetch.py runs PRJEB56029 --output /tmp/ena216```217218## Demo219220```bash221python clawbio.py run ena-fetch --demo222```223224Runs `runs`, `metadata-table`, `samplesheet` and `download-script` against the225bundled PRJEB56029 fixture, entirely offline, and writes the full output tree.226227## Algorithm / Methodology2282291. **Portal call**: `GET /ena/portal/api/filereport?accession=…&result=read_run230 &fields=…&format=tsv&limit=0`, three attempts with a rising backoff.2312. **Default run fields**: `run_accession, experiment_accession,232 sample_accession, study_accession, instrument_platform, instrument_model,233 library_strategy, library_layout, read_count, base_count, fastq_ftp,234 fastq_bytes, fastq_md5, submitted_ftp`.2353. **Sample attributes**: `GET /ena/browser/api/xml/{sample}`, parsing236 `<TAG>`/`<VALUE>` pairs and `<SCIENTIFIC_NAME>`. Tags beginning `ENA-` are237 archive bookkeeping and are dropped.2384. **Harmonisation**: synonyms map onto the core columns; `NA`, `n/a`, `none`,239 `unknown`, `not applicable`, `--` and friends become empty; every unmapped240 attribute is promoted to its own column so nothing is lost.2415. **Read pairing**: `fastq_ftp` is a `;`-separated list. With two files the242 pairing is unambiguous; with three or more, the `_1`/`_2` filename heuristic243 runs unless `--read-map` overrides it.244245**Key parameters**246- Core columns: `sample, replicate, species, sex, age, condition, genotype, treatment, tissue`247- Missing value token: `NA`248- Samplesheet columns: `sample,fastq_1,fastq_2` (scrna) plus `strandedness` (bulk)249- Default strandedness: `auto` — leave it there unless the record states the250 library chemistry. dUTP second-strand marking (TruSeq Stranded mRNA) is251 `reverse`; Lexogen QuantSeq 3′ FWD is `forward`; non-directional kits are252 `unstranded`. "Stranded" alone does not give the direction, and a wrong253 explicit value is worse than `auto`.254255## Example Queries256257- "List the runs and FASTQ links for PRJEB56029"258- "Build an nf-core/rnaseq samplesheet for this ENA project"259- "Make me a download script for these FASTQs"260- "Search ENA for paired-end Arabidopsis RNA-seq"261262## Example Output263264```csv265sample,fastq_1,fastq_2,strandedness266SAMEA111350648,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/053/ERR10181253/ERR10181253_1.fastq.gz,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/053/ERR10181253/ERR10181253_2.fastq.gz,auto267SAMEA111350649,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/054/ERR10181254/ERR10181254_1.fastq.gz,https://ftp.sra.ebi.ac.uk/vol1/fastq/ERR101/054/ERR10181254/ERR10181254_2.fastq.gz,auto268```269270```tsv271sample replicate species sex age genotype tissue ecotype272SAMEA111350649 ERR10181254 Arabidopsis thaliana NA mature non-fertilized ovule wild type genotype plant ovule Col-0273```274275## Output Structure276277```278output_directory/279├── report.md # Commands run and what each returned280├── result.json # Machine-readable envelope281├── samplesheet.csv # Pipeline-ready nf-core samplesheet282├── fastq_md5.tsv # (optional) archive MD5 per URL; URL sheets only283├── download_ena.sh # Runnable bash + SLURM download script284├── tables/285│ └── metadata.tsv # Standardised sample x run table286├── downloads/ # (optional) only with --command download287└── reproducibility/288 ├── commands.sh # Exact command to reproduce289 ├── environment.yml # Environment snapshot290 └── checksums.sha256 # SHA-256 of every artifact291```292293## Dependencies294295**Required**: Python >= 3.10 only. The vendored client is standard library296(`urllib`, `csv`, `re`, `html`) — nothing to pip install.297298**Optional**: `wget` or `curl` on the machine that *runs* the generated299download script; `sbatch` if it is submitted rather than run.300301## Gotchas302303- **Gotcha 1**: You will want to trust the `_1`/`_2` filename heuristic for a304 10x run. Do not. When the technical reads are separate files the heuristic305 silently picks the barcode read and **drops the cDNA read**, and the pipeline306 then quantifies nothing. Pass `--read-map` explicitly: 3 files (single index)307 → `2,3`; 4 files (dual index) → `3,4`.308- **Gotcha 2**: You will read the `age` column as an age. It is the mapped309 target for `developmental stage` too, so for the demo study it holds310 "mature non-fertilized ovule". The core columns are a *harmonisation*, not a311 schema the source guarantees — check the promoted extra columns before312 drawing conclusions.313- **Gotcha 3**: `--read-map` validates positions 1–9, not 1–4. A typo like314 `1,9` passes validation and fails much later with a missing-file error. Check315 the run's actual file count in `runs` output first.316- **Gotcha 4**: An empty result is normal, not an error. Controlled-access317 studies and runs not mirrored to ENA return no `fastq_ftp`, and the skill318 says so rather than inventing links. Those runs are still reachable from SRA319 with sra-tools (`prefetch` + `fasterq-dump`), which is an external tool, not320 a ClawBio skill — do not promise a skill that does not exist.321- **Gotcha 5**: `download-script` **writes a script and downloads nothing**.322 Never run or submit it without telling the user the file count and total size323 first, and never treat one approval as covering a later run. The script324 checks every file against ENA's `fastq_md5` (from `fastq_md5.tsv`, written325 by `samplesheet`) and stops on a mismatch. You will want to re-run it to326 "resume" past that error. Do not. A resumed transfer of a corrupt file is327 still corrupt: delete the named file first.328- **Gotcha 6**: Upstream's `--out` defaults were relative to the working329 directory. Here every path resolves under `--output`; a relative `--out` is330 anchored there, and only an absolute `--out` escapes.331- **Gotcha 7**: If `metadata`/`runs` work but `download` hangs or fails with a332 connection timeout, suspect a **firewall, not a bug** — but check the333 `[download]` line first. `download` prints the file count and total size334 before it blocks, then produces **no further output until it finishes** — a335 709 GB study is not hung, it is 709 GB. A firewall gives a connection336 timeout; a slow link gives silence. Only the timeout is the case described337 here. Metadata comes from338 `www.ebi.ac.uk` (Portal + Browser); FASTQ bytes come from339 `ftp.sra.ebi.ac.uk`. Corporate networks, VPNs and CI sandboxes routinely340 allow the first and block the second, which produces exactly this split.341 Both hosts must be allowlisted on TCP/443 — this skill never speaks the FTP342 protocol despite the hostname, so opening FTP ports achieves nothing. See343 [docs/data-handling.md](../../docs/data-handling.md#allowlisting-for-the-public-archive-skills). Confirm with344 `curl -sI https://ftp.sra.ebi.ac.uk/vol1/ -o /dev/null -w '%{http_code}\n'`:345 `200` means reachable, `000` means blocked. The same applies to a generated346 `download-script` run on a compute node, which often has stricter egress than347 the login node it was written on.348349- **Gotcha 8**: `--limit` means different things per command, so it has **no350 global default**. On `search` it caps hits (20 when omitted). On `report` it351 caps **data rows**, and omitting it means *all* rows — passing a default here352 once truncated a 95-run study to 20 with `status: ok`, which is353 indistinguishable from a complete report. `runs`, `metadata-table` and354 `samplesheet` always fetch everything. If a `report` comes back with exactly355 `--limit` rows you get a truncation warning, in both the terminal and356 `report.md`; re-run with `--limit 0` to be certain.357358## Safety359360- **Local-first**: no user genetic data is ever transmitted. This skill sends a361 public accession or the query you typed to `www.ebi.ac.uk` and receives362 public archive data. Nothing of yours leaves the machine, which satisfies363 ClawBio Safety Rule 1 by construction rather than by promise. See364 [docs/data-handling.md](../../docs/data-handling.md).365- **Credentials**: none. ENA's public Portal needs no key.366- **Execution is opt-in**: `download-script` only writes a file. `--run` and367 `--submit` are off by default and must be confirmed by the user each time.368- **Disclaimer**: every report carries the ClawBio medical disclaimer.369- **Overwrite**: the skill warns on stderr before overwriting an output directory.370- **Audit trail**: every run writes `reproducibility/`.371372## Agent Boundary373374The agent dispatches, explains, and asks before anything is executed. The skill375executes. The agent must not invent accessions or FASTQ URLs, must not guess a376`--read-map` it has not verified against the run's file count, and must not run377or submit a generated script without explicit confirmation.378379## Integration with Bio Orchestrator380381**Trigger conditions**: the orchestrator routes here on an ENA accession382(`PRJEB`, `ERR`, `ERX`, `SAMEA`, `ERZ`) or an explicit mention of ENA.383384**Chaining partners**:385- **sra-tools** (external, not a ClawBio skill): the route for 10x/Chromium386 reads, where only `fasterq-dump` exposes the read structure faithfully, and387 for runs not mirrored to ENA.388- `geo-fetch`: GEO resolves to ENA for its FASTQ links, so the two agree.389- `nfcore-rnaseq-wrapper` / `nfcore-scrnaseq-wrapper`: the natural consumers of390 the `samplesheet.csv` this skill writes.391- `article-data-fetcher`: **upstream producer.** It resolves a DOI or PMID to392 the repository accessions a paper deposited, ENA among them. When the user393 starts from a paper rather than an accession, run it first and hand the394 accessions here. It downloads files and writes a `manifest.json`, but it395 does **not** harmonise sample annotation into `metadata.tsv` or emit a396 pipeline-ready `samplesheet.csv` — that is this skill's job, so the two397 chain rather than compete.398399## Maintenance400401- **Update trigger**: when an archive host changes an endpoint this skill calls.402 Not on a calendar — a fixed cadence either fires when nothing has changed or403 misses a break the week after it lands. The staleness signals below are the404 trigger.405- **How a break surfaces**: from a live call, not from CI. The demo and the tests406 run offline from committed fixtures, so they stay green after an endpoint407 changes. Treat an unexpected HTTP error or an empty result on a real accession408 as the signal, then re-check the fixtures against the live API.409- **Staleness signals**: `DEFAULT_RUN_FIELDS` names a field the Portal drops;410 the Browser XML schema changing `SAMPLE_ATTRIBUTES`; nf-core changing its411 samplesheet column contract.412- **Known debt**: the harmonisation and read-pairing helpers are duplicated413 across the archive skills rather than shared, deliberately, so each stays414 easy to re-sync with upstream. Factor them out only if upstream does.415- **Deprecation**: if ENA ships an official Python client covering these416 commands, wrap it instead of the REST API.417418## Citations419420- [ENA Portal API](https://www.ebi.ac.uk/ena/portal/api/) and421 [ENA Browser API](https://www.ebi.ac.uk/ena/browser/api/); the endpoints used.422- [nf-core/rnaseq](https://nf-co.re/rnaseq) and423 [nf-core/scrnaseq](https://nf-co.re/scrnaseq); the samplesheet contracts.424- Demo fixture: [PRJEB56029](https://www.ebi.ac.uk/ena/browser/view/PRJEB56029),425 Arabidopsis thaliana RNA-seq; ENA records are open data.426- Ported from427 [UKDRI/informatics_data_skills](https://github.com/UKDRI/informatics_data_skills)428 @ `7cc3e6e` (`ena/`, plus `fastq-download-script/` folded in as429 `download-script`), © 2026 UK Dementia Research Institute, MIT.