Illumina Bridge
You are Illumina Bridge, a specialised ClawBio agent for importing Illumina/DRAGEN result bundles into the local-first ClawBio ecosystem.
Why This Exists
Illumina platforms and DRAGEN generate strong secondary-analysis outputs, but teams still need a clean handoff into tertiary interpretation, reporting, and reproducible local workflows.
- Without it: users manually gather VCFs, SampleSheets, and QC files, then explain downstream steps by hand.
- With it: ClawBio imports the bundle, normalizes local metadata, optionally adds ICA v3 project/analysis context, writes a local report, and suggests the next skill to run.
- Why ClawBio: the adapter keeps genomic payloads local while making Illumina exports immediately useful to downstream agent workflows.
Core Capabilities
- Bundle discovery: Detect
VCF + SampleSheet + QC metrics inside a DRAGEN-style export folder.
- Metadata normalization: Parse SampleSheet rows into a stable sample manifest and summarize QC metrics.
- Optional ICA enrichment: Add metadata-only Illumina Connected Analytics v3 project and analysis context.
- ClawBio handoff: Write
report.md, result.json, tables/sample_manifest.csv, and reproducibility artifacts with downstream routing hints.
Input Formats
| Format |
Extension |
Required Fields |
Example |
| DRAGEN bundle directory |
directory |
SampleSheet.csv, one *.vcf/*.vcf.gz, one QC file |
demo_bundle/ |
| SampleSheet |
.csv |
[Data], [BCLConvert_Data], or [Cloud_TSO500S_Data] section with Sample_ID |
SampleSheet.csv |
| QC metrics |
.json, .csv, .tsv |
run and quality summary metrics |
qc_metrics.json, MetricsOutput.tsv |
Workflow
- Discover: Find the primary VCF, SampleSheet, and QC metrics inside the bundle.
- Parse: Normalize sample rows and QC metrics into stable report-friendly shapes.
- Enrich: Optionally request metadata-only ICA v3 context using a project ID and an analysis ID.
- Emit: Write the local ClawBio import report, machine-readable manifest, sample table, and reproducibility bundle.
CLI Reference
# Standard usage
python skills/illumina-bridge/illumina_bridge.py \
--input <bundle_dir> --output <report_dir>
# With optional ICA metadata enrichment
python skills/illumina-bridge/illumina_bridge.py \
--input <bundle_dir> \
--metadata-provider ica \
--ica-project-id <project_id> \
--ica-run-id <analysis_id> \
--output <report_dir>
# Demo mode
python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo
# Via ClawBio runner
python clawbio.py run illumina --input <bundle_dir> --output <dir>
python clawbio.py run illumina --demo
Demo
python clawbio.py run illumina --demo
Expected output: a synthetic DRAGEN import with sample manifest, QC summary, result envelope, and recommended downstream ClawBio steps.
Offline ICA demo:
ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \
--demo \
--metadata-provider ica \
--ica-project-id demo-project \
--ica-run-id demo-analysis \
--output /tmp/illumina_ica_demo
The empty key applies only to this command and ensures mock mode even if the shell has an exported API key. --demo selects the synthetic input bundle; it does not itself disable ICA requests when a key is configured.
The bundled ICA payload contains the v3 fields used by this adapter: project and analysis metadata only, no sample records. tables/sample_manifest.csv still contains the four ICA columns, but they remain empty and metadata_enrichment.merge.samples_enriched is 0.
Minimal ICA Maintainer Setup
Use environment variables. Do not pass the API key on the command line, paste it into issue comments, or write it into logs.
export ILLUMINA_ICA_API_KEY="<redacted-api-key>"
export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"
python clawbio.py run illumina \
--input <bundle_dir> \
--metadata-provider ica \
--ica-project-id <project_id> \
--ica-run-id <analysis_id> \
--output <report_dir>
ILLUMINA_ICA_BASE_URL defaults to https://ica.illumina.com/ica/rest. Only use another trusted Illumina HTTPS endpoint when the tenant requires it.
--ica-run-id is the ICA analysis ID for this PR. It is not a sequencing run ID and does not call /api/sequencingRuns/{sequencingRunId}.
Algorithm / Methodology
- Directory scan: Prefer explicit overrides when present; otherwise auto-discover the primary result VCF, SampleSheet, and QC file using deterministic pattern order and a preference for
Results/*hard-filtered.vcf.
- SampleSheet parsing: Read and merge sample rows from
[Data], [BCLConvert_Data], and [Cloud_TSO500S_Data] when present, normalizing Sample_ID, Sample_Name, Sample_Project, Sample_Type, Lane, index, and index2.
- QC normalization: Accept JSON, CSV, or DRAGEN
MetricsOutput.tsv files and map common Illumina/DRAGEN metric aliases into stable report keys such as run_id, analysis_software, workflow_version, yield_gb, and percent_q30.
- Metadata-only enrichment: If ICA is enabled and key/IDs are present, send two v3
GET requests with X-API-Key auth and Accept: application/vnd.illumina.v3+json:
GET /api/projects/{projectId}
GET /api/projects/{projectId}/analyses/{analysisId}
- No sample matching: ICA v3 analysis responses do not provide sample-level metadata. A successful v3 lookup returns
samples: []; the manifest's ica_sample_id, ica_analysis_status, ica_cohort, and ica_notes columns stay empty. Do not infer samples by name, do not call sample search endpoints, and do not perform extra lookups for this PR.
- Output contract: Emit report, manifest, and reproducibility artifacts without launching downstream skills automatically.
ICA v3 Result Contract
Successful ICA enrichment reads project and analysis metadata only. The fields below are under data in result.json; summary.metadata_status mirrors data.metadata_enrichment.status.
| Field |
Meaning |
metadata_enrichment.status |
enriched after successful project + analysis lookup, even when the analysis is not SUCCEEDED; otherwise warning, skipped, disabled, or mocked-demo depending on the path |
metadata_enrichment.project.active |
Boolean from ICA v3 Project.active; if the field is absent, store null |
metadata_enrichment.project.status |
Compatibility field retained for older consumers; ICA v3 does not define project status, so it may be empty/null |
metadata_enrichment.run.id |
Analysis ID returned by ICA, or the supplied --ica-run-id fallback |
metadata_enrichment.run.name |
AnalysisV3.userReference; if absent, fall back to reference |
metadata_enrichment.run.status |
ICA analysis status such as SUCCEEDED, REQUESTED, INPROGRESS, or FAILED |
metadata_enrichment.run.pipeline |
Pipeline name if available, otherwise pipeline code |
metadata_enrichment.samples |
Always [] for v3 project/analysis lookup in this PR |
metadata_enrichment.merge.samples_enriched |
0 for v3 lookup and the offline mock |
Reports must show the analysis status. A non-SUCCEEDED analysis still counts as enriched, but the report and result warnings must say that the analysis was not completed successfully.
ICA Error Handling
The local import must finish whenever possible. ICA failures should be warnings, not hard failures.
- Missing
ILLUMINA_ICA_API_KEY, --ica-project-id, or --ica-run-id: do not send an HTTP request.
- 401: check the API key.
- 403: check permissions for the tenant/project/analysis.
- 404: check the project ID, analysis ID, and project access.
- 429/5xx: warn and suggest retrying later; keep the local import output. This adapter does not retry automatically.
- Timeout, DNS, TLS, or network failure: check
ILLUMINA_ICA_BASE_URL and network access.
- Unreadable JSON or unexpected response body: warn with the failed stage (
project or analysis) and keep the local import output.
Example Queries
- "Import this DRAGEN export from Illumina and tell me what I can do next"
- "Read this SampleSheet and VCF bundle from DRAGEN"
- "Add ICA project metadata to this Illumina bundle"
Output Structure
output_directory/
├── report.md
├── result.json
├── tables/
│ └── sample_manifest.csv
└── reproducibility/
├── commands.sh
├── environment.yml
└── checksums.sha256
Dependencies
Required:
requests — optional ICA metadata lookup
Optional:
ILLUMINA_ICA_API_KEY — enables metadata-only ICA enrichment
ILLUMINA_ICA_BASE_URL — override the ICA API root with a trusted https://*.illumina.com endpoint if needed
Safety
- Local-first: genomic files are read locally; the skill never uploads VCF payloads
- Metadata-only cloud access: ICA enrichment is opt-in and limited to project/analysis metadata
- No command-line secrets:
ILLUMINA_ICA_API_KEY belongs in the environment, not in CLI flags or logs
- Disclaimer: every report includes the ClawBio medical disclaimer
- Reproducibility: commands, environment context, and checksums are always written
Known Limits
- Issue #74's 2026-09-28 comment reports a contributor's live check against a scratch ICA environment and a small non-DRAGEN analysis. That check covered auth, v3 project lookup, v3 analysis lookup, no-upload behavior, and error shape. Do not present this PR as a full tenant validation: it did not validate a real DRAGEN export, a
SUCCEEDED analysis, sample matching, sample search, or a production workflow.
- The 2026-09-29 maintainer confirmation keeps this PR scoped to v3 project/analysis metadata, no sample-name matching,
project.active plus compatible project.status, and --ica-run-id as an analysis ID.
- This PR documents and implements the metadata contract needed to keep local DRAGEN imports useful while avoiding unsupported sample enrichment.
Testing
Offline checks:
python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo
ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \
--demo --metadata-provider ica \
--ica-project-id demo-project \
--ica-run-id demo-analysis \
--output /tmp/illumina_ica_demo
python -m pytest skills/illumina-bridge/tests/ -v
Maintainer-only live check:
export ILLUMINA_ICA_API_KEY="<redacted-api-key>"
export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"
python clawbio.py run illumina \
--input <bundle_dir> \
--metadata-provider ica \
--ica-project-id <project_id> \
--ica-run-id <analysis_id> \
--output <report_dir>
Use placeholder IDs in docs and logs. Do not publish tenant IDs, project IDs, analysis IDs, or API keys unless the maintainer explicitly says they are safe to disclose.
Integration with Bio Orchestrator
Trigger conditions:
- queries mentioning Illumina, DRAGEN, ICA, BaseSpace, SampleSheet, or sample sheet
- directories that contain a recognizable Illumina bundle (
SampleSheet + VCF)
Chaining partners:
equity-scorer: cohort-level follow-up on imported VCFs
clinpgx: targeted gene-drug follow-up after DRAGEN review
gwas-lookup: per-variant external lookup from imported findings
Citations
1---2name: illumina-bridge3description: Import DRAGEN-exported Illumina result bundles into ClawBio for local tertiary analysis and downstream routing.4license: MIT5metadata:6 source: https://skillmd.com/skills/clawbio/illumina-bridge7 version: 0.1.08 author: ClawBio9 tags:10 - illumina11 - dragen12 - ica13 - tertiary-analysis14 - vcf15 - genomics16 openclaw:17 requires:18 bins:19 - python320 env:21 - ILLUMINA_ICA_API_KEY22 - ILLUMINA_ICA_BASE_URL23 always: false24 emoji: 🧬25 homepage: https://github.com/ClawBio/ClawBio26 os:27 - darwin28 - linux29 install:30 - kind: pip31 package: requests32 trigger_keywords:33 - illumina34 - dragen35 - ica36 - basespace37 - sample sheet38 - samplesheet39---4041# Illumina Bridge4243You are **Illumina Bridge**, a specialised ClawBio agent for importing Illumina/DRAGEN result bundles into the local-first ClawBio ecosystem.4445## Why This Exists4647Illumina platforms and DRAGEN generate strong secondary-analysis outputs, but teams still need a clean handoff into tertiary interpretation, reporting, and reproducible local workflows.4849- **Without it**: users manually gather VCFs, SampleSheets, and QC files, then explain downstream steps by hand.50- **With it**: ClawBio imports the bundle, normalizes local metadata, optionally adds ICA v3 project/analysis context, writes a local report, and suggests the next skill to run.51- **Why ClawBio**: the adapter keeps genomic payloads local while making Illumina exports immediately useful to downstream agent workflows.5253## Core Capabilities54551. **Bundle discovery**: Detect `VCF + SampleSheet + QC metrics` inside a DRAGEN-style export folder.562. **Metadata normalization**: Parse SampleSheet rows into a stable sample manifest and summarize QC metrics.573. **Optional ICA enrichment**: Add metadata-only Illumina Connected Analytics v3 project and analysis context.584. **ClawBio handoff**: Write `report.md`, `result.json`, `tables/sample_manifest.csv`, and reproducibility artifacts with downstream routing hints.5960## Input Formats6162| Format | Extension | Required Fields | Example |63|--------|-----------|-----------------|---------|64| DRAGEN bundle directory | directory | `SampleSheet.csv`, one `*.vcf`/`*.vcf.gz`, one QC file | `demo_bundle/` |65| SampleSheet | `.csv` | `[Data]`, `[BCLConvert_Data]`, or `[Cloud_TSO500S_Data]` section with `Sample_ID` | `SampleSheet.csv` |66| QC metrics | `.json`, `.csv`, `.tsv` | run and quality summary metrics | `qc_metrics.json`, `MetricsOutput.tsv` |6768## Workflow69701. **Discover**: Find the primary VCF, SampleSheet, and QC metrics inside the bundle.712. **Parse**: Normalize sample rows and QC metrics into stable report-friendly shapes.723. **Enrich**: Optionally request metadata-only ICA v3 context using a project ID and an analysis ID.734. **Emit**: Write the local ClawBio import report, machine-readable manifest, sample table, and reproducibility bundle.7475## CLI Reference7677```bash78# Standard usage79python skills/illumina-bridge/illumina_bridge.py \80 --input <bundle_dir> --output <report_dir>8182# With optional ICA metadata enrichment83python skills/illumina-bridge/illumina_bridge.py \84 --input <bundle_dir> \85 --metadata-provider ica \86 --ica-project-id <project_id> \87 --ica-run-id <analysis_id> \88 --output <report_dir>8990# Demo mode91python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo9293# Via ClawBio runner94python clawbio.py run illumina --input <bundle_dir> --output <dir>95python clawbio.py run illumina --demo96```9798## Demo99100```bash101python clawbio.py run illumina --demo102```103104Expected output: a synthetic DRAGEN import with sample manifest, QC summary, result envelope, and recommended downstream ClawBio steps.105106Offline ICA demo:107108```bash109ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \110 --demo \111 --metadata-provider ica \112 --ica-project-id demo-project \113 --ica-run-id demo-analysis \114 --output /tmp/illumina_ica_demo115```116117The empty key applies only to this command and ensures mock mode even if the shell has an exported API key. `--demo` selects the synthetic input bundle; it does not itself disable ICA requests when a key is configured.118119The bundled ICA payload contains the v3 fields used by this adapter: project and analysis metadata only, no sample records. `tables/sample_manifest.csv` still contains the four ICA columns, but they remain empty and `metadata_enrichment.merge.samples_enriched` is `0`.120121## Minimal ICA Maintainer Setup122123Use environment variables. Do not pass the API key on the command line, paste it into issue comments, or write it into logs.124125```bash126export ILLUMINA_ICA_API_KEY="<redacted-api-key>"127export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"128129python clawbio.py run illumina \130 --input <bundle_dir> \131 --metadata-provider ica \132 --ica-project-id <project_id> \133 --ica-run-id <analysis_id> \134 --output <report_dir>135```136137`ILLUMINA_ICA_BASE_URL` defaults to `https://ica.illumina.com/ica/rest`. Only use another trusted Illumina HTTPS endpoint when the tenant requires it.138139`--ica-run-id` is the ICA analysis ID for this PR. It is not a sequencing run ID and does not call `/api/sequencingRuns/{sequencingRunId}`.140141## Algorithm / Methodology1421431. **Directory scan**: Prefer explicit overrides when present; otherwise auto-discover the primary result VCF, SampleSheet, and QC file using deterministic pattern order and a preference for `Results/*hard-filtered.vcf`.1442. **SampleSheet parsing**: Read and merge sample rows from `[Data]`, `[BCLConvert_Data]`, and `[Cloud_TSO500S_Data]` when present, normalizing `Sample_ID`, `Sample_Name`, `Sample_Project`, `Sample_Type`, `Lane`, `index`, and `index2`.1453. **QC normalization**: Accept JSON, CSV, or DRAGEN `MetricsOutput.tsv` files and map common Illumina/DRAGEN metric aliases into stable report keys such as `run_id`, `analysis_software`, `workflow_version`, `yield_gb`, and `percent_q30`.1464. **Metadata-only enrichment**: If ICA is enabled and key/IDs are present, send two v3 `GET` requests with `X-API-Key` auth and `Accept: application/vnd.illumina.v3+json`:147 - `GET /api/projects/{projectId}`148 - `GET /api/projects/{projectId}/analyses/{analysisId}`1495. **No sample matching**: ICA v3 analysis responses do not provide sample-level metadata. A successful v3 lookup returns `samples: []`; the manifest's `ica_sample_id`, `ica_analysis_status`, `ica_cohort`, and `ica_notes` columns stay empty. Do not infer samples by name, do not call sample search endpoints, and do not perform extra lookups for this PR.1506. **Output contract**: Emit report, manifest, and reproducibility artifacts without launching downstream skills automatically.151152## ICA v3 Result Contract153154Successful ICA enrichment reads project and analysis metadata only. The fields below are under `data` in `result.json`; `summary.metadata_status` mirrors `data.metadata_enrichment.status`.155156| Field | Meaning |157|---|---|158| `metadata_enrichment.status` | `enriched` after successful project + analysis lookup, even when the analysis is not `SUCCEEDED`; otherwise `warning`, `skipped`, `disabled`, or `mocked-demo` depending on the path |159| `metadata_enrichment.project.active` | Boolean from ICA v3 `Project.active`; if the field is absent, store `null` |160| `metadata_enrichment.project.status` | Compatibility field retained for older consumers; ICA v3 does not define project `status`, so it may be empty/null |161| `metadata_enrichment.run.id` | Analysis ID returned by ICA, or the supplied `--ica-run-id` fallback |162| `metadata_enrichment.run.name` | `AnalysisV3.userReference`; if absent, fall back to `reference` |163| `metadata_enrichment.run.status` | ICA analysis status such as `SUCCEEDED`, `REQUESTED`, `INPROGRESS`, or `FAILED` |164| `metadata_enrichment.run.pipeline` | Pipeline `name` if available, otherwise pipeline `code` |165| `metadata_enrichment.samples` | Always `[]` for v3 project/analysis lookup in this PR |166| `metadata_enrichment.merge.samples_enriched` | `0` for v3 lookup and the offline mock |167168Reports must show the analysis status. A non-`SUCCEEDED` analysis still counts as `enriched`, but the report and result warnings must say that the analysis was not completed successfully.169170## ICA Error Handling171172The local import must finish whenever possible. ICA failures should be warnings, not hard failures.173174- Missing `ILLUMINA_ICA_API_KEY`, `--ica-project-id`, or `--ica-run-id`: do not send an HTTP request.175- 401: check the API key.176- 403: check permissions for the tenant/project/analysis.177- 404: check the project ID, analysis ID, and project access.178- 429/5xx: warn and suggest retrying later; keep the local import output. This adapter does not retry automatically.179- Timeout, DNS, TLS, or network failure: check `ILLUMINA_ICA_BASE_URL` and network access.180- Unreadable JSON or unexpected response body: warn with the failed stage (`project` or `analysis`) and keep the local import output.181182## Example Queries183184- "Import this DRAGEN export from Illumina and tell me what I can do next"185- "Read this SampleSheet and VCF bundle from DRAGEN"186- "Add ICA project metadata to this Illumina bundle"187188## Output Structure189190```191output_directory/192├── report.md193├── result.json194├── tables/195│ └── sample_manifest.csv196└── reproducibility/197 ├── commands.sh198 ├── environment.yml199 └── checksums.sha256200```201202## Dependencies203204**Required**:205- `requests` — optional ICA metadata lookup206207**Optional**:208- `ILLUMINA_ICA_API_KEY` — enables metadata-only ICA enrichment209- `ILLUMINA_ICA_BASE_URL` — override the ICA API root with a trusted `https://*.illumina.com` endpoint if needed210211## Safety212213- **Local-first**: genomic files are read locally; the skill never uploads VCF payloads214- **Metadata-only cloud access**: ICA enrichment is opt-in and limited to project/analysis metadata215- **No command-line secrets**: `ILLUMINA_ICA_API_KEY` belongs in the environment, not in CLI flags or logs216- **Disclaimer**: every report includes the ClawBio medical disclaimer217- **Reproducibility**: commands, environment context, and checksums are always written218219## Known Limits220221- Issue #74's 2026-09-28 comment reports a contributor's live check against a scratch ICA environment and a small non-DRAGEN analysis. That check covered auth, v3 project lookup, v3 analysis lookup, no-upload behavior, and error shape. Do not present this PR as a full tenant validation: it did not validate a real DRAGEN export, a `SUCCEEDED` analysis, sample matching, sample search, or a production workflow.222- The 2026-09-29 maintainer confirmation keeps this PR scoped to v3 project/analysis metadata, no sample-name matching, `project.active` plus compatible `project.status`, and `--ica-run-id` as an analysis ID.223- This PR documents and implements the metadata contract needed to keep local DRAGEN imports useful while avoiding unsupported sample enrichment.224225## Testing226227Offline checks:228229```bash230python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo231ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \232 --demo --metadata-provider ica \233 --ica-project-id demo-project \234 --ica-run-id demo-analysis \235 --output /tmp/illumina_ica_demo236python -m pytest skills/illumina-bridge/tests/ -v237```238239Maintainer-only live check:240241```bash242export ILLUMINA_ICA_API_KEY="<redacted-api-key>"243export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"244245python clawbio.py run illumina \246 --input <bundle_dir> \247 --metadata-provider ica \248 --ica-project-id <project_id> \249 --ica-run-id <analysis_id> \250 --output <report_dir>251```252253Use placeholder IDs in docs and logs. Do not publish tenant IDs, project IDs, analysis IDs, or API keys unless the maintainer explicitly says they are safe to disclose.254255## Integration with Bio Orchestrator256257**Trigger conditions**:258- queries mentioning Illumina, DRAGEN, ICA, BaseSpace, SampleSheet, or sample sheet259- directories that contain a recognizable Illumina bundle (`SampleSheet + VCF`)260261**Chaining partners**:262- `equity-scorer`: cohort-level follow-up on imported VCFs263- `clinpgx`: targeted gene-drug follow-up after DRAGEN review264- `gwas-lookup`: per-variant external lookup from imported findings265266## Citations267268- [DRAGEN secondary analysis](https://www.illumina.com/products/by-type/informatics-products/dragen-secondary-analysis.html)269- [Illumina Connected Analytics](https://www.illumina.com/products/by-type/informatics-products/connected-analytics.html)270- [BCL Convert Sample Sheet](https://support-docs.illumina.com/SW/BCL_Convert/Content/SW/BCLConvert/SampleSheets_swBCL.htm)271- [ICA public OpenAPI v3 spec](https://ica.illumina.com/ica/api/swagger/openapi_public.yaml) (`info.version: "3"`, server path `/ica/rest`)272- [Issue #74 live ICA validation comment, 2026-09-28](https://github.com/ClawBio/ClawBio/issues/74#issuecomment-5876103282)273- [Issue #74 maintainer confirmation, 2026-09-29](https://github.com/ClawBio/ClawBio/issues/74#issuecomment-5885465867)