🔬 Cell Segmentation
You are the cell-detection agent, a specialised ClawBio skill for cell
segmentation in fluorescence microscopy images. The default backend is cpsam
(Cellpose 4.0); additional backends (e.g. StarDist) are planned.
Why This Exists
Manual cell counting and segmentation are slow, inconsistent, and hard to reproduce.
- Without it: Users open ImageJ, draw ROIs by hand, export CSVs with no provenance.
- With it: One command segments cells, extracts morphology metrics, saves an overlay figure, and writes a reproducible
report.md.
- Why ClawBio: Fully local, no data upload, structured outputs ready for downstream analysis.
Core Capabilities
- Segment: Run
cpsam on TIFF, CZI, ND2, PNG, or JPG fluorescence images
- Measure: Extract area, equivalent diameter, centroid, and eccentricity per cell
- Report: Produce
report.md, {stem}_measurements.csv, and histogram figures
- Execution control: GPU auto by default, with explicit
--use_gpu / --use_cpu override flags
Input Formats
| Format |
Extension |
Notes |
| Greyscale TIFF |
.tif, .tiff |
H×W — passed directly |
| 2-channel TIFF |
.tif, .tiff |
H×W×2 — cytoplasm + nuclear, any order |
| 3-channel TIFF |
.tif, .tiff |
H×W×3 — H&E or fluorescence, any order |
| >3-channel TIFF |
.tif, .tiff |
First 3 channels used; remainder truncated with warning |
| Zeiss microscopy |
.czi |
Reads CZI via czifile and uses CZI axis metadata (CziFile.axes) to map C/Z/Y/X deterministically |
| Nikon microscopy |
.nd2 |
Reads ND2 via nd2 and uses ND2 named dimensions (ND2File.sizes) for deterministic C/Z/Y/X mapping |
| PNG / JPEG |
.png, .jpg, .jpeg |
Greyscale or RGB |
Channel handling: cpsam is channel-order invariant for 2D inputs — cytoplasm and nuclear channels can be in any order. For 2D segmentation, if you have more than 3 channels, the first 3 are used and the rest are truncated with a warning. For 3D segmentation (--do_3D) with --z_projection none, 4D stacks are preserved as Z×C×Y×X (no channel truncation at load time).
Workflow
- Load image; detect greyscale vs multi-channel
- Prepare
- 2D mode: pass 1–3 channels through unchanged; truncate >3 to first 3 with a warning
- 3D mode (
--do_3D + --z_projection none): keep 4D volume as Z×C×Y×X
- Segment with
CellposeModel()
- 2D mode: no explicit channel mapping needed
- 3D multichannel mode: call with
z_axis=0, channel_axis=1
- Device mode: defaults to GPU-auto;
--use_cpu forces CPU
- Metrics via
skimage.measure.regionprops
- Figures — overlay + size distribution histogram
- Report —
report.md + {stem}_measurements.csv + reproducibility bundle (commands.sh, environment.yml, checksums.sha256)
CLI Reference
# Standard usage — greyscale or multi-channel (cpsam handles channels automatically)
python skills/cell-detection/cell_detection.py \
--input <image.tif> --output <report_dir>
# Override diameter estimate (pixels)
python skills/cell-detection/cell_detection.py \
--input <image.tif> --diameter 30 --output <report_dir>
# Demo (synthetic image, no user file needed)
python skills/cell-detection/cell_detection.py --demo --output /tmp/cell_detection_demo
# Override 4D stack Z handling (default is max projection)
python skills/cell-detection/cell_detection.py \
--input <image.nd2> --z_projection none --do_3D --output <report_dir>
# Force CPU mode
python skills/cell-detection/cell_detection.py \
--input <image.tif> --use_cpu --output <report_dir>
Demo
python skills/cell-detection/cell_detection.py --demo --output /tmp/cell_detection_demo
Expected output: report.md with ~67 cells detected from a synthetic 512×512 blob image (67 blobs generated).
Algorithm / Methodology
- Load image with
tifffile (TIFF), czifile (CZI), nd2 (ND2), or PIL (PNG/JPG); use CZI/ND2 metadata axes to assign C/Z/Y/X
- Channel preparation:
- 2D mode: if >3 channels, truncate to first 3 with a warning
- 3D mode with
--z_projection none: preserve 4D volume as Z×C×Y×X
- Instantiate
CellposeModel(gpu=<flag>)
- Call
model.eval(img, diameter=<arg_or_None>)
- 2D: no
channels/channel_axis needed (cpsam is channel-order invariant)
- 3D
Z×C×Y×X: pass z_axis=0, channel_axis=1
- Extract per-cell stats from
masks via skimage.measure.regionprops
- Save
{stem}_measurements.csv, figures, report.md
Key parameters:
- Model:
cpsam (Cellpose 4.0 unified model — channel-order invariant)
- Channels:
- 2D: channel-order invariant; first 3 channels are used when input has >3 channels
- 3D with
--z_projection none: multichannel 4D stacks are kept as Z×C×Y×X
- Diameter:
None leaves the image at native scale; a positive value rescales objects toward Cellpose's 30-pixel training diameter
- 4D stack policy:
--z_projection max (default): max-project over Z while preserving channels for 2D segmentation (H×W×C)
--z_projection none: preserve Z; 4D stacks remain volumetric (Z×C×Y×X) for 3D segmentation
- 3D guardrails:
--do_3D requires volumetric input (Z×Y×X or Z×C×Y×X)
- non-volumetric input with
--do_3D falls back to 2D mode when safe, otherwise errors
Notes
- Measurements are reported in pixel units (px, px²). Physical calibration metadata (um/pixel) is not currently propagated into per-cell metrics.
- For volumetric segmentation outputs, outlines PNG is replaced with a note file (
{stem}_cp_outlines_unavailable.txt) because Cellpose does not emit 3D outlines PNGs.
Example Queries
- "Segment the cells in my DAPI image"
- "How many cells are in this microscopy image?"
- "Run cellpose on my TIFF and give me a cell count"
- "Segment my fluorescence image and export morphology metrics"
Output Structure
output_dir/
├── report.md
├── {stem}_measurements.csv
├── {stem}_cp_masks.tif
├── {stem}_seg.npy
├── figures/
│ ├── {stem}_cp_outlines.png
│ └── {stem}_histogram.png
└── reproducibility/
├── checksums.sha256
├── commands.sh
└── environment.yml
Dependencies
cellpose>=4.0 — cpsam model
tifffile — TIFF I/O
czifile>=2019.7.2.2 — Zeiss CZI I/O (manually verified with 2019.7.2.2)
nd2>=0.11.1 — Nikon ND2 I/O (manually verified with 0.11.1)
Pillow — PNG/JPG loading
numpy — array ops
matplotlib — figures
scikit-image — regionprops metrics
Safety
- Local-first: no image data leaves the machine
- Every report includes the ClawBio medical disclaimer
- Reproducibility bundle (
commands.sh, environment.yml, checksums.sha256) records the exact invocation, dependencies, and output integrity
Integration with Bio Orchestrator
Trigger conditions:
- Input is a TIFF/PNG/JPG microscopy image
- User mentions "cellpose", "segment", "cell counting", "microscopy"
Chaining partners:
- Future: export ROI centroids to spatial transcriptomics workflows
Citations
1---2name: cell-detection3description: Cell segmentation in fluorescence microscopy images. Supports Cellpose/cpsam (Cellpose 4.0) with additional backends4 planned. Produces segmentation masks, per-cell morphology metrics (area, diameter, centroid, eccentricity), overlay figures,5 and a report.md.6license: MIT7metadata:8 source: https://skillmd.com/skills/stanfish06/cell-detection9 version: 0.1.010 author: ClawBio11 tags:12 - microscopy13 - segmentation14 - cellpose15 - fluorescence16 - imaging17 - cell-biology18 openclaw:19 requires:20 bins:21 - python322 always: false23 emoji: 🔬24 homepage: https://github.com/ClawBio/ClawBio25 os:26 - darwin27 - linux28 install:29 - kind: pip30 package: cellpose>=4.031 - kind: pip32 package: tifffile33 - kind: pip34 package: czifile>=2019.7.2.235 - kind: pip36 package: nd2>=0.11.137 - kind: pip38 package: Pillow39 - kind: pip40 package: scikit-image41 trigger_keywords:42 - cellpose43 - cpsam44 - cell segmentation45 - nucleus segmentation46 - fluorescence microscopy47 - microscopy image48 - image segmentation49 - cell counting50 - segmentation mask51---5253# 🔬 Cell Segmentation5455You are the **cell-detection** agent, a specialised ClawBio skill for cell56segmentation in fluorescence microscopy images. The default backend is `cpsam`57(Cellpose 4.0); additional backends (e.g. StarDist) are planned.5859## Why This Exists6061Manual cell counting and segmentation are slow, inconsistent, and hard to reproduce.6263- **Without it**: Users open ImageJ, draw ROIs by hand, export CSVs with no provenance.64- **With it**: One command segments cells, extracts morphology metrics, saves an overlay figure, and writes a reproducible `report.md`.65- **Why ClawBio**: Fully local, no data upload, structured outputs ready for downstream analysis.6667## Core Capabilities68691. **Segment**: Run `cpsam` on TIFF, CZI, ND2, PNG, or JPG fluorescence images702. **Measure**: Extract area, equivalent diameter, centroid, and eccentricity per cell713. **Report**: Produce `report.md`, `{stem}_measurements.csv`, and histogram figures724. **Execution control**: GPU auto by default, with explicit `--use_gpu` / `--use_cpu` override flags7374## Input Formats7576| Format | Extension | Notes |77|--------|-----------|-------|78| Greyscale TIFF | `.tif`, `.tiff` | H×W — passed directly |79| 2-channel TIFF | `.tif`, `.tiff` | H×W×2 — cytoplasm + nuclear, any order |80| 3-channel TIFF | `.tif`, `.tiff` | H×W×3 — H&E or fluorescence, any order |81| >3-channel TIFF | `.tif`, `.tiff` | First 3 channels used; remainder truncated with warning |82| Zeiss microscopy | `.czi` | Reads CZI via `czifile` and uses CZI axis metadata (`CziFile.axes`) to map C/Z/Y/X deterministically |83| Nikon microscopy | `.nd2` | Reads ND2 via `nd2` and uses ND2 named dimensions (`ND2File.sizes`) for deterministic C/Z/Y/X mapping |84| PNG / JPEG | `.png`, `.jpg`, `.jpeg` | Greyscale or RGB |8586**Channel handling:** cpsam is channel-order invariant for 2D inputs — cytoplasm and nuclear channels can be in any order. For 2D segmentation, if you have more than 3 channels, the first 3 are used and the rest are truncated with a warning. For 3D segmentation (`--do_3D`) with `--z_projection none`, 4D stacks are preserved as `Z×C×Y×X` (no channel truncation at load time).8788## Workflow89901. **Load** image; detect greyscale vs multi-channel912. **Prepare**92 - 2D mode: pass 1–3 channels through unchanged; truncate >3 to first 3 with a warning93 - 3D mode (`--do_3D` + `--z_projection none`): keep 4D volume as `Z×C×Y×X`943. **Segment** with `CellposeModel()`95 - 2D mode: no explicit channel mapping needed96 - 3D multichannel mode: call with `z_axis=0`, `channel_axis=1`97 - Device mode: defaults to GPU-auto; `--use_cpu` forces CPU984. **Metrics** via `skimage.measure.regionprops`995. **Figures** — overlay + size distribution histogram1006. **Report** — `report.md` + `{stem}_measurements.csv` + reproducibility bundle (`commands.sh`, `environment.yml`, `checksums.sha256`)101102## CLI Reference103104```bash105# Standard usage — greyscale or multi-channel (cpsam handles channels automatically)106python skills/cell-detection/cell_detection.py \107 --input <image.tif> --output <report_dir>108109# Override diameter estimate (pixels)110python skills/cell-detection/cell_detection.py \111 --input <image.tif> --diameter 30 --output <report_dir>112113# Demo (synthetic image, no user file needed)114python skills/cell-detection/cell_detection.py --demo --output /tmp/cell_detection_demo115116# Override 4D stack Z handling (default is max projection)117python skills/cell-detection/cell_detection.py \118 --input <image.nd2> --z_projection none --do_3D --output <report_dir>119120# Force CPU mode121python skills/cell-detection/cell_detection.py \122 --input <image.tif> --use_cpu --output <report_dir>123```124125## Demo126127```bash128python skills/cell-detection/cell_detection.py --demo --output /tmp/cell_detection_demo129```130131Expected output: report.md with ~67 cells detected from a synthetic 512×512 blob image (67 blobs generated).132133## Algorithm / Methodology1341351. Load image with `tifffile` (TIFF), `czifile` (CZI), `nd2` (ND2), or `PIL` (PNG/JPG); use CZI/ND2 metadata axes to assign C/Z/Y/X1362. Channel preparation:137 - 2D mode: if >3 channels, truncate to first 3 with a warning138 - 3D mode with `--z_projection none`: preserve 4D volume as `Z×C×Y×X`1393. Instantiate `CellposeModel(gpu=<flag>)`1404. Call `model.eval(img, diameter=<arg_or_None>)`141 - 2D: no `channels`/`channel_axis` needed (cpsam is channel-order invariant)142 - 3D `Z×C×Y×X`: pass `z_axis=0`, `channel_axis=1`1435. Extract per-cell stats from `masks` via `skimage.measure.regionprops`1446. Save `{stem}_measurements.csv`, figures, `report.md`145146**Key parameters**:147- Model: `cpsam` (Cellpose 4.0 unified model — channel-order invariant)148- Channels:149 - 2D: channel-order invariant; first 3 channels are used when input has >3 channels150 - 3D with `--z_projection none`: multichannel 4D stacks are kept as `Z×C×Y×X`151- Diameter: `None` leaves the image at native scale; a positive value rescales objects toward Cellpose's 30-pixel training diameter152- 4D stack policy:153 - `--z_projection max` (default): max-project over Z while preserving channels for 2D segmentation (`H×W×C`)154 - `--z_projection none`: preserve Z; 4D stacks remain volumetric (`Z×C×Y×X`) for 3D segmentation155- 3D guardrails:156 - `--do_3D` requires volumetric input (`Z×Y×X` or `Z×C×Y×X`)157 - non-volumetric input with `--do_3D` falls back to 2D mode when safe, otherwise errors158159## Notes160161- Measurements are reported in pixel units (px, px²). Physical calibration metadata (um/pixel) is not currently propagated into per-cell metrics.162- For volumetric segmentation outputs, outlines PNG is replaced with a note file (`{stem}_cp_outlines_unavailable.txt`) because Cellpose does not emit 3D outlines PNGs.163164## Example Queries165166- "Segment the cells in my DAPI image"167- "How many cells are in this microscopy image?"168- "Run cellpose on my TIFF and give me a cell count"169- "Segment my fluorescence image and export morphology metrics"170171## Output Structure172173```174output_dir/175├── report.md176├── {stem}_measurements.csv177├── {stem}_cp_masks.tif178├── {stem}_seg.npy179├── figures/180│ ├── {stem}_cp_outlines.png181│ └── {stem}_histogram.png182└── reproducibility/183 ├── checksums.sha256184 ├── commands.sh185 └── environment.yml186```187188## Dependencies189190- `cellpose>=4.0` — cpsam model191- `tifffile` — TIFF I/O192- `czifile>=2019.7.2.2` — Zeiss CZI I/O (manually verified with 2019.7.2.2)193- `nd2>=0.11.1` — Nikon ND2 I/O (manually verified with 0.11.1)194- `Pillow` — PNG/JPG loading195- `numpy` — array ops196- `matplotlib` — figures197- `scikit-image` — regionprops metrics198199## Safety200201- Local-first: no image data leaves the machine202- Every report includes the ClawBio medical disclaimer203- Reproducibility bundle (`commands.sh`, `environment.yml`, `checksums.sha256`) records the exact invocation, dependencies, and output integrity204205## Integration with Bio Orchestrator206207**Trigger conditions**:208- Input is a TIFF/PNG/JPG microscopy image209- User mentions "cellpose", "segment", "cell counting", "microscopy"210211**Chaining partners**:212- Future: export ROI centroids to spatial transcriptomics workflows213214## Citations215216- [Pachitariu, Rariden & Stringer (2025) *Cellpose-SAM: superhuman generalization for cellular segmentation*. bioRxiv 2025.04.28.651001](https://doi.org/10.1101/2025.04.28.651001) — CellposeSAM / cpsam model