scRNA Preprocessing And Clustering
Version Compatibility
Reference examples assume:
scanpy 1.10+
anndata 0.10+
pandas 2.2+
matplotlib 3.8+
Before using code patterns, verify installed versions match the environment:
- Python:
python -c "import scanpy, anndata; print(scanpy.__version__, anndata.__version__)"
- If signatures differ, inspect the installed API and adapt the pattern instead of retrying unchanged.
Overview
Use this skill to turn raw or minimally processed scRNA-seq data into an analysis-ready object with:
- QC-filtered cells and genes
- normalized expression values
- highly variable genes
- PCA and UMAP embeddings
- Leiden clusters
- saved
h5ad artifact for annotation, DE, integration, or trajectory analysis
When To Use This Skill
- raw 10x matrices, filtered count matrices, or
h5ad inputs need standard preprocessing
- the user wants UMAP, clustering, or marker discovery
- downstream tasks depend on a stable single-cell object rather than ad hoc plots
Quick Route
- If the input is already a processed
h5ad, inspect adata.raw, embeddings, cluster columns, and QC columns before rerunning preprocessing.
- If the input is raw counts, do QC first and only normalize after filtering obvious low-quality cells.
- If multiple batches are present, preprocess cleanly first, then consider integration instead of hiding batch effects with aggressive filtering.
Progressive Disclosure
- Read technical_reference.md for QC decision rules, assay caveats, and integration branching.
- Read commands_and_thresholds.md for concrete Scanpy code, default thresholds, and output conventions.
Default Rules
- Keep raw counts recoverable. Prefer
adata.raw = adata.copy() before regression or scaling.
- Report thresholds explicitly. Do not silently drop cells or genes.
- Show QC distributions before applying hard filters.
- Use vector outputs such as
.pdf or .svg for final figures when possible.
Expected Inputs
- 10x directory,
.h5, .h5ad, or count matrix
- cell metadata if available
- species context for mitochondrial or ribosomal gene detection
Expected Outputs
results/processed.h5ad
qc/cell_qc_metrics.tsv
qc/gene_qc_metrics.tsv
figures/qc_violin.pdf
figures/pca_variance_ratio.pdf
figures/umap_leiden.pdf
Preferred Tools
scanpy
anndata
pandas
matplotlib
seaborn
Starter Pattern
import scanpy as sc
adata = sc.read_10x_mtx("counts/")
adata.var_names_make_unique()
adata.var["mt"] = adata.var_names.str.upper().str.startswith("MT-")
sc.pp.calculate_qc_metrics(adata, qc_vars=["mt"], inplace=True)
adata = adata[
(adata.obs["n_genes_by_counts"] >= 200)
& (adata.obs["n_genes_by_counts"] <= 6000)
& (adata.obs["pct_counts_mt"] < 15),
:
].copy()
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
adata.raw = adata.copy()
sc.pp.highly_variable_genes(adata, n_top_genes=3000, flavor="seurat")
adata = adata[:, adata.var["highly_variable"]].copy()
sc.pp.scale(adata, max_value=10)
sc.tl.pca(adata, svd_solver="arpack")
sc.pp.neighbors(adata, n_neighbors=15, n_pcs=30)
sc.tl.umap(adata)
sc.tl.leiden(adata, resolution=0.5, key_added="leiden_r05")
adata.write("results/processed.h5ad")
Workflow
1. Load and validate the object
- confirm orientation is cells by genes
- make gene names unique
- record sample IDs and batch labels before merging or filtering
2. Compute QC metrics and inspect distributions
n_genes_by_counts
total_counts
pct_counts_mt
- optional ribosomal or hemoglobin fractions
Plot distributions before filtering. Thresholds vary by chemistry, tissue, and nucleus versus whole-cell assay.
3. Filter cells and genes
Use dataset-aware thresholds. Good first-pass defaults:
min_genes >= 200
max_genes <= 5000-8000 to remove likely doublets in many droplet datasets
pct_counts_mt < 10-20 depending on tissue stress
min_cells >= 3 for genes
4. Normalize, log-transform, and select HVGs
- normalize with
target_sum=1e4
log1p
- select
2000-4000 HVGs
- save raw counts before heavy transformations
5. Reduce dimensions and cluster
- PCA on HVGs
- neighbor graph using
10-30 PCs and 10-30 neighbors as a starting range
- UMAP for visualization
- Leiden across a small resolution grid such as
0.2, 0.5, 0.8, 1.0
6. Export analysis-ready artifacts
Always save:
- processed
h5ad
- QC tables
- cluster assignments
- publication-ready QC and UMAP figures
Output Artifacts
results/processed.h5ad: main reusable AnnData object
results/cluster_assignments.tsv: barcode plus cluster labels
qc/filter_summary.tsv: counts before and after filtering
figures/umap_leiden.pdf: main embedding figure
Quality Review
- Median genes per cell should be plausible for the chemistry and tissue.
- Mitochondrial fraction should not dominate retained cells.
- PCA variance should decay smoothly rather than showing obvious technical axes only.
- UMAP should be reviewed together with QC metrics and batch labels, not alone.
- Cluster labels should not be finalized before marker inspection.
Anti-Patterns
- reprocessing an already integrated object as if it were raw counts
- using a single universal mitochondrial threshold for every tissue
- interpreting UMAP separation as biology before checking batch and QC covariates
- discarding raw counts needed later for DE or pseudobulk
Related Skills
- Cell Annotation
- Cell Communication
- Trajectory And Lineage
- Multiome And scATAC
Optional Supplements
1---2name: scrna-preprocessing-clustering3description: Standard scRNA-seq preprocessing and clustering with Scanpy. Use for QC, normalization, HVG selection, PCA, neighbor graph construction, UMAP, Leiden clustering, and export of an analysis-ready AnnData object.4tool_type: python5primary_tool: scanpy6metadata:7 source: https://skillmd.com/skills/stanfish06/scrna-preprocessing-clustering8---910# scRNA Preprocessing And Clustering1112## Version Compatibility1314Reference examples assume:1516- `scanpy` 1.10+17- `anndata` 0.10+18- `pandas` 2.2+19- `matplotlib` 3.8+2021Before using code patterns, verify installed versions match the environment:2223- Python: `python -c "import scanpy, anndata; print(scanpy.__version__, anndata.__version__)"`24- If signatures differ, inspect the installed API and adapt the pattern instead of retrying unchanged.2526## Overview2728Use this skill to turn raw or minimally processed scRNA-seq data into an analysis-ready object with:2930- QC-filtered cells and genes31- normalized expression values32- highly variable genes33- PCA and UMAP embeddings34- Leiden clusters35- saved `h5ad` artifact for annotation, DE, integration, or trajectory analysis3637## When To Use This Skill3839- raw 10x matrices, filtered count matrices, or `h5ad` inputs need standard preprocessing40- the user wants UMAP, clustering, or marker discovery41- downstream tasks depend on a stable single-cell object rather than ad hoc plots4243## Quick Route4445- If the input is already a processed `h5ad`, inspect `adata.raw`, embeddings, cluster columns, and QC columns before rerunning preprocessing.46- If the input is raw counts, do QC first and only normalize after filtering obvious low-quality cells.47- If multiple batches are present, preprocess cleanly first, then consider integration instead of hiding batch effects with aggressive filtering.4849## Progressive Disclosure5051- Read [technical_reference.md](technical_reference.md) for QC decision rules, assay caveats, and integration branching.52- Read [commands_and_thresholds.md](commands_and_thresholds.md) for concrete Scanpy code, default thresholds, and output conventions.5354## Default Rules5556- Keep raw counts recoverable. Prefer `adata.raw = adata.copy()` before regression or scaling.57- Report thresholds explicitly. Do not silently drop cells or genes.58- Show QC distributions before applying hard filters.59- Use vector outputs such as `.pdf` or `.svg` for final figures when possible.6061## Expected Inputs6263- 10x directory, `.h5`, `.h5ad`, or count matrix64- cell metadata if available65- species context for mitochondrial or ribosomal gene detection6667## Expected Outputs6869- `results/processed.h5ad`70- `qc/cell_qc_metrics.tsv`71- `qc/gene_qc_metrics.tsv`72- `figures/qc_violin.pdf`73- `figures/pca_variance_ratio.pdf`74- `figures/umap_leiden.pdf`7576## Preferred Tools7778- `scanpy`79- `anndata`80- `pandas`81- `matplotlib`82- `seaborn`8384## Starter Pattern8586```python87import scanpy as sc8889adata = sc.read_10x_mtx("counts/")90adata.var_names_make_unique()91adata.var["mt"] = adata.var_names.str.upper().str.startswith("MT-")92sc.pp.calculate_qc_metrics(adata, qc_vars=["mt"], inplace=True)9394adata = adata[95 (adata.obs["n_genes_by_counts"] >= 200)96 & (adata.obs["n_genes_by_counts"] <= 6000)97 & (adata.obs["pct_counts_mt"] < 15),98 :99].copy()100101sc.pp.filter_genes(adata, min_cells=3)102sc.pp.normalize_total(adata, target_sum=1e4)103sc.pp.log1p(adata)104adata.raw = adata.copy()105106sc.pp.highly_variable_genes(adata, n_top_genes=3000, flavor="seurat")107adata = adata[:, adata.var["highly_variable"]].copy()108sc.pp.scale(adata, max_value=10)109sc.tl.pca(adata, svd_solver="arpack")110sc.pp.neighbors(adata, n_neighbors=15, n_pcs=30)111sc.tl.umap(adata)112sc.tl.leiden(adata, resolution=0.5, key_added="leiden_r05")113adata.write("results/processed.h5ad")114```115116## Workflow117118### 1. Load and validate the object119120- confirm orientation is cells by genes121- make gene names unique122- record sample IDs and batch labels before merging or filtering123124### 2. Compute QC metrics and inspect distributions125126- `n_genes_by_counts`127- `total_counts`128- `pct_counts_mt`129- optional ribosomal or hemoglobin fractions130131Plot distributions before filtering. Thresholds vary by chemistry, tissue, and nucleus versus whole-cell assay.132133### 3. Filter cells and genes134135Use dataset-aware thresholds. Good first-pass defaults:136137- `min_genes >= 200`138- `max_genes <= 5000-8000` to remove likely doublets in many droplet datasets139- `pct_counts_mt < 10-20` depending on tissue stress140- `min_cells >= 3` for genes141142### 4. Normalize, log-transform, and select HVGs143144- normalize with `target_sum=1e4`145- `log1p`146- select `2000-4000` HVGs147- save raw counts before heavy transformations148149### 5. Reduce dimensions and cluster150151- PCA on HVGs152- neighbor graph using `10-30` PCs and `10-30` neighbors as a starting range153- UMAP for visualization154- Leiden across a small resolution grid such as `0.2`, `0.5`, `0.8`, `1.0`155156### 6. Export analysis-ready artifacts157158Always save:159160- processed `h5ad`161- QC tables162- cluster assignments163- publication-ready QC and UMAP figures164165## Output Artifacts166167- `results/processed.h5ad`: main reusable AnnData object168- `results/cluster_assignments.tsv`: barcode plus cluster labels169- `qc/filter_summary.tsv`: counts before and after filtering170- `figures/umap_leiden.pdf`: main embedding figure171172## Quality Review173174- Median genes per cell should be plausible for the chemistry and tissue.175- Mitochondrial fraction should not dominate retained cells.176- PCA variance should decay smoothly rather than showing obvious technical axes only.177- UMAP should be reviewed together with QC metrics and batch labels, not alone.178- Cluster labels should not be finalized before marker inspection.179180## Anti-Patterns181182- reprocessing an already integrated object as if it were raw counts183- using a single universal mitochondrial threshold for every tissue184- interpreting UMAP separation as biology before checking batch and QC covariates185- discarding raw counts needed later for DE or pseudobulk186187## Related Skills188189- Cell Annotation190- Cell Communication191- Trajectory And Lineage192- Multiome And scATAC193194## Optional Supplements195196- `anndata`197- `scanpy`