# Txpipe

> How to work with TXPipe, the DESC 3x2pt weak-lensing pipeline. Covers all pipeline phases (ingestion → photo-z → selection → calibration → masks → maps → noise maps → 2pt → covariance → theory → diagnostics), key stage names, file tags, pipeline YAML structure, and critical gotchas. Use this skill whenever the user imports txpipe; works with any TXPipe stage (TXTwoPoint, TXTwoPointFourier, TXSourceMaps, TXLensMaps, TXSourceSelector*, TXShearCalibration, TXTracerMetadata, TXSimpleMask, TXCustomMask, TXIngest*, TXBlinding, TXFourierNamasterCovariance, TXRealNamasterCovariance, TXJackknifeCenters, TXPSFDiagnostics, TXRoweStatistics, TXTauStatistics, etc.); writes a TXPipe pipeline YAML; asks about shape or shear catalogs, metacalibration or metadetect, source or lens selection, tomographic binning, 3x2pt two-point functions (shear-shear, galaxy-galaxy lensing, angular clustering), map generation, noise maps, jackknife covariance, PSF diagnostics, or theory predictions.

- Skill: `jfcrenshaw/txpipe` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add jfcrenshaw/txpipe`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jfcrenshaw/txpipe/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: jfcrenshaw (https://skillmd.com/u/jfcrenshaw)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/jfcrenshaw/txpipe

---


# TXPipe — DESC 3×2pt weak-lensing pipeline

You're helping a user work with TXPipe, the main DESC analysis workhorse.
TXPipe processes shear and photometric catalogs through sample selection, map generation, two-point measurements (real and Fourier space), covariance estimation, theory comparison, and diagnostics.
All stages are ceci `PipelineStage` subclasses; for ceci conventions see the `ceci` skill.
For NERSC job submission, see the `nersc` skill.
Key library dependencies: NaMaster (`namaster` skill) for Fourier C_ℓ measurements and covariance; CCL (`ccl` skill) for theory predictions; Firecrown (`firecrown` skill) is used internally by theory/blinding stages and consumes TXPipe's sacc output for parameter inference.

## Pipeline phases (in order)

1. **Ingestion** — `TXIngestDataPreview02`, `TXCosmoDC2Mock`, etc. — source-specific; using the wrong ingester silently reads wrong columns
2. **Photo-z** — RAIL stages (`PZEstimatorLens/Source`, `PZRailSummarizeLens/Source`); must produce `QPNOfZFile` before 2pt
3. **Selection** — `TXSourceSelectorMetadetect/Metacal/Lensfit/HSC` + `TXSourceTomography`; `TXMeanLensSelector` for lenses
4. **Calibration** — `TXShearCalibration` then `TXTracerMetadata` (both required; calibration must precede tomography)
5. **Masks** — `TXSimpleMask` or `TXCustomMask`; nside must match map nside everywhere
6. **Maps** — `TXSourceMaps`, `TXLensMaps`, `TXAuxiliarySourceMaps/LensMaps`, `TXLSSWeights`
7. **Noise maps** — `TXSourceNoiseMaps` (~30 realisations), `TXLensNoiseMaps` (~5); required for covariance
8. **Two-point real** — `TXTwoPoint` (TreeCorr; ξ±, γ_t, w(θ)); uses `TXJackknifeCenters` for jackknife cov
9. **Two-point Fourier** — `TXTwoPointFourier` (NaMaster; C_ℓ for shear-shear, shear-pos, pos-pos; see `namaster` skill)
10. **Covariance** — `TXFourierNamasterCovariance` / `TXRealNamasterCovariance` (expensive; use `TXFourierGaussianCovariance` for quick tests; see `namaster` skill)
11. **Theory & blinding** — `TXTwoPointTheoryReal/Fourier` (CCL; see `ccl` skill); `TXBlinding` (Firecrown internally; see `firecrown` skill)
12. **Diagnostics** — `TXPSFDiagnostics`, `TXRoweStatistics`, `TXTauStatistics`, `TXSourceDiagnosticPlots`, `TXTwoPointPlots`

The `examples/metadetect/` pipeline is the canonical full-pipeline reference.

## Reference

| Topic | Reference |
|---|---|
| Pipeline phases and stage names | [`references/txpipe-stages.md`](references/txpipe-stages.md) |
| Key file types, pipeline YAML structure, example pipelines, gotchas | [`references/txpipe-pipeline.md`](references/txpipe-pipeline.md) |