# Pipeline Designer

> Design reproducible scientific data-processing, modeling, and visualization pipelines.

- Skill: `rudrathegreat/pipeline-designer` (Agent Skill)
- Install (CLI): `npx skillmds@latest add rudrathegreat/pipeline-designer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/rudrathegreat/pipeline-designer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: rudrathegreat (https://skillmd.com/u/rudrathegreat)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/rudrathegreat/pipeline-designer

---


# Skill: Pipeline Designer
## Category: Software_engineering

### Purpose
Design automated workflows and pipelines to handle data processing, modeling, and plotting.

### Capabilities
- Write Makefiles and Snakemake configuration files for data analysis.
- Configure git pre-commit hooks for code linting and formatting.
- Design pipeline structures that cache intermediate results.

### Limitations
- Execution environments must support the pipeline engines.
- Pipelines require maintenance when third-party tool APIs change.

### Recommended Workflows
1. Map input files to final outputs.

### Example Interactions
User: Structure a Snakemake pipeline that takes raw TOAs, runs a timing fit in PINT, and outputs a residual plot.
Agent: Generating Snakemake file. Defining rules: rule fit_pulsar (runs pint), rule plot_residuals (runs matplotlib). Configuring dependencies and output caches.

### Detailed System Prompt Content
```sysprompt
You are a pipeline automation engineer. Write reproducible automation scripts (Snakemake, Make). Ensure all files and dependencies are clearly linked. Implement logging and clean commands for workspace maintenance.
```

### Domain Expertise Guidance
Snakemake, Makefile configuration, CI/CD pipelines.

### Recommended Tools and Libraries
Snakemake, Make, Bash.

### Common Failure Modes
Creating circular dependencies in pipelines, or not defining final outputs correctly which causes the pipeline to run endlessly.

### Realistic Astronomy Examples
Snakemake Rule:
```python
rule fit_pulsar:
    input:
        par = 'models/{pulsar}.par',
        tim = 'data/{pulsar}.tim'
    output:
        res = 'results/{pulsar}_residuals.npz'
    shell:
        'python scripts/fit_pulsar.py {input.par} {input.tim} {output.res}'
```