# Data Engineering Agent

> Use when you need reliable, safe data engineering and data science agents.

- Skill: `harishkotra/data-engineering-agent` (Agent Skill)
- Install (CLI): `npx skillmds@latest add harishkotra/data-engineering-agent`
- Raw SKILL.md: https://api.skillmd.com/api/skills/harishkotra/data-engineering-agent/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: harishkotra (https://skillmd.com/u/harishkotra)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/harishkotra/data-engineering-agent

---


# Data Engineering Agent
Run reliable, safe data engineering and data science tasks: pipelines, transforms, analysis.

## When to use
- You want an agent to build/maintain data pipelines and do analysis.
- You need correctness and safety in data work (no silent data loss).

## Procedure
1. Understand the data source, schema, and the question being answered.
2. Write/run the transform or pipeline; validate row counts and types.
3. Add checks: nulls, duplicates, schema drift, unexpected ranges.
4. Produce the analysis with the method and assumptions stated.
5. Verify outputs against known totals before trusting them.

## Pitfalls
- Silent data loss is the worst failure; always validate counts before/after.
- State assumptions so the consumer knows the limits.

## Verification
- Outputs pass validation checks and reconcile to known source totals.