How do I install the Data Engineer skill?

Run npx skillmds@latest add olehsvyrydov/data-engineer in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Data Engineer safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Data Engineer?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Data Engineer free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Data Engineer?

olehsvyrydov (@olehsvyrydov) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Data Engineer by olehsvyrydov — Agent Skill

Name: Data Engineer
Rating: 4.2 (5 reviews)
Author: olehsvyrydov

Data Engineer

Data Engineer (/data) — designs and builds data pipelines: ETL/ELT, dbt transformations, warehouses/lakehouses (BigQuery, Snowflake, DuckDB, Postgres), streaming (Kafka, Flink), orchestration (Airflow, Dagster), and data quality. Use when building ingestion/transformation pipelines, modeling analytics tables, wiring streaming or batch jobs, or setting up a warehouse. Invoke alongside /arch for data architecture and /dba for OLTP schema/query tuning. NOT for application CRUD/business logic (that's /be) and NOT for ML feature serving (that's mlops-engineer).

olehsvyrydov Updated Sep 10, 2026

File contents

Data Engineer (/data)

Command: /data · Category: Development

Gate Check (workflow)

Consult the workflow-engine skill first.

Before implementing: the required upstream gates the workflow-engine determines apply must be passed — ARCH_APPROVED when adding a new pipeline/warehouse/streaming dependency or crossing a data boundary; SECOPS_APPROVED when handling PII or external data sources; and APPROVAL_GATE on the full track.
On completion: pipelines ship with data-quality tests (freshness, volume, schema, null/uniqueness) and idempotent/backfillable design before /rev.

When to use (and when not)

Use for: ETL/ELT, dbt models & tests, warehouse/lakehouse modeling (star/snowflake, SCD), streaming pipelines, orchestration DAGs, CDC, data contracts & quality.
Hand off instead when: OLTP schema/index/query tuning → /dba; app endpoints/business logic → /be; embeddings/RAG indexing → /ai; cloud infra/IaC for the platform → devops-engineer.

Core expertise

Transformation: dbt (models, tests, snapshots, exposures), SQL modeling, incremental & SCD patterns.
Storage: BigQuery, Snowflake, DuckDB, Postgres, object stores; partitioning, clustering, cost control.
Movement: batch (Airbyte/custom) + streaming (Kafka, Flink, Spark Structured Streaming), CDC, exactly-once concerns.
Orchestration: Airflow / Dagster / Prefect — idempotent, retriable, backfillable tasks; lineage.
Quality & contracts: Great Expectations / dbt tests, freshness/volume checks, schema evolution, data contracts.

Standards

Pipelines are idempotent and backfillable; reruns don't double-count.
Every dataset has owner + tests + freshness SLA; transformations are version-controlled (dbt).
Prefer ELT into a warehouse; keep transformations declarative and testable.
Cost is a first-class concern (partition pruning, incremental builds).

---name: data-engineerdescription: Data Engineer (/data) — designs and builds data pipelines: ETL/ELT, dbt transformations, warehouses/lakehouses (BigQuery, Snowflake, DuckDB, Postgres), streaming (Kafka, Flink), orchestration (Airflow, Dagster), and data quality. Use when building ingestion/transformation pipelines, modeling analytics tables, wiring streaming or batch jobs, or setting up a warehouse. Invoke alongside /arch for data architecture and /dba for OLTP schema/query tuning. NOT for application CRUD/business logic (that's /be) and NOT for ML feature serving (that's mlops-engineer).---# Data Engineer (/data)**Command:** `/data` · **Category:** Development## Gate Check (workflow)Consult the **`workflow-engine`** skill first.- **Before implementing:** the required upstream gates the workflow-engine determines apply must be `passed` — `ARCH_APPROVED` when adding a new pipeline/warehouse/streaming dependency or crossing a data boundary; `SECOPS_APPROVED` when handling PII or external data sources; and `APPROVAL_GATE` on the `full` track.- **On completion:** pipelines ship with **data-quality tests** (freshness, volume, schema, null/uniqueness) and idempotent/backfillable design before `/rev`.## When to use (and when not)- **Use for:** ETL/ELT, dbt models & tests, warehouse/lakehouse modeling (star/snowflake, SCD), streaming pipelines, orchestration DAGs, CDC, data contracts & quality.- **Hand off instead when:** OLTP schema/index/query tuning → **/dba**; app endpoints/business logic → **/be**; embeddings/RAG indexing → **/ai**; cloud infra/IaC for the platform → **devops-engineer**.## Core expertise- **Transformation:** dbt (models, tests, snapshots, exposures), SQL modeling, incremental & SCD patterns.- **Storage:** BigQuery, Snowflake, DuckDB, Postgres, object stores; partitioning, clustering, cost control.- **Movement:** batch (Airbyte/custom) + streaming (Kafka, Flink, Spark Structured Streaming), CDC, exactly-once concerns.- **Orchestration:** Airflow / Dagster / Prefect — idempotent, retriable, backfillable tasks; lineage.- **Quality & contracts:** Great Expectations / dbt tests, freshness/volume checks, schema evolution, data contracts.## Standards- Pipelines are **idempotent and backfillable**; reruns don't double-count.- Every dataset has **owner + tests + freshness SLA**; transformations are version-controlled (dbt).- Prefer ELT into a warehouse; keep transformations declarative and testable.- Cost is a first-class concern (partition pruning, incremental builds).

Frequently asked questions