De Lead Routing

DE Lead Routing Skill

Purpose

Routes data engineering tasks to appropriate DE expert agents. This skill contains the coordination logic for orchestrating data engineering agents across orchestration, modeling, processing, streaming, and warehouse specializations.

Engineers Under Management

Tool/Framework Detection

Keyword Mapping

File Pattern Mapping

Routing Decision (Priority Order)

Before routing via Agent tool, evaluate in this order:

Step 1: Agent Teams Eligibility (R018)

Check if Agent Teams is available (CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 or TeamCreate/SendMessage tools present).

Step 2: Codex-Exec Hybrid (Code Generation)

For new pipeline code, DAG scaffolding, or SQL model generation:

1. Check /tmp/.claude-env-status-* for codex, gemini, and rtk availability
2. If codex available AND task involves new file creation → automatically delegate to /codex-exec for scaffolding:
   • Display: [Codex Hybrid] Delegating to codex-exec...
   • codex-exec generates initial code (strength: fast generation)
   • Selected DE expert reviews and refines codex output (strength: reasoning, quality)
3. If codex unavailable but gemini available → delegate to /gemini-exec for scaffolding:
   • Display: [Gemini Hybrid] Delegating to gemini-exec...
   • gemini-exec generates initial code
   • Selected DE expert reviews and refines output
4. If RTK available (RTK=available in env status) → optionally wrap DE expert output through RTK to reduce token consumption by 60-90%:
   • Display: [RTK Proxy] Token optimization active via rtk-exec
   • RTK acts as a transparent proxy — no change to expert selection
5. If none available → display [External CLI] Unavailable — proceeding with {expert} directly and use DE expert directly

Suitable: New DAG files, dbt model scaffolding, SQL template generation Unsuitable: Existing pipeline modification, architecture decisions, data quality analysis

Step 3: Expert Selection

Route to appropriate DE expert based on tool/framework detection.

Permission Mode: When spawning agents, pass mode: "bypassPermissions" in the Agent tool call if the session uses bypassPermissions. Without explicit mode, CC defaults to acceptEdits.

Step 4: Ontology-RAG Enrichment (R019)

If get_agent_for_task MCP tool is available, call it with the original query and inject suggested_skills into the agent prompt. Skip silently on failure.

Step 5: Soul Injection (R006)

If the selected agent has soul: true in frontmatter, read and prepend .claude/agents/souls/{agent-name}.soul.md content to the prompt. Skip silently if file doesn't exist.

Command Routing

DE Request → Detection → Expert Agent

Airflow DAG → de-airflow-expert
dbt model   → de-dbt-expert
Spark job   → de-spark-expert
Kafka topic → de-kafka-expert
Snowflake   → de-snowflake-expert
Pipeline    → de-pipeline-expert
Multi-tool  → Multiple experts (parallel)

Routing Rules

1. Pipeline Development Workflow

1. Receive pipeline task request
2. Identify tools and components:
   - DAG orchestration → de-airflow-expert
   - SQL transformations → de-dbt-expert
   - Distributed processing → de-spark-expert
   - Event streaming → de-kafka-expert
   - Warehouse operations → de-snowflake-expert
   - Architecture decisions → de-pipeline-expert
3. Select appropriate experts
4. Distribute tasks (parallel if 2+ tools)
5. Aggregate results
6. Present unified report

Example:

User: "Design a pipeline that runs dbt models from Airflow and loads into Snowflake"

Detection:
  - Airflow DAG → de-airflow-expert
  - dbt model → de-dbt-expert
  - Snowflake loading → de-snowflake-expert
  - Pipeline architecture → de-pipeline-expert

Route (parallel where independent):
  Agent(de-pipeline-expert → overall architecture design)
  Agent(de-airflow-expert → DAG structure)
  Agent(de-dbt-expert → model design)
  Agent(de-snowflake-expert → warehouse setup)

Aggregate:
  Pipeline architecture defined
  Airflow DAG: 5 tasks designed
  dbt: 12 models structured
  Snowflake: warehouse + schema configured

2. Data Quality Workflow

1. Analyze data quality requirements
2. Route to appropriate experts:
   - dbt tests → de-dbt-expert
   - Pipeline validation → de-pipeline-expert
   - Source freshness → de-airflow-expert
3. Coordinate cross-tool quality strategy

3. Multi-Tool Projects

For projects spanning multiple DE tools:

1. Detect all DE tools in project
2. Identify primary tool (most files/configs)
3. Route to appropriate experts:
   - If task spans multiple tools → parallel experts
   - If task is tool-specific → single expert
4. Coordinate cross-tool consistency

Sub-agent Model Selection

Model Mapping by Task Type

Model Mapping by Agent

No Match Fallback

When a data engineering tool is detected but no matching agent exists:

User Input → No matching DE agent
  ↓
Detect: DE tool keyword or config file pattern
  ↓
Delegate to mgr-creator with context:
  domain: detected DE tool
  type: de-engineer
  keywords: extracted tool names
  file_patterns: detected config patterns
  skills: auto-discover from .claude/skills/
  guides: auto-discover from templates/guides/

Examples of dynamic creation triggers:

• New data tools (e.g., "Dagster DAG 만들어줘", "Flink 스트리밍 설정해줘")
• Unfamiliar data formats or connectors
• Data tool detected in project but no specialist agent

Usage

This skill is NOT user-invocable. It should be automatically triggered when the main conversation detects data engineering intent.

Detection criteria:

• User requests pipeline design or data engineering
• User mentions DE tool names (Airflow, dbt, Spark, Kafka, Snowflake)
• User provides DE-related file paths (dags/, models/, etc.)
• User requests data quality or lineage work

Converted and distributed by TomeVault — claim your Tome and manage your conversions.