Data Architecture

Data Architecture Skill

Purpose

Design data architectures including data models, pipeline designs, governance frameworks, and quality management for operational and analytical systems.

Parameters

Execution Flow

┌──────────────────────────────────────────────────────────┐
│ 1. VALIDATE: Check data domain and requirements          │
│ 2. DISCOVER: Identify data sources and entities          │
│ 3. MODEL: Create conceptual/logical/physical model       │
│ 4. DESIGN: Pipeline or governance framework              │
│ 5. QUALITY: Define data quality rules                    │
│ 6. VALIDATE: Check model consistency                     │
│ 7. DOCUMENT: Return data architecture                    │
└──────────────────────────────────────────────────────────┘

Retry Logic

Logging & Observability

log_points:
  - event: design_started
    level: info
    data: [design_type, data_type]
  - event: entities_identified
    level: info
    data: [entity_count, relationship_count]
  - event: quality_rules_defined
    level: info
    data: [rule_count, dimensions_covered]

metrics:
  - name: models_created
    type: counter
    labels: [design_type]
  - name: design_time_ms
    type: histogram
  - name: entity_count
    type: gauge

Error Handling

Unit Test Template

test_cases:
  - name: "E-commerce data model"
    input:
      data_domain: "E-commerce order management"
      design_type: "model"
      output_format: "erd"
    expected:
      has_entities: true
      entities_include: ["Customer", "Order", "Product"]
      has_relationships: true
      valid_erd: true

  - name: "Analytics pipeline"
    input:
      data_domain: "Customer analytics"
      design_type: "pipeline"
      data_type: "analytical"
    expected:
      has_ingestion: true
      has_transformation: true
      has_serving: true

  - name: "Data quality rules"
    input:
      data_domain: "User profiles"
      design_type: "quality"
    expected:
      has_dimensions: true
      dimensions_include: ["completeness", "accuracy"]
      has_rules: true

Troubleshooting

Common Issues

Debug Checklist

□ Is data domain clearly defined?
□ Are all entities identified?
□ Are relationships correctly typed?
□ Is output format valid?
□ Are quality dimensions covered?

Data Quality Dimensions

Integration

Quality Standards

• Normalized: 3NF for operational, denormalized for analytical
• Documented: All entities and relationships described
• Quality-first: DQ rules for all critical fields

Version History