Data Lineage Architecture Lens
Cognitive Mode: Data-Centric
Primary Question: "Where is the data?"
Focus: Information Flow, Transformations, Storage Locations, Format Conversions
When to Use
- Need to understand how data flows through the system
- Documenting data transformations and conversions
- Identifying storage destinations and access patterns
- User invokes
/autoskillit:arch-lens-data-lineage or /autoskillit:make-arch-diag data
Critical Constraints
NEVER:
- Modify any source code files
- Focus on runtime behavior (that's process flow lens)
- Show static structure without data context
- Run subagents in the background (
run_in_background: true is prohibited)
ALWAYS:
Arguments
/autoskillit:arch-lens-data-lineage [context_path]
- context_path (optional) — Absolute path to a PR context file containing new files
(★-prefixed) and modified files (●-prefixed) from the PR diff. When provided, read
this file before beginning analysis and focus the diagram on the architectural areas
affected by these specific files. When absent, explore the full CWD.
Analysis Workflow
Step 0: Read PR context (when provided)
If a context_path positional argument is present:
- Read the file at
context_path
- Extract: new files list (★-prefixed), modified files list (●-prefixed)
- Focus Step 1 exploration on the modules/components these files belong to
- Apply ★ prefix on diagram nodes representing new files/components
- Apply ● prefix on diagram nodes representing modified files/components
If no context_path is provided, skip this step and explore the full CWD in Step 1.
Step 1: Launch Parallel Exploration Subagents
Spawn Explore subagents to investigate:
Data Origins (Inputs)
- Find user input handling
- Identify external data sources
- Look for: CLI args, API requests, file reads, imports, user input, data ingestion
Transformation Stages
- Find data conversion/transformation code
- Identify adapters and converters
- Look for: Adapter, Converter, transform, parse, serialize, from_*, to_*, mapping, conversion
Format Changes
- Find schema definitions and conversions
- Identify format boundaries (JSON, XML, protobuf, etc.)
- Look for: schema models, type definitions, serialization, deserialization, format conversion
Storage Destinations
- Find database operations
- Identify file outputs
- Look for: database operations, persistence, .save(), .create(), .write(), storage
Access Patterns
- Find data retrieval code
- Identify query patterns
- Look for: .get(), .query(), .find(), .load(), read operations, data access layer
Step 2: Map Data Flow
Document the journey of key data entities:
- Origin: Where does it come from?
- Transformations: What changes happen?
- Storage: Where is it persisted?
- Retrieval: How is it accessed later?
CRITICAL - Analyze Read/Write Direction:
For EVERY storage location and data flow:
- Read sources (inputs): Components that READ from this location
- Write destinations (outputs): Components that WRITE to this location
- Read-write (primary storage): Both read and written by the system
- Write-only (artifacts): Written but NEVER read back by the system
Clearly distinguish:
- Primary storage (source of truth) - system reads AND writes
- Write-only artifacts (debugging, logging) - system writes but never reads back
- External inputs - system reads only
Use different arrow styles:
- Solid arrows for read/write primary storage
- Dashed arrows for write-only artifacts
Step 3: Identify Conversion Boundaries
Find format changes:
- External format -> Internal format
- Internal format -> Database format
- Database format -> API response
- Note naming convention changes
Step 4: Create the Diagram
Use flowchart with:
Direction: LR (left-to-right) for data flow, or TB for hierarchical
Subgraphs for Stages:
- Input/Origins
- Transformation/Processing
- Storage (primary)
- Artifacts (secondary/write-only)
- External Sync (if applicable)
Node Styling:
cli class: Data origins, user input
handler class: Transformation, adapters
stateNode class: Database tables, primary storage
output class: Write-only artifacts, files
integration class: External sync, APIs
Connection Types:
- Solid arrows for primary data flow
- Dashed arrows for write-only/secondary
- Label with operation names
Database Nodes:
- Use cylinder shape:
[(Label)]
- Show table relationships
Step 5: Write Output
Write the diagram to: {{AUTOSKILLIT_TEMP}}/arch-lens-data-lineage/arch_diag_data_lineage_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
After writing the diagram file, emit a structured output line:
IMPORTANT: Emit the structured output tokens as literal plain text with no
markdown formatting on the token names. Do not wrap token names in **bold**,
*italic*, or any other markdown. The adjudicator performs a regex match on the
exact token name — decorators cause match failure.
diagram_path = {absolute_path_to_diagram_file}
Output Template
# Data Lineage Diagram: {System Name}
**Lens:** Data Lineage (Data-Centric)
**Question:** Where is the data?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}
## Data Flow Overview
| Stage | Format | Key Transformation |
|-------|--------|-------------------|
| Input | {format} | {description} |
| Processing | {format} | {description} |
| Storage | {format} | {description} |
## Lineage Diagram
```mermaid
%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart LR
%% CLASS DEFINITIONS %%
classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
classDef integration fill:#c62828,stroke:#ef9a9a,stroke-width:2px,color:#fff;
subgraph Input ["Data Origins"]
USER["User Input<br/>━━━━━━━━━━<br/>Source type<br/>Format"]
end
subgraph Transform ["Transformation"]
direction TB
ADAPTER["Adapter<br/>━━━━━━━━━━<br/>Conversion type"]
end
subgraph Storage ["Primary Storage (Source of Truth)"]
direction TB
DB[("Database Table<br/>━━━━━━━━━━<br/>Key fields")]
end
subgraph Artifacts ["Write-Only Artifacts"]
direction TB
FILE["output.json<br/>━━━━━━━━━━<br/>For debugging"]
end
%% FLOWS %%
USER -->|"input"| ADAPTER
ADAPTER -->|"save()"| DB
DB -.->|"write-only"| FILE
%% CLASS ASSIGNMENTS %%
class USER cli;
class ADAPTER handler;
class DB stateNode;
class FILE output;
Color Legend:
| Color |
Category |
Description |
| Dark Blue |
Input |
Data origins (user, external) |
| Orange |
Transform |
Format conversion and adapters |
| Teal |
Storage |
Primary storage (source of truth) |
| Dark Teal |
Artifacts |
Write-only outputs |
| Red |
Sync |
External sync services |
Data Transformation Summary
| Stage |
Format |
Key Conversion |
| {stage} |
{format} |
{conversion} |
Storage Destinations
| Entity |
Primary Storage |
Secondary |
Access Pattern |
| {entity} |
{location} |
{artifact} |
{how accessed} |
Critical Design Principle
Source of Truth: {e.g., "Database is single source of truth. File outputs are write-only."}
---
## Pre-Diagram Checklist
Before creating the diagram, verify:
- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table
---
## Related Skills
- `/autoskillit:make-arch-diag` - Parent skill for lens selection
- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram
- `/autoskillit:arch-lens-c4-container` - For container-level storage view
1---2name: arch-lens-data-lineage3categories: [arch-lens]4activate_deps: [mermaid]5description: Create Data Lineage architecture diagram showing information flow, transformations, and storage destinations. Data-centric lens answering "Where is the data?"6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Data Lineage Lens - Tracing information flow...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/arch-lens-data-lineage15---1617# Data Lineage Architecture Lens1819**Cognitive Mode:** Data-Centric20**Primary Question:** "Where is the data?"21**Focus:** Information Flow, Transformations, Storage Locations, Format Conversions2223## When to Use2425- Need to understand how data flows through the system26- Documenting data transformations and conversions27- Identifying storage destinations and access patterns28- User invokes `/autoskillit:arch-lens-data-lineage` or `/autoskillit:make-arch-diag data`2930## Critical Constraints3132**NEVER:**33- Modify any source code files34- Focus on runtime behavior (that's process flow lens)35- Show static structure without data context36- Run subagents in the background (`run_in_background: true` is prohibited)3738**ALWAYS:**39- Trace data from INPUT to STORAGE40- Show transformation stages and format changes41- Identify the single source of truth42- Distinguish read vs write operations43- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY44- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.45- After writing the diagram file, emit the **absolute path** as a structured output46 token as your final output. Resolve the relative `temp/arch-lens-data-lineage/...`47 save path to absolute by prepending the full CWD:48 ```49 diagram_path = /absolute/cwd/temp/arch-lens-data-lineage/{filename}.md50 ```51 This token is MANDATORY — the pipeline cannot proceed without it.5253## Arguments5455`/autoskillit:arch-lens-data-lineage [context_path]`5657- **context_path** (optional) — Absolute path to a PR context file containing new files58 (★-prefixed) and modified files (●-prefixed) from the PR diff. When provided, read59 this file before beginning analysis and focus the diagram on the architectural areas60 affected by these specific files. When absent, explore the full CWD.6162---6364## Analysis Workflow6566### Step 0: Read PR context (when provided)6768If a `context_path` positional argument is present:691. Read the file at `context_path`702. Extract: new files list (★-prefixed), modified files list (●-prefixed)713. Focus Step 1 exploration on the modules/components these files belong to724. Apply ★ prefix on diagram nodes representing new files/components735. Apply ● prefix on diagram nodes representing modified files/components7475If no `context_path` is provided, skip this step and explore the full CWD in Step 1.7677### Step 1: Launch Parallel Exploration Subagents7879Spawn Explore subagents to investigate:8081**Data Origins (Inputs)**82- Find user input handling83- Identify external data sources84- Look for: CLI args, API requests, file reads, imports, user input, data ingestion8586**Transformation Stages**87- Find data conversion/transformation code88- Identify adapters and converters89- Look for: Adapter, Converter, transform, parse, serialize, from_*, to_*, mapping, conversion9091**Format Changes**92- Find schema definitions and conversions93- Identify format boundaries (JSON, XML, protobuf, etc.)94- Look for: schema models, type definitions, serialization, deserialization, format conversion9596**Storage Destinations**97- Find database operations98- Identify file outputs99- Look for: database operations, persistence, .save(), .create(), .write(), storage100101**Access Patterns**102- Find data retrieval code103- Identify query patterns104- Look for: .get(), .query(), .find(), .load(), read operations, data access layer105106### Step 2: Map Data Flow107108Document the journey of key data entities:109- **Origin**: Where does it come from?110- **Transformations**: What changes happen?111- **Storage**: Where is it persisted?112- **Retrieval**: How is it accessed later?113114**CRITICAL - Analyze Read/Write Direction:**115For EVERY storage location and data flow:116- **Read sources (inputs)**: Components that READ from this location117- **Write destinations (outputs)**: Components that WRITE to this location118- **Read-write (primary storage)**: Both read and written by the system119- **Write-only (artifacts)**: Written but NEVER read back by the system120121Clearly distinguish:122- Primary storage (source of truth) - system reads AND writes123- Write-only artifacts (debugging, logging) - system writes but never reads back124- External inputs - system reads only125126Use different arrow styles:127- Solid arrows for read/write primary storage128- Dashed arrows for write-only artifacts129130### Step 3: Identify Conversion Boundaries131132Find format changes:133- External format -> Internal format134- Internal format -> Database format135- Database format -> API response136- Note naming convention changes137138### Step 4: Create the Diagram139140Use flowchart with:141142**Direction:** `LR` (left-to-right) for data flow, or `TB` for hierarchical143144**Subgraphs for Stages:**145- Input/Origins146- Transformation/Processing147- Storage (primary)148- Artifacts (secondary/write-only)149- External Sync (if applicable)150151**Node Styling:**152- `cli` class: Data origins, user input153- `handler` class: Transformation, adapters154- `stateNode` class: Database tables, primary storage155- `output` class: Write-only artifacts, files156- `integration` class: External sync, APIs157158**Connection Types:**159- Solid arrows for primary data flow160- Dashed arrows for write-only/secondary161- Label with operation names162163**Database Nodes:**164- Use cylinder shape: `[(Label)]`165- Show table relationships166167### Step 5: Write Output168169Write the diagram to: `{{AUTOSKILLIT_TEMP}}/arch-lens-data-lineage/arch_diag_data_lineage_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)170171After writing the diagram file, emit a structured output line:172173> **IMPORTANT:** Emit the structured output tokens as **literal plain text with no174> markdown formatting on the token names**. Do not wrap token names in `**bold**`,175> `*italic*`, or any other markdown. The adjudicator performs a regex match on the176> exact token name — decorators cause match failure.177178```179diagram_path = {absolute_path_to_diagram_file}180```181182---183184## Output Template185186```markdown187# Data Lineage Diagram: {System Name}188189**Lens:** Data Lineage (Data-Centric)190**Question:** Where is the data?191**Date:** {YYYY-MM-DD}192**Scope:** {What was analyzed}193194## Data Flow Overview195196| Stage | Format | Key Transformation |197|-------|--------|-------------------|198| Input | {format} | {description} |199| Processing | {format} | {description} |200| Storage | {format} | {description} |201202## Lineage Diagram203204```mermaid205%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%206flowchart LR207 %% CLASS DEFINITIONS %%208 classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;209 classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;210 classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;211 classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;212 classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;213 classDef integration fill:#c62828,stroke:#ef9a9a,stroke-width:2px,color:#fff;214215 subgraph Input ["Data Origins"]216 USER["User Input<br/>━━━━━━━━━━<br/>Source type<br/>Format"]217 end218219 subgraph Transform ["Transformation"]220 direction TB221 ADAPTER["Adapter<br/>━━━━━━━━━━<br/>Conversion type"]222 end223224 subgraph Storage ["Primary Storage (Source of Truth)"]225 direction TB226 DB[("Database Table<br/>━━━━━━━━━━<br/>Key fields")]227 end228229 subgraph Artifacts ["Write-Only Artifacts"]230 direction TB231 FILE["output.json<br/>━━━━━━━━━━<br/>For debugging"]232 end233234 %% FLOWS %%235 USER -->|"input"| ADAPTER236 ADAPTER -->|"save()"| DB237 DB -.->|"write-only"| FILE238239 %% CLASS ASSIGNMENTS %%240 class USER cli;241 class ADAPTER handler;242 class DB stateNode;243 class FILE output;244```245246**Color Legend:**247| Color | Category | Description |248|-------|----------|-------------|249| Dark Blue | Input | Data origins (user, external) |250| Orange | Transform | Format conversion and adapters |251| Teal | Storage | Primary storage (source of truth) |252| Dark Teal | Artifacts | Write-only outputs |253| Red | Sync | External sync services |254255## Data Transformation Summary256257| Stage | Format | Key Conversion |258|-------|--------|----------------|259| {stage} | {format} | {conversion} |260261## Storage Destinations262263| Entity | Primary Storage | Secondary | Access Pattern |264|--------|-----------------|-----------|----------------|265| {entity} | {location} | {artifact} | {how accessed} |266267## Critical Design Principle268269> **Source of Truth**: {e.g., "Database is single source of truth. File outputs are write-only."}270```271272---273274## Pre-Diagram Checklist275276Before creating the diagram, verify:277278- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool279- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)280- [ ] Diagram will include a color legend table281282---283284## Related Skills285286- `/autoskillit:make-arch-diag` - Parent skill for lens selection287- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram288- `/autoskillit:arch-lens-c4-container` - For container-level storage view