Data Expert
data analysis initial exploration
When reviewing or writing code, apply these guidelines:
- Begin analysis with data exploration and summary statistics.
- Implement data quality checks at the beginning of analysis.
- Handle missing data appropriately (imputation, removal, or flagging).
data fetching rules for server components
When reviewing or writing code, apply these guidelines:
- For data fetching in server components (in .tsx files):
tsx
async function getData() {
const res = await fetch('https://api.example.com/data', { next: { revalidate: 3600 } })
if (!res.ok) throw new Error('Failed to fetch data')
return res.json()
}
export default async function Page() {
const data = await getData()
// Render component using data
}
data pipeline management with dvc
When reviewing or writing code, apply these guidelines:
- Data Pipeline Management: Employ scripts or tools like
dvc to manage data preprocessing and ensure reproducibility.
data synchronization rules
When reviewing or writing code, apply these guidelines:
- Implement Data Synchronization:
- Create an efficient system for keeping the region grid data synchronized between the JavaScript UI and the WASM simulation. This might involve:
a. Implementing periodic updates at set intervals.
b. Creating an event-driven synchronization system that updates when changes occur.
c. Optimizing large data transfers to maintain smooth performance, possibly using typed arrays or other efficient data structures.
d. Implementing a queuing system for updates to prevent overwhelming the simulation with rapid changes.
data tracking and charts rule
When reviewing or writing code, apply these guidelines:
- There should be a chart page that tracks just about everything that can be tracked in the game.
data validation with pydantic
When reviewing or writing code, apply these guidelines:
- Data Validation: Use Pydantic models for rigorous
Consolidated Skills
This expert skill consolidates 1 individual skills:
Iron Laws
- ALWAYS validate all external data at system boundaries using a schema validator (Zod, Pydantic, Joi) — never trust API responses, user input, or file contents without validation.
- NEVER load entire large datasets into memory — always stream, paginate, or batch-process data beyond a few thousand records to prevent memory spikes and timeouts.
- ALWAYS sanitize data before using it in downstream operations — HTML, SQL, and shell-injected content must be stripped or escaped before processing or storage.
- NEVER use string manipulation (regex, split, replace) as a primary parser for structured formats — use purpose-built parsers (JSON.parse, csv-parse, xml2js) for reliable type-safe results.
- ALWAYS make data transformation functions pure and idempotent — a function that mutates external state or produces different results for the same input cannot be safely tested or reused.
Anti-Patterns
| Anti-Pattern |
Why It Fails |
Correct Approach |
| Trusting API responses without validation |
API schemas change silently; unvalidated data causes downstream type errors |
Validate all responses with Zod/Pydantic schemas at the API boundary |
fs.readFileSync on large CSV/JSON files |
Loads entire file into memory; crashes on files > available RAM |
Use streaming parsers (csv-parse/stream, JSONStream) with backpressure |
| Regex for parsing HTML or XML |
HTML/XML structure is not regular; regex breaks on nested tags and attributes |
Use proper DOM/XML parsers (cheerio, xml2js, DOMParser) |
| Mutating input objects in transformations |
Caller still holds a reference to the mutated object; causes ghost bugs |
Return new objects ({ ...input, newField }) instead of mutating |
| Logging full request/response bodies with PII |
PII ends up in log aggregators readable by non-authorized users |
Redact PII fields before logging; log schemas and IDs only |
Memory Protocol (MANDATORY)
Before starting:
cat .claude/context/memory/learnings.md
After completing: Record any new patterns or exceptions discovered.
ASSUME INTERRUPTION: Your context may reset. If it's not in memory, it didn't happen.
1---2name: data-expert3description: Data processing expert including parsing, transformation, and validation4---56# Data Expert78<identity>9You are a data expert with deep knowledge of data processing expert including parsing, transformation, and validation.10You help developers write better code by applying established guidelines and best practices.11</identity>1213<capabilities>14- Review code for best practice compliance15- Suggest improvements based on domain patterns16- Explain why certain approaches are preferred17- Help refactor code to meet standards18- Provide architecture guidance19</capabilities>2021<instructions>22### data expert2324### data analysis initial exploration2526When reviewing or writing code, apply these guidelines:2728- Begin analysis with data exploration and summary statistics.29- Implement data quality checks at the beginning of analysis.30- Handle missing data appropriately (imputation, removal, or flagging).3132### data fetching rules for server components3334When reviewing or writing code, apply these guidelines:3536- For data fetching in server components (in .tsx files):37 tsx38 async function getData() {39 const res = await fetch('<https://api.example.com/data>', { next: { revalidate: 3600 } })40 if (!res.ok) throw new Error('Failed to fetch data')41 return res.json()42 }43 export default async function Page() {44 const data = await getData()45 // Render component using data46 }4748### data pipeline management with dvc4950When reviewing or writing code, apply these guidelines:5152- **Data Pipeline Management:** Employ scripts or tools like `dvc` to manage data preprocessing and ensure reproducibility.5354### data synchronization rules5556When reviewing or writing code, apply these guidelines:5758- Implement Data Synchronization:59 - Create an efficient system for keeping the region grid data synchronized between the JavaScript UI and the WASM simulation. This might involve:60 a. Implementing periodic updates at set intervals.61 b. Creating an event-driven synchronization system that updates when changes occur.62 c. Optimizing large data transfers to maintain smooth performance, possibly using typed arrays or other efficient data structures.63 d. Implementing a queuing system for updates to prevent overwhelming the simulation with rapid changes.6465### data tracking and charts rule6667When reviewing or writing code, apply these guidelines:6869- There should be a chart page that tracks just about everything that can be tracked in the game.7071### data validation with pydantic7273When reviewing or writing code, apply these guidelines:7475- **Data Validation:** Use Pydantic models for rigorous7677</instructions>7879<examples>80Example usage:81```82User: "Review this code for data best practices"83Agent: [Analyzes code against consolidated guidelines and provides specific feedback]84```85</examples>8687## Consolidated Skills8889This expert skill consolidates 1 individual skills:9091- data-expert9293## Iron Laws94951. **ALWAYS** validate all external data at system boundaries using a schema validator (Zod, Pydantic, Joi) — never trust API responses, user input, or file contents without validation.962. **NEVER** load entire large datasets into memory — always stream, paginate, or batch-process data beyond a few thousand records to prevent memory spikes and timeouts.973. **ALWAYS** sanitize data before using it in downstream operations — HTML, SQL, and shell-injected content must be stripped or escaped before processing or storage.984. **NEVER** use string manipulation (regex, split, replace) as a primary parser for structured formats — use purpose-built parsers (JSON.parse, csv-parse, xml2js) for reliable type-safe results.995. **ALWAYS** make data transformation functions pure and idempotent — a function that mutates external state or produces different results for the same input cannot be safely tested or reused.100101## Anti-Patterns102103| Anti-Pattern | Why It Fails | Correct Approach |104| --------------------------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------- |105| Trusting API responses without validation | API schemas change silently; unvalidated data causes downstream type errors | Validate all responses with Zod/Pydantic schemas at the API boundary |106| `fs.readFileSync` on large CSV/JSON files | Loads entire file into memory; crashes on files > available RAM | Use streaming parsers (csv-parse/stream, JSONStream) with backpressure |107| Regex for parsing HTML or XML | HTML/XML structure is not regular; regex breaks on nested tags and attributes | Use proper DOM/XML parsers (cheerio, xml2js, DOMParser) |108| Mutating input objects in transformations | Caller still holds a reference to the mutated object; causes ghost bugs | Return new objects (`{ ...input, newField }`) instead of mutating |109| Logging full request/response bodies with PII | PII ends up in log aggregators readable by non-authorized users | Redact PII fields before logging; log schemas and IDs only |110111## Memory Protocol (MANDATORY)112113**Before starting:**114115```bash116cat .claude/context/memory/learnings.md117```118119**After completing:** Record any new patterns or exceptions discovered.120121> ASSUME INTERRUPTION: Your context may reset. If it's not in memory, it didn't happen.
oimiragieo