Ddia

Designing Data-Intensive Applications Skill

Reference for distributed systems and data architecture concepts from Martin Kleppmann's "Designing Data-Intensive Applications."

Activation Triggers

Use this skill when discussing:

• Database selection and data modeling
• Replication and high availability
• Partitioning/sharding strategies
• Distributed transactions
• Consistency models and guarantees
• Stream vs batch processing
• Event sourcing and CQRS

Quick Reference

Data Models

Storage Engines

Replication Strategies

Partitioning Strategies

Consistency Models

Transaction Isolation Levels

CAP Theorem

"In the presence of a network partition, choose Consistency OR Availability."

Batch vs Stream Processing

Directory Structure

ddia/
├── SKILL.md
├── data-models/
│   ├── relational.md
│   ├── document.md
│   └── graph.md
├── storage/
│   ├── b-trees.md
│   ├── lsm-trees.md
│   └── column-storage.md
├── replication/
│   ├── leader-follower.md
│   ├── multi-leader.md
│   └── leaderless.md
├── partitioning/
│   ├── strategies.md
│   └── rebalancing.md
├── transactions/
│   ├── acid.md
│   ├── isolation-levels.md
│   └── distributed-transactions.md
├── consistency/
│   ├── models.md
│   └── linearizability.md
├── consensus/
│   └── algorithms.md
└── processing/
    ├── batch.md
    ├── stream.md
    └── event-sourcing.md

Usage Examples

Choosing a Database

Question: "Should I use PostgreSQL or MongoDB?"

Consider:
- Data relationships → See data-models/
- Query patterns → See storage/
- Scale requirements → See partitioning/
- Consistency needs → See consistency/

Designing for Scale

Question: "How do I handle millions of users?"

Consider:
- Read scaling → See replication/leader-follower.md
- Write scaling → See partitioning/strategies.md
- Geographic distribution → See replication/multi-leader.md

Handling Failures

Question: "What happens when a node fails?"

Consider:
- Data durability → See replication/
- Consistency trade-offs → See consistency/models.md
- Recovery → See consensus/algorithms.md

Based on concepts from "Designing Data-Intensive Applications" by Martin Kleppmann.