Data Strategy
Required Inputs
- Organization: Company name and industry.
- Current State: Brief description of existing data landscape (tools, team, maturity).
- Strategic Objectives: Business goals the data strategy must support.
- Scope: Enterprise-wide, business unit, or specific domain.
- Time Horizon: Strategy period (default: 3 years with 90-day quick wins).
- Constraints: Budget range, regulatory requirements, technology preferences.
Execution Steps
1. Data Maturity Assessment
Assess maturity across five pillars using the standard 1-5 maturity model. Score each sub-dimension and calculate pillar averages.
Pillar 1: Data Governance (Weight: 25%)
| Sub-Dimension |
Level 1: Ad Hoc |
Level 3: Defined |
Level 5: Optimizing |
Score |
| Ownership & Accountability |
No data owners |
Domain owners assigned |
Federated ownership with stewardship council |
|
| Policies & Standards |
No policies |
Documented policies for key domains |
Automated policy enforcement, continuous review |
|
| Metadata Management |
No metadata catalog |
Central catalog with manual updates |
Automated metadata harvesting, lineage, active catalog |
|
| Data Lifecycle Management |
No retention rules |
Defined retention policies |
Automated lifecycle with archival and purging |
|
| Change Management |
Ad hoc schema changes |
Change request process |
Versioned schemas, backward compatibility enforcement |
|
| Pillar Average |
|
|
|
/5.0 |
Pillar 2: Data Architecture (Weight: 25%)
| Sub-Dimension |
Level 1: Ad Hoc |
Level 3: Defined |
Level 5: Optimizing |
Score |
| Integration Patterns |
Point-to-point, manual ETL |
Central ETL/ELT, some real-time |
Event-driven, streaming, self-serve integration |
|
| Storage Architecture |
Siloed databases, spreadsheets |
Central warehouse or lake |
Lakehouse or mesh with governed self-serve access |
|
| Data Modeling |
No standards |
Dimensional models, documented schemas |
Domain-driven design, semantic layer, versioned models |
|
| API & Access Layer |
Direct database access |
REST APIs for some systems |
GraphQL/REST APIs, data products, self-serve portal |
|
| Scalability |
Manual scaling, capacity limits |
Auto-scaling for some workloads |
Elastic, serverless, cost-optimized at scale |
|
| Pillar Average |
|
|
|
/5.0 |
Pillar 3: Data Quality (Weight: 20%)
| Sub-Dimension |
Level 1: Ad Hoc |
Level 3: Defined |
Level 5: Optimizing |
Score |
| Quality Monitoring |
No monitoring |
Scheduled quality checks on key tables |
Real-time quality monitoring with anomaly detection |
|
| Quality Rules |
No rules defined |
Rules for critical data elements |
Comprehensive rules, automated remediation |
|
| Data Profiling |
None |
Periodic profiling |
Continuous profiling with trend analysis |
|
| Issue Resolution |
Reactive, manual fixes |
Defined escalation process |
Automated root cause analysis and remediation |
|
| Quality Metrics |
No measurement |
DQ dashboards for key domains |
SLA-based quality targets with executive visibility |
|
| Pillar Average |
|
|
|
/5.0 |
Pillar 4: Analytics & Insights (Weight: 20%)
| Sub-Dimension |
Level 1: Ad Hoc |
Level 3: Defined |
Level 5: Optimizing |
Score |
| Reporting |
Spreadsheet-based, manual |
BI platform, standard dashboards |
Self-serve analytics, embedded analytics |
|
| Analytics Depth |
Descriptive only |
Diagnostic analytics, root cause |
Predictive and prescriptive analytics at scale |
|
| Data Science Capability |
None |
Small team, ad hoc projects |
Industrialized ML, MLOps, model governance |
|
| Decision Integration |
Reports ignored or delayed |
Data informs some decisions |
Data-driven culture, real-time decision support |
|
| Experimentation |
No A/B testing |
Some experimentation |
Systematic experimentation, feature flagging |
|
| Pillar Average |
|
|
|
/5.0 |
Pillar 5: Data Culture & Literacy (Weight: 10%)
| Sub-Dimension |
Level 1: Ad Hoc |
Level 3: Defined |
Level 5: Optimizing |
Score |
| Data Literacy |
Low; spreadsheet-only skills |
Training programs, data champions |
Organization-wide fluency, data in every role |
|
| Self-Service Access |
IT bottleneck for all requests |
Business users access standard reports |
Governed self-serve with guardrails |
|
| Data-Driven Decisions |
Gut-feel dominant |
Data referenced in some decisions |
Data required for all major decisions |
|
| Community of Practice |
None |
Informal knowledge sharing |
Formal CoP, internal conferences, shared best practices |
|
| Executive Sponsorship |
No data executive |
CDO or equivalent exists |
Data strategy integrated with business strategy |
|
| Pillar Average |
|
|
|
/5.0 |
Overall Data Maturity Score:
Overall Score = (Governance x 0.25) + (Architecture x 0.25) + (Quality x 0.20) +
(Analytics x 0.20) + (Culture x 0.10)
2. Data Governance Framework Design
Governance Operating Model:
| Role |
Responsibility |
Accountability Level |
| Data Governance Council |
Strategic direction, policy approval, dispute resolution |
Executive (CDO, CIO, business leaders) |
| Data Domain Owners |
Domain-level data strategy, quality targets, access decisions |
VP / Director per business domain |
| Data Stewards |
Day-to-day quality, metadata, issue resolution |
Manager / Senior IC per domain |
| Data Engineers |
Pipeline quality, schema management, technical implementation |
Engineering team |
| Data Consumers |
Adherence to policies, issue reporting, feedback |
All data users |
Core Governance Policies (prioritize by maturity level):
| Policy |
Priority at Level 1-2 |
Priority at Level 3-4 |
Priority at Level 5 |
| Data classification (public, internal, confidential, restricted) |
Critical |
Maintain |
Optimize |
| Access control and provisioning |
Critical |
Maintain |
Automate |
| Data retention and deletion |
High |
Maintain |
Automate |
| Data quality standards per domain |
High |
Critical |
Optimize |
| Metadata and documentation standards |
Medium |
Critical |
Automate |
| Master data management |
Medium |
High |
Optimize |
| Data sharing and external data use |
Medium |
High |
Critical |
| Privacy and consent management |
Critical (if regulated) |
Maintain |
Automate |
Metadata Management Framework:
- Business metadata: Definitions, ownership, classifications, business rules
- Technical metadata: Schemas, lineage, transformations, freshness, row counts
- Operational metadata: Job runs, SLAs, quality scores, access logs
- Tool selection: Evaluate data catalog solutions (see System Selection skill for vendor evaluation)
3. Data Architecture Design
Architecture Pattern Selection:
| Pattern |
Best For |
Avoid When |
Complexity |
Cost Profile |
| Data Warehouse |
Structured analytics, BI reporting, known query patterns |
Unstructured data, ML workloads, schema evolution |
Low-Medium |
Predictable |
| Data Lake |
Raw data storage, ML, diverse data types |
Governance-heavy environments (risk of data swamp) |
Medium |
Storage-cheap, compute-variable |
| Data Lakehouse |
Unified analytics + ML, ACID transactions on lake |
Very small data volumes, simple BI-only needs |
Medium-High |
Moderate |
| Data Mesh |
Large orgs with autonomous domains, federated ownership |
Small teams, centralized culture, early maturity |
High |
Distributed |
Architecture Decision Framework:
| Factor |
Weight |
Warehouse |
Lake |
Lakehouse |
Mesh |
Weighted Score |
| Data variety (structured vs. unstructured) |
15% |
|
|
|
|
|
| Analytics workload types (BI vs. ML vs. both) |
15% |
|
|
|
|
|
| Team size and autonomy |
15% |
|
|
|
|
|
| Governance requirements |
15% |
|
|
|
|
|
| Real-time requirements |
10% |
|
|
|
|
|
| Cost sensitivity |
10% |
|
|
|
|
|
| Existing technology stack fit |
10% |
|
|
|
|
|
| Scalability needs (data volume growth) |
10% |
|
|
|
|
|
| Weighted Total |
100% |
|
|
|
|
|
Reference Architecture Components:
Data Sources → Ingestion → Storage → Processing → Serving → Consumption
| | | | | |
APIs Batch ETL Raw Zone Transform Data API BI Tools
Databases Streaming Curated ML Training Semantic Notebooks
SaaS Apps CDC Archive Feature Eng Cache Applications
Files/IoT Event Bus Aggregation Search Dashboards
4. Data Quality Framework
Six Dimensions of Data Quality:
| Dimension |
Definition |
Measurement Method |
Target Threshold |
| Accuracy |
Data correctly reflects the real-world entity |
Validation against source/reference data |
>99% for critical, >95% for standard |
| Completeness |
Required fields populated, no missing records |
Null/empty checks, referential integrity |
>98% for critical, >90% for standard |
| Consistency |
Same data yields same value across systems |
Cross-system reconciliation |
>99% for master data |
| Timeliness |
Data available when needed, within freshness SLA |
Freshness monitoring, lag measurement |
Per-domain SLA (e.g., <1hr, <24hr) |
| Validity |
Data conforms to defined formats and rules |
Schema validation, business rule checks |
>99.5% for critical |
| Uniqueness |
No unintended duplicate records |
Deduplication analysis, entity resolution |
>99% for master data |
Data Quality Scoring Formula:
Domain Quality Score = Sum of (Dimension Score x Dimension Weight) for each dimension
Overall DQ Score = Sum of (Domain Score x Domain Criticality Weight)
Quality Level:
95-100%: Excellent -- maintain and optimize
85-94%: Good -- targeted improvements
70-84%: Fair -- dedicated remediation program needed
<70%: Poor -- critical investment required
Data Quality Implementation Roadmap:
| Phase |
Timeline |
Activities |
Expected Outcome |
| Foundation |
Months 1-3 |
Profile critical data, define rules, establish baselines |
DQ baselines for top 10 data domains |
| Monitoring |
Months 3-6 |
Deploy automated monitoring, alert on anomalies |
Real-time quality visibility |
| Remediation |
Months 6-12 |
Root cause analysis, upstream fixes, process changes |
15-25% improvement in DQ scores |
| Optimization |
Months 12-18 |
Predictive quality, automated remediation |
Self-healing data pipelines |
5. Analytics Capability Roadmap
Analytics Maturity Progression:
| Stage |
Capability |
Key Question Answered |
Technologies |
Typical Timeline |
| Descriptive |
Reporting, dashboards, KPI tracking |
"What happened?" |
BI tools, SQL, data warehouse |
Months 0-6 |
| Diagnostic |
Drill-down, root cause, cohort analysis |
"Why did it happen?" |
Advanced BI, statistical tools |
Months 3-12 |
| Predictive |
Forecasting, propensity models, risk scoring |
"What will happen?" |
ML platforms, Python/R, feature stores |
Months 6-18 |
| Prescriptive |
Optimization, recommendation, automation |
"What should we do?" |
Decision engines, reinforcement learning, real-time ML |
Months 12-36 |
Analytics Use Case Prioritization:
| Use Case |
Business Impact (1-5) |
Data Readiness (1-5) |
Technical Feasibility (1-5) |
Priority Score |
Stage |
|
|
|
|
Impact x Readiness x Feasibility |
|
6. Data Lineage and Impact Analysis
Lineage Coverage Requirements:
| Level |
Scope |
Purpose |
| Source-to-target |
End-to-end data flow from origin to consumption |
Impact analysis for source changes |
| Column-level |
Field-level transformations and derivations |
Root cause analysis for quality issues |
| Business-level |
Business process to report/KPI mapping |
Business impact assessment |
7. Privacy and Compliance Integration
Privacy Framework Mapping:
| Requirement |
GDPR |
CCPA/CPRA |
HIPAA |
Industry-Specific |
Implementation |
| Data inventory/mapping |
Required |
Required |
Required |
Varies |
Automated discovery tools |
| Consent management |
Required |
Required |
N/A |
Varies |
Consent platform |
| Right to deletion |
Required |
Required |
Limited |
Varies |
Deletion pipeline |
| Data minimization |
Required |
Recommended |
Required |
Varies |
Retention automation |
| Breach notification |
72 hours |
"Without unreasonable delay" |
60 days |
Varies |
Incident response plan |
| Cross-border transfer |
Restricted |
Limited |
Restricted |
Varies |
Data residency controls |
| Privacy impact assessment |
Required for high risk |
Recommended |
Required |
Varies |
PIA process |
8. Data Monetization Opportunity Assessment
Monetization Models:
| Model |
Description |
Revenue Type |
Risk Level |
Data Maturity Required |
| Internal optimization |
Better decisions from better data |
Cost savings, efficiency |
Low |
Level 2+ |
| Product enhancement |
Data-powered features for customers |
Product differentiation |
Low-Medium |
Level 3+ |
| Data-as-a-Service |
Sell data or insights to third parties |
New revenue stream |
Medium-High |
Level 4+ |
| Data marketplace |
Platform for data exchange |
Platform revenue |
High |
Level 5 |
| Embedded analytics |
Analytics built into customer-facing products |
Premium pricing |
Medium |
Level 3+ |
9. Data Team Organizational Model
Model Comparison:
| Model |
Description |
Best For |
Risks |
| Centralized |
Single data team serves all domains |
Early maturity (Level 1-2), small orgs |
Bottleneck, disconnected from business |
| Embedded |
Data people sit within business units |
Strong domain alignment needed |
Inconsistent standards, duplication |
| Hub-and-Spoke |
Central CoE + embedded analysts |
Medium maturity (Level 3), growing orgs |
Coordination overhead |
| Data Mesh |
Domain-owned data products, platform team |
High maturity (Level 4-5), large orgs |
Requires mature governance, skilled domains |
Recommended Team Sizing (by data maturity):
| Maturity Level |
Data Team : Total Employee Ratio |
Key Roles to Hire First |
| Level 1-2 |
1:200 - 1:100 |
Data engineer, analytics engineer, data analyst |
| Level 3 |
1:80 - 1:50 |
+ Data governance lead, data platform engineer |
| Level 4 |
1:40 - 1:25 |
+ Data scientists, ML engineers, data product managers |
| Level 5 |
1:20 - 1:15 |
+ AI/ML specialists, data mesh platform team |
10. Build vs. Buy Decision Framework for Data Platforms
Evaluation Criteria:
| Factor |
Weight |
Build |
Buy (Commercial) |
Buy (Open Source + Managed) |
Score |
| Time to value |
20% |
Slow (6-18 months) |
Fast (1-3 months) |
Medium (2-6 months) |
|
| Customization |
15% |
Full control |
Limited to vendor roadmap |
Moderate flexibility |
|
| Total cost (3 year) |
20% |
High labor, low license |
Predictable, potentially high |
Moderate |
|
| Operational burden |
15% |
High |
Low (SaaS) |
Medium |
|
| Talent availability |
10% |
Specialized skills needed |
Lower barrier |
Community skills available |
|
| Vendor lock-in risk |
10% |
None |
High |
Low-Medium |
|
| Innovation pace |
10% |
Depends on team |
Vendor roadmap |
Community velocity |
|
| Weighted Total |
100% |
|
|
|
|
Output Template
# Data Strategy: [Organization Name]
**Prepared for**: [Stakeholder] | **Date**: [Date] | **Horizon**: [X]-Year Strategy
## Executive Summary
**Overall Data Maturity Score**: [X.X] / 5.0 -- Level [N]: [Ad Hoc / Managed / Defined / Quantified / Optimizing]
[3-5 sentence synthesis: where the organization is today, where it needs to be, and
the critical investments required to close the gap.]
### Maturity Snapshot
| Pillar | Current Score | Target Score (Year 1) | Target Score (Year 3) |
|---|---|---|---|
| Governance | X.X | X.X | X.X |
| Architecture | X.X | X.X | X.X |
| Quality | X.X | X.X | X.X |
| Analytics | X.X | X.X | X.X |
| Culture | X.X | X.X | X.X |
| **Overall** | **X.X** | **X.X** | **X.X** |
### Estimated Investment: $[X]M - $[Y]M over [N] years
## 1. Data Governance Framework
[Governance operating model, roles, priority policies]
## 2. Data Architecture Recommendation
**Recommended Pattern**: [Warehouse / Lake / Lakehouse / Mesh]
[Architecture design, technology selections, migration plan]
## 3. Data Quality Program
**Current Overall DQ Score**: [X]%
[Quality framework, priority dimensions, improvement targets]
## 4. Analytics Roadmap
**Current Analytics Stage**: [Descriptive / Diagnostic / Predictive / Prescriptive]
[Capability build plan, prioritized use cases, technology needs]
## 5. Privacy & Compliance
[Applicable regulations, gap analysis, remediation plan]
## 6. Data Organization & Talent
**Recommended Model**: [Centralized / Hub-and-Spoke / Embedded / Mesh]
[Team structure, hiring plan, capability development]
## 7. Data Monetization Opportunities
[Assessed opportunities with feasibility and timeline]
## Implementation Roadmap
### Phase 1: Foundation (Months 1-6)
- [Key initiative with owner, cost, and success metric]
### Phase 2: Scale (Months 7-18)
- [Key initiative with owner, cost, and success metric]
### Phase 3: Optimize (Months 19-36)
- [Key initiative with owner, cost, and success metric]
### Quick Wins (First 90 Days)
1. [Quick win with expected impact]
2. [Quick win with expected impact]
3. [Quick win with expected impact]
## Investment Summary
| Category | Year 1 | Year 2 | Year 3 | Total |
|---|---|---|---|---|
| Technology (platforms, tools) | | | | |
| People (hiring, training) | | | | |
| Process (governance, quality) | | | | |
| External (consultants, vendors) | | | | |
| **Total** | | | | |
## Risk & Dependencies
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| [Risk description] | H/M/L | H/M/L | [Mitigation action] |
## Success Metrics
| Metric | Baseline | Year 1 Target | Year 3 Target |
|---|---|---|---|
| Data maturity score | X.X | X.X | X.X |
| Data quality score (overall) | X% | X% | X% |
| Time to insight (new data request) | X days | X days | X hours |
| Self-service adoption rate | X% | X% | X% |
| Analytics ROI (documented value) | $0 | $XM | $XM |
Quality Checks
1---2name: data-strategy3description: Comprehensive data strategy covering governance, architecture, quality, analytics, and organizational culture. USE THIS SKILL when the user asks about data governance frameworks, data architecture design, analytics roadmaps, data platform selection, data quality improvement, data mesh vs data warehouse decisions, data catalog implementation, data monetization, or building a data-driven organization. Covers the full stack from governance to culture with actionable maturity assessments.4---56# Data Strategy78## Required Inputs910- **Organization**: Company name and industry.11- **Current State**: Brief description of existing data landscape (tools, team, maturity).12- **Strategic Objectives**: Business goals the data strategy must support.13- **Scope**: Enterprise-wide, business unit, or specific domain.14- **Time Horizon**: Strategy period (default: 3 years with 90-day quick wins).15- **Constraints**: Budget range, regulatory requirements, technology preferences.1617## Execution Steps1819### 1. Data Maturity Assessment2021Assess maturity across five pillars using the standard 1-5 maturity model. Score each sub-dimension and calculate pillar averages.2223**Pillar 1: Data Governance (Weight: 25%)**2425| Sub-Dimension | Level 1: Ad Hoc | Level 3: Defined | Level 5: Optimizing | Score |26|---|---|---|---|---|27| Ownership & Accountability | No data owners | Domain owners assigned | Federated ownership with stewardship council | |28| Policies & Standards | No policies | Documented policies for key domains | Automated policy enforcement, continuous review | |29| Metadata Management | No metadata catalog | Central catalog with manual updates | Automated metadata harvesting, lineage, active catalog | |30| Data Lifecycle Management | No retention rules | Defined retention policies | Automated lifecycle with archival and purging | |31| Change Management | Ad hoc schema changes | Change request process | Versioned schemas, backward compatibility enforcement | |32| **Pillar Average** | | | | **/5.0** |3334**Pillar 2: Data Architecture (Weight: 25%)**3536| Sub-Dimension | Level 1: Ad Hoc | Level 3: Defined | Level 5: Optimizing | Score |37|---|---|---|---|---|38| Integration Patterns | Point-to-point, manual ETL | Central ETL/ELT, some real-time | Event-driven, streaming, self-serve integration | |39| Storage Architecture | Siloed databases, spreadsheets | Central warehouse or lake | Lakehouse or mesh with governed self-serve access | |40| Data Modeling | No standards | Dimensional models, documented schemas | Domain-driven design, semantic layer, versioned models | |41| API & Access Layer | Direct database access | REST APIs for some systems | GraphQL/REST APIs, data products, self-serve portal | |42| Scalability | Manual scaling, capacity limits | Auto-scaling for some workloads | Elastic, serverless, cost-optimized at scale | |43| **Pillar Average** | | | | **/5.0** |4445**Pillar 3: Data Quality (Weight: 20%)**4647| Sub-Dimension | Level 1: Ad Hoc | Level 3: Defined | Level 5: Optimizing | Score |48|---|---|---|---|---|49| Quality Monitoring | No monitoring | Scheduled quality checks on key tables | Real-time quality monitoring with anomaly detection | |50| Quality Rules | No rules defined | Rules for critical data elements | Comprehensive rules, automated remediation | |51| Data Profiling | None | Periodic profiling | Continuous profiling with trend analysis | |52| Issue Resolution | Reactive, manual fixes | Defined escalation process | Automated root cause analysis and remediation | |53| Quality Metrics | No measurement | DQ dashboards for key domains | SLA-based quality targets with executive visibility | |54| **Pillar Average** | | | | **/5.0** |5556**Pillar 4: Analytics & Insights (Weight: 20%)**5758| Sub-Dimension | Level 1: Ad Hoc | Level 3: Defined | Level 5: Optimizing | Score |59|---|---|---|---|---|60| Reporting | Spreadsheet-based, manual | BI platform, standard dashboards | Self-serve analytics, embedded analytics | |61| Analytics Depth | Descriptive only | Diagnostic analytics, root cause | Predictive and prescriptive analytics at scale | |62| Data Science Capability | None | Small team, ad hoc projects | Industrialized ML, MLOps, model governance | |63| Decision Integration | Reports ignored or delayed | Data informs some decisions | Data-driven culture, real-time decision support | |64| Experimentation | No A/B testing | Some experimentation | Systematic experimentation, feature flagging | |65| **Pillar Average** | | | | **/5.0** |6667**Pillar 5: Data Culture & Literacy (Weight: 10%)**6869| Sub-Dimension | Level 1: Ad Hoc | Level 3: Defined | Level 5: Optimizing | Score |70|---|---|---|---|---|71| Data Literacy | Low; spreadsheet-only skills | Training programs, data champions | Organization-wide fluency, data in every role | |72| Self-Service Access | IT bottleneck for all requests | Business users access standard reports | Governed self-serve with guardrails | |73| Data-Driven Decisions | Gut-feel dominant | Data referenced in some decisions | Data required for all major decisions | |74| Community of Practice | None | Informal knowledge sharing | Formal CoP, internal conferences, shared best practices | |75| Executive Sponsorship | No data executive | CDO or equivalent exists | Data strategy integrated with business strategy | |76| **Pillar Average** | | | | **/5.0** |7778**Overall Data Maturity Score:**7980```81Overall Score = (Governance x 0.25) + (Architecture x 0.25) + (Quality x 0.20) +82 (Analytics x 0.20) + (Culture x 0.10)83```8485### 2. Data Governance Framework Design8687**Governance Operating Model:**8889| Role | Responsibility | Accountability Level |90|---|---|---|91| **Data Governance Council** | Strategic direction, policy approval, dispute resolution | Executive (CDO, CIO, business leaders) |92| **Data Domain Owners** | Domain-level data strategy, quality targets, access decisions | VP / Director per business domain |93| **Data Stewards** | Day-to-day quality, metadata, issue resolution | Manager / Senior IC per domain |94| **Data Engineers** | Pipeline quality, schema management, technical implementation | Engineering team |95| **Data Consumers** | Adherence to policies, issue reporting, feedback | All data users |9697**Core Governance Policies (prioritize by maturity level):**9899| Policy | Priority at Level 1-2 | Priority at Level 3-4 | Priority at Level 5 |100|---|---|---|---|101| Data classification (public, internal, confidential, restricted) | Critical | Maintain | Optimize |102| Access control and provisioning | Critical | Maintain | Automate |103| Data retention and deletion | High | Maintain | Automate |104| Data quality standards per domain | High | Critical | Optimize |105| Metadata and documentation standards | Medium | Critical | Automate |106| Master data management | Medium | High | Optimize |107| Data sharing and external data use | Medium | High | Critical |108| Privacy and consent management | Critical (if regulated) | Maintain | Automate |109110**Metadata Management Framework:**111- **Business metadata**: Definitions, ownership, classifications, business rules112- **Technical metadata**: Schemas, lineage, transformations, freshness, row counts113- **Operational metadata**: Job runs, SLAs, quality scores, access logs114- **Tool selection**: Evaluate data catalog solutions (see System Selection skill for vendor evaluation)115116### 3. Data Architecture Design117118**Architecture Pattern Selection:**119120| Pattern | Best For | Avoid When | Complexity | Cost Profile |121|---|---|---|---|---|122| **Data Warehouse** | Structured analytics, BI reporting, known query patterns | Unstructured data, ML workloads, schema evolution | Low-Medium | Predictable |123| **Data Lake** | Raw data storage, ML, diverse data types | Governance-heavy environments (risk of data swamp) | Medium | Storage-cheap, compute-variable |124| **Data Lakehouse** | Unified analytics + ML, ACID transactions on lake | Very small data volumes, simple BI-only needs | Medium-High | Moderate |125| **Data Mesh** | Large orgs with autonomous domains, federated ownership | Small teams, centralized culture, early maturity | High | Distributed |126127**Architecture Decision Framework:**128129| Factor | Weight | Warehouse | Lake | Lakehouse | Mesh | Weighted Score |130|---|---|---|---|---|---|---|131| Data variety (structured vs. unstructured) | 15% | | | | | |132| Analytics workload types (BI vs. ML vs. both) | 15% | | | | | |133| Team size and autonomy | 15% | | | | | |134| Governance requirements | 15% | | | | | |135| Real-time requirements | 10% | | | | | |136| Cost sensitivity | 10% | | | | | |137| Existing technology stack fit | 10% | | | | | |138| Scalability needs (data volume growth) | 10% | | | | | |139| **Weighted Total** | **100%** | | | | | |140141**Reference Architecture Components:**142143```144Data Sources → Ingestion → Storage → Processing → Serving → Consumption145 | | | | | |146 APIs Batch ETL Raw Zone Transform Data API BI Tools147 Databases Streaming Curated ML Training Semantic Notebooks148 SaaS Apps CDC Archive Feature Eng Cache Applications149 Files/IoT Event Bus Aggregation Search Dashboards150```151152### 4. Data Quality Framework153154**Six Dimensions of Data Quality:**155156| Dimension | Definition | Measurement Method | Target Threshold |157|---|---|---|---|158| **Accuracy** | Data correctly reflects the real-world entity | Validation against source/reference data | >99% for critical, >95% for standard |159| **Completeness** | Required fields populated, no missing records | Null/empty checks, referential integrity | >98% for critical, >90% for standard |160| **Consistency** | Same data yields same value across systems | Cross-system reconciliation | >99% for master data |161| **Timeliness** | Data available when needed, within freshness SLA | Freshness monitoring, lag measurement | Per-domain SLA (e.g., <1hr, <24hr) |162| **Validity** | Data conforms to defined formats and rules | Schema validation, business rule checks | >99.5% for critical |163| **Uniqueness** | No unintended duplicate records | Deduplication analysis, entity resolution | >99% for master data |164165**Data Quality Scoring Formula:**166167```168Domain Quality Score = Sum of (Dimension Score x Dimension Weight) for each dimension169Overall DQ Score = Sum of (Domain Score x Domain Criticality Weight)170171Quality Level:172 95-100%: Excellent -- maintain and optimize173 85-94%: Good -- targeted improvements174 70-84%: Fair -- dedicated remediation program needed175 <70%: Poor -- critical investment required176```177178**Data Quality Implementation Roadmap:**179180| Phase | Timeline | Activities | Expected Outcome |181|---|---|---|---|182| **Foundation** | Months 1-3 | Profile critical data, define rules, establish baselines | DQ baselines for top 10 data domains |183| **Monitoring** | Months 3-6 | Deploy automated monitoring, alert on anomalies | Real-time quality visibility |184| **Remediation** | Months 6-12 | Root cause analysis, upstream fixes, process changes | 15-25% improvement in DQ scores |185| **Optimization** | Months 12-18 | Predictive quality, automated remediation | Self-healing data pipelines |186187### 5. Analytics Capability Roadmap188189**Analytics Maturity Progression:**190191| Stage | Capability | Key Question Answered | Technologies | Typical Timeline |192|---|---|---|---|---|193| **Descriptive** | Reporting, dashboards, KPI tracking | "What happened?" | BI tools, SQL, data warehouse | Months 0-6 |194| **Diagnostic** | Drill-down, root cause, cohort analysis | "Why did it happen?" | Advanced BI, statistical tools | Months 3-12 |195| **Predictive** | Forecasting, propensity models, risk scoring | "What will happen?" | ML platforms, Python/R, feature stores | Months 6-18 |196| **Prescriptive** | Optimization, recommendation, automation | "What should we do?" | Decision engines, reinforcement learning, real-time ML | Months 12-36 |197198**Analytics Use Case Prioritization:**199200| Use Case | Business Impact (1-5) | Data Readiness (1-5) | Technical Feasibility (1-5) | Priority Score | Stage |201|---|---|---|---|---|---|202| | | | | Impact x Readiness x Feasibility | |203204### 6. Data Lineage and Impact Analysis205206**Lineage Coverage Requirements:**207208| Level | Scope | Purpose |209|---|---|---|210| **Source-to-target** | End-to-end data flow from origin to consumption | Impact analysis for source changes |211| **Column-level** | Field-level transformations and derivations | Root cause analysis for quality issues |212| **Business-level** | Business process to report/KPI mapping | Business impact assessment |213214### 7. Privacy and Compliance Integration215216**Privacy Framework Mapping:**217218| Requirement | GDPR | CCPA/CPRA | HIPAA | Industry-Specific | Implementation |219|---|---|---|---|---|---|220| Data inventory/mapping | Required | Required | Required | Varies | Automated discovery tools |221| Consent management | Required | Required | N/A | Varies | Consent platform |222| Right to deletion | Required | Required | Limited | Varies | Deletion pipeline |223| Data minimization | Required | Recommended | Required | Varies | Retention automation |224| Breach notification | 72 hours | "Without unreasonable delay" | 60 days | Varies | Incident response plan |225| Cross-border transfer | Restricted | Limited | Restricted | Varies | Data residency controls |226| Privacy impact assessment | Required for high risk | Recommended | Required | Varies | PIA process |227228### 8. Data Monetization Opportunity Assessment229230**Monetization Models:**231232| Model | Description | Revenue Type | Risk Level | Data Maturity Required |233|---|---|---|---|---|234| **Internal optimization** | Better decisions from better data | Cost savings, efficiency | Low | Level 2+ |235| **Product enhancement** | Data-powered features for customers | Product differentiation | Low-Medium | Level 3+ |236| **Data-as-a-Service** | Sell data or insights to third parties | New revenue stream | Medium-High | Level 4+ |237| **Data marketplace** | Platform for data exchange | Platform revenue | High | Level 5 |238| **Embedded analytics** | Analytics built into customer-facing products | Premium pricing | Medium | Level 3+ |239240### 9. Data Team Organizational Model241242**Model Comparison:**243244| Model | Description | Best For | Risks |245|---|---|---|---|246| **Centralized** | Single data team serves all domains | Early maturity (Level 1-2), small orgs | Bottleneck, disconnected from business |247| **Embedded** | Data people sit within business units | Strong domain alignment needed | Inconsistent standards, duplication |248| **Hub-and-Spoke** | Central CoE + embedded analysts | Medium maturity (Level 3), growing orgs | Coordination overhead |249| **Data Mesh** | Domain-owned data products, platform team | High maturity (Level 4-5), large orgs | Requires mature governance, skilled domains |250251**Recommended Team Sizing (by data maturity):**252253| Maturity Level | Data Team : Total Employee Ratio | Key Roles to Hire First |254|---|---|---|255| Level 1-2 | 1:200 - 1:100 | Data engineer, analytics engineer, data analyst |256| Level 3 | 1:80 - 1:50 | + Data governance lead, data platform engineer |257| Level 4 | 1:40 - 1:25 | + Data scientists, ML engineers, data product managers |258| Level 5 | 1:20 - 1:15 | + AI/ML specialists, data mesh platform team |259260### 10. Build vs. Buy Decision Framework for Data Platforms261262**Evaluation Criteria:**263264| Factor | Weight | Build | Buy (Commercial) | Buy (Open Source + Managed) | Score |265|---|---|---|---|---|---|266| Time to value | 20% | Slow (6-18 months) | Fast (1-3 months) | Medium (2-6 months) | |267| Customization | 15% | Full control | Limited to vendor roadmap | Moderate flexibility | |268| Total cost (3 year) | 20% | High labor, low license | Predictable, potentially high | Moderate | |269| Operational burden | 15% | High | Low (SaaS) | Medium | |270| Talent availability | 10% | Specialized skills needed | Lower barrier | Community skills available | |271| Vendor lock-in risk | 10% | None | High | Low-Medium | |272| Innovation pace | 10% | Depends on team | Vendor roadmap | Community velocity | |273| **Weighted Total** | **100%** | | | | |274275## Output Template276277```markdown278# Data Strategy: [Organization Name]279280**Prepared for**: [Stakeholder] | **Date**: [Date] | **Horizon**: [X]-Year Strategy281282## Executive Summary283284**Overall Data Maturity Score**: [X.X] / 5.0 -- Level [N]: [Ad Hoc / Managed / Defined / Quantified / Optimizing]285286[3-5 sentence synthesis: where the organization is today, where it needs to be, and287the critical investments required to close the gap.]288289### Maturity Snapshot290| Pillar | Current Score | Target Score (Year 1) | Target Score (Year 3) |291|---|---|---|---|292| Governance | X.X | X.X | X.X |293| Architecture | X.X | X.X | X.X |294| Quality | X.X | X.X | X.X |295| Analytics | X.X | X.X | X.X |296| Culture | X.X | X.X | X.X |297| **Overall** | **X.X** | **X.X** | **X.X** |298299### Estimated Investment: $[X]M - $[Y]M over [N] years300301## 1. Data Governance Framework302[Governance operating model, roles, priority policies]303304## 2. Data Architecture Recommendation305**Recommended Pattern**: [Warehouse / Lake / Lakehouse / Mesh]306[Architecture design, technology selections, migration plan]307308## 3. Data Quality Program309**Current Overall DQ Score**: [X]%310[Quality framework, priority dimensions, improvement targets]311312## 4. Analytics Roadmap313**Current Analytics Stage**: [Descriptive / Diagnostic / Predictive / Prescriptive]314[Capability build plan, prioritized use cases, technology needs]315316## 5. Privacy & Compliance317[Applicable regulations, gap analysis, remediation plan]318319## 6. Data Organization & Talent320**Recommended Model**: [Centralized / Hub-and-Spoke / Embedded / Mesh]321[Team structure, hiring plan, capability development]322323## 7. Data Monetization Opportunities324[Assessed opportunities with feasibility and timeline]325326## Implementation Roadmap327328### Phase 1: Foundation (Months 1-6)329- [Key initiative with owner, cost, and success metric]330331### Phase 2: Scale (Months 7-18)332- [Key initiative with owner, cost, and success metric]333334### Phase 3: Optimize (Months 19-36)335- [Key initiative with owner, cost, and success metric]336337### Quick Wins (First 90 Days)3381. [Quick win with expected impact]3392. [Quick win with expected impact]3403. [Quick win with expected impact]341342## Investment Summary343344| Category | Year 1 | Year 2 | Year 3 | Total |345|---|---|---|---|---|346| Technology (platforms, tools) | | | | |347| People (hiring, training) | | | | |348| Process (governance, quality) | | | | |349| External (consultants, vendors) | | | | |350| **Total** | | | | |351352## Risk & Dependencies353| Risk | Likelihood | Impact | Mitigation |354|---|---|---|---|355| [Risk description] | H/M/L | H/M/L | [Mitigation action] |356357## Success Metrics358| Metric | Baseline | Year 1 Target | Year 3 Target |359|---|---|---|---|360| Data maturity score | X.X | X.X | X.X |361| Data quality score (overall) | X% | X% | X% |362| Time to insight (new data request) | X days | X days | X hours |363| Self-service adoption rate | X% | X% | X% |364| Analytics ROI (documented value) | $0 | $XM | $XM |365```366367## Quality Checks368369- [ ] All five maturity pillars assessed with sub-dimension scoring and evidence.370- [ ] Overall maturity score calculated using weighted formula.371- [ ] Data governance framework includes roles, policies, and metadata management.372- [ ] Architecture recommendation justified with decision framework scoring.373- [ ] Data quality framework covers all six dimensions with measurement methods.374- [ ] Analytics roadmap maps progression from descriptive to prescriptive.375- [ ] Privacy and compliance requirements mapped to specific regulations.376- [ ] Data team model recommendation aligned with current and target maturity.377- [ ] Build vs. buy analysis completed for major platform decisions.378- [ ] Investment estimate provided as a range with year-by-year breakdown.379- [ ] Implementation roadmap includes quick wins (90 days) and phased execution.380- [ ] Success metrics defined with baselines and targets.381- [ ] All estimates carry confidence levels (High/Medium/Low).382- [ ] Strategy aligns data investments to stated business objectives.
Kaakati