cloudthinker-ai Data Quality Assessment
Phase 1: Scope & Profiling
- Define assessment scope
- Tables/datasets to assess
- Critical fields per dataset
- Business rules and constraints
- Expected data volumes and update frequency
- Downstream consumers and their requirements
- Run data profiling
- Row counts and growth trends
- Column-level statistics (null rate, distinct values, min/max)
- Data type distribution
- Value frequency analysis for categorical fields
- Pattern analysis for text fields
Data Profile Summary
| Table | Rows | Columns | Null Rate (avg) | Last Updated | Update Frequency |
|---|---|---|---|---|---|
| % | hourly/daily/etc |
Phase 2: Quality Dimension Assessment
Completeness (required fields populated)
| Table | Field | Expected | Actual | Null Rate | Score |
|---|---|---|---|---|---|
| 100% | % | /100 |
Accuracy (values reflect reality)
| Table | Field | Validation Rule | Pass Rate | Sample Failures | Score |
|---|---|---|---|---|---|
| % | /100 |
Consistency (data matches across systems)
| Field | Source A | Source B | Match Rate | Discrepancy Count | Score |
|---|---|---|---|---|---|
| % | /100 |
Timeliness (data available when needed)
| Dataset | Expected Freshness | Actual Freshness | SLA Met | Score |
|---|---|---|---|---|
| < hours | hours | Yes/No | /100 |
Uniqueness (no unintended duplicates)
| Table | Key Fields | Total Rows | Duplicate Rows | Duplicate Rate | Score |
|---|---|---|---|---|---|
| % | /100 |
Validity (values conform to rules)
| Table | Field | Rule | Valid Rate | Invalid Examples | Score |
|---|---|---|---|---|---|
| format/range/enum | % | /100 |
Phase 3: Issue Prioritization
- Quantify business impact per issue
- Revenue impact (incorrect pricing, missed orders)
- Operational impact (failed processes, manual workarounds)
- Compliance risk (regulatory data requirements)
- Analytics impact (incorrect insights, poor ML models)
- Customer impact (wrong communications, data errors)
- Prioritize by impact and fixability
Issue Priority Matrix
| Issue | Dimension | Affected Records | Business Impact | Root Cause | Fix Effort | Priority |
|---|---|---|---|---|---|---|
| High/Med/Low | Low/Med/High | 1-5 |
Phase 4: Root Cause Analysis
- Investigate top quality issues
- Source system data entry errors
- ETL transformation bugs
- Missing validation rules at ingestion
- Schema evolution without migration
- Integration failures dropping or corrupting data
- Stale reference data
- Document root cause per issue
Phase 5: Remediation Plan
- Fix existing data quality issues
- Implement preventive controls
- Input validation at source
- Schema enforcement in pipelines
- Data contracts between producers and consumers
- Automated quality checks in ETL/ELT pipelines
- Referential integrity enforcement
- Set up data quality monitoring
Phase 6: Automated Quality Monitoring
- Implement continuous quality checks
- Automated quality tests in data pipelines (dbt tests, Great Expectations)
- Quality score dashboards per domain
- Anomaly detection for quality metrics
- Alerting when quality drops below threshold
- Quality trend reporting (weekly/monthly)
- Define quality SLAs per domain
Counter-Rationalizations
| Shortcut | Counter | Why |
|---|---|---|
| "We can skip some steps for this case" | Adapt the workflow steps, don't skip them | Skipped steps are where incidents and oversights originate |
| "The user seems to already know what to do" | Complete all workflow phases with the user | The workflow catches blind spots that experience alone misses |
| "This is a minor case, full process is overkill" | Scale the process down, don't turn it off | Minor cases become major when unstructured; the process scales, not disappears |
| "I'll fill in the details later" | Complete each section before moving on | Deferred details are forgotten; real-time capture is more accurate |
| "The template output isn't necessary" | Always produce the structured output format | Structured output enables comparison, audit trails, and handoff to other teams |
Output Format
- Data Profile Report: Statistical profile of all assessed datasets
- Quality Scorecard: Score per dimension per dataset
- Issue Inventory: All issues with severity and root cause
- Remediation Plan: Fixes and preventive controls with timelines
- Monitoring Configuration: Automated quality checks and alerts
Action Items
- Run data profiling on all in-scope datasets
- Assess quality across all six dimensions
- Prioritize issues by business impact
- Investigate and document root causes
- Implement data fixes and preventive controls
- Deploy automated quality monitoring
- Establish monthly quality review cadence