Data Governance
Framework Components
A complete data governance program covers:
- Policies — rules for how data is managed (classification, retention, access)
- Standards — technical specifications (naming conventions, formats, quality thresholds)
- Processes — how governance is executed (request handling, data quality remediation)
- Roles — who owns, stewards, and uses data
- Technology — catalog, lineage, quality tooling
- Metrics — how governance health is measured
Data Classification Tiers
| Tier |
Label |
Examples |
Default handling |
| 1 |
Public |
Press releases, marketing content, public docs |
No controls needed |
| 2 |
Internal |
Internal policies, employee directories, meeting notes |
Internal use only, not for external sharing |
| 3 |
Confidential |
Customer PII, financials, contracts, source code |
Encrypted at rest and in transit; access-controlled |
| 4 |
Restricted |
Health records, payment card data, credentials, trade secrets |
Strict need-to-know; regulatory controls apply |
Classification Decision Tree
Does it contain PII, PHI, PCI, credentials, or trade secrets?
Yes → Restricted
Does it contain customer data, financial data, or internal IP?
Yes → Confidential
Is it intended for internal use only?
Yes → Internal
Otherwise → Public
Data Catalog Structure
A data catalog entry should include:
| Field |
Description |
| Asset name |
Technical name of the table/dataset/API |
| Display name |
Human-readable name |
| Description |
What the asset contains and its purpose |
| Owner |
Business owner (accountable for the data) |
| Steward |
Technical steward (maintains quality and definitions) |
| Classification |
Public / Internal / Confidential / Restricted |
| System of record |
Authoritative source for this data |
| Consuming systems |
Downstream systems that use this data |
| Tags |
Domain, subject area, regulation tags |
| Last updated |
Date metadata was last verified |
Data Dictionary Template
For each field in a dataset:
| Field name |
Data type |
Description |
Example |
Owner |
PII? |
Nullable |
Source |
| customer_id |
UUID |
Unique identifier for a customer |
550e8400-... |
Data Platform |
No |
No |
CRM |
| email_address |
VARCHAR(255) |
Customer email |
user@example.com |
Marketing |
Yes |
No |
Sign-up form |
| date_of_birth |
DATE |
Customer date of birth (YYYY-MM-DD) |
1985-03-22 |
Compliance |
Yes (Sensitive) |
Yes |
KYC flow |
Data Lineage Documentation
Lineage Levels
- Column-level — tracks individual field transformations (gold standard)
- Table-level — tracks which tables feed which tables
- System-level — tracks data flows between systems
Lineage Capture Methods
| Method |
Effort |
Coverage |
| Manual documentation |
High effort, low accuracy |
Legacy systems |
| ETL metadata parsing |
Medium effort |
ETL-based pipelines |
| Query log analysis |
Medium effort |
SQL-based transformations |
| Automated scanners (OpenMetadata, DataHub, Atlan) |
Low ongoing effort |
Modern stacks |
Minimum Lineage Record
Source: [System/Table/Column]
Transformation: [Description of logic applied]
Destination: [System/Table/Column]
Frequency: [Real-time / Hourly / Daily / Weekly]
Owner: [Team responsible for the pipeline]
Last validated: [Date]
Data Ownership RACI
| Activity |
Data Owner |
Data Steward |
Data Consumer |
Governance Team |
| Define data policies |
A |
C |
I |
R |
| Maintain data dictionary |
I |
A/R |
I |
C |
| Approve access requests |
A |
R |
I |
C |
| Resolve data quality issues |
A |
R |
I |
C |
| Classify new data assets |
A |
R |
I |
C |
| Conduct data audits |
I |
R |
I |
A |
R = Responsible, A = Accountable, C = Consulted, I = Informed
Data Quality Dimensions
| Dimension |
Definition |
Measurement |
| Completeness |
No missing required values |
% non-null for required fields |
| Accuracy |
Values correct and reflect reality |
Spot check vs source system |
| Consistency |
Same data, same value across systems |
Cross-system reconciliation |
| Timeliness |
Data available when needed |
Lag from event to availability |
| Uniqueness |
No unintended duplicates |
Duplicate record count |
| Validity |
Values conform to defined formats/ranges |
% values passing validation rules |
Quality Threshold Example
| Dimension |
Target |
Alert threshold |
| Completeness |
≥ 99% |
< 95% |
| Accuracy |
≥ 99.5% |
< 98% |
| Timeliness |
≤ 1 hour lag |
> 4 hours |
| Uniqueness |
0 duplicates |
Any duplicates |
Retention Policy by Data Type
| Data type |
Retention period |
Basis |
Disposal method |
| Customer PII |
Duration of relationship + 7 years |
Legal/contractual |
Secure deletion |
| Financial records |
7 years |
Tax/audit |
Secure deletion |
| Employee records |
Employment + 7 years |
Employment law |
Secure deletion |
| System logs |
12 months (hot) + 24 months (cold) |
Security/PCI |
Secure deletion |
| Marketing data |
Until opt-out + 30 days |
CCPA/GDPR |
Secure deletion |
| Backup data |
90 days |
Operational |
Encrypted overwrite |
| Contracts |
Term + 10 years |
Legal |
Secure archival |
Governance Committee Charter
Purpose
Oversee enterprise data governance program; resolve cross-domain data disputes; approve policies.
Membership
- Chief Data Officer (chair)
- Representatives from: Legal, Compliance, IT Security, Finance, Product, Analytics
- Data stewards from key domains (rotating)
Meeting Cadence
- Monthly governance committee meeting
- Quarterly data quality review
- Annual policy review
Decision Authority
| Decision |
Authority level |
| New data classification policy |
Committee approval |
| Access to Restricted data |
Data owner + Compliance |
| New data retention schedule |
Committee approval |
| Data quality remediation plan |
Steward (inform committee) |
| Cross-border data transfer |
Legal + Compliance |
Data Incident Process
Severity Classification
| Severity |
Definition |
Response time |
| P1 |
Restricted data exposed externally |
Immediate (< 1 hour) |
| P2 |
Confidential data accessed without authorization |
< 4 hours |
| P3 |
Data quality issue affecting decisions |
< 24 hours |
| P4 |
Policy violation, no data exposure |
< 1 week |
Response Steps
- Detect and contain the incident
- Assess scope — what data, how many records, who affected
- Notify: Data Owner → Governance Team → Legal/Compliance (if P1/P2)
- Preserve evidence (logs, access records)
- Remediate root cause
- Document lessons learned
- Update controls to prevent recurrence
1---2name: data-governance3description: When to activate: data governance, data catalog, data dictionary, data lineage, data ownership, data quality, data classification, retention policy, data steward4---56# Data Governance78## Framework Components910A complete data governance program covers:11121. **Policies** — rules for how data is managed (classification, retention, access)132. **Standards** — technical specifications (naming conventions, formats, quality thresholds)143. **Processes** — how governance is executed (request handling, data quality remediation)154. **Roles** — who owns, stewards, and uses data165. **Technology** — catalog, lineage, quality tooling176. **Metrics** — how governance health is measured1819## Data Classification Tiers2021| Tier | Label | Examples | Default handling |22|------|-------|----------|-----------------|23| 1 | Public | Press releases, marketing content, public docs | No controls needed |24| 2 | Internal | Internal policies, employee directories, meeting notes | Internal use only, not for external sharing |25| 3 | Confidential | Customer PII, financials, contracts, source code | Encrypted at rest and in transit; access-controlled |26| 4 | Restricted | Health records, payment card data, credentials, trade secrets | Strict need-to-know; regulatory controls apply |2728### Classification Decision Tree29```30Does it contain PII, PHI, PCI, credentials, or trade secrets?31 Yes → Restricted32Does it contain customer data, financial data, or internal IP?33 Yes → Confidential34Is it intended for internal use only?35 Yes → Internal36Otherwise → Public37```3839## Data Catalog Structure4041A data catalog entry should include:4243| Field | Description |44|-------|-------------|45| Asset name | Technical name of the table/dataset/API |46| Display name | Human-readable name |47| Description | What the asset contains and its purpose |48| Owner | Business owner (accountable for the data) |49| Steward | Technical steward (maintains quality and definitions) |50| Classification | Public / Internal / Confidential / Restricted |51| System of record | Authoritative source for this data |52| Consuming systems | Downstream systems that use this data |53| Tags | Domain, subject area, regulation tags |54| Last updated | Date metadata was last verified |5556## Data Dictionary Template5758For each field in a dataset:5960| Field name | Data type | Description | Example | Owner | PII? | Nullable | Source |61|-----------|-----------|-------------|---------|-------|------|----------|--------|62| customer_id | UUID | Unique identifier for a customer | 550e8400-... | Data Platform | No | No | CRM |63| email_address | VARCHAR(255) | Customer email | user@example.com | Marketing | Yes | No | Sign-up form |64| date_of_birth | DATE | Customer date of birth (YYYY-MM-DD) | 1985-03-22 | Compliance | Yes (Sensitive) | Yes | KYC flow |6566## Data Lineage Documentation6768### Lineage Levels69- **Column-level** — tracks individual field transformations (gold standard)70- **Table-level** — tracks which tables feed which tables71- **System-level** — tracks data flows between systems7273### Lineage Capture Methods74| Method | Effort | Coverage |75|--------|--------|----------|76| Manual documentation | High effort, low accuracy | Legacy systems |77| ETL metadata parsing | Medium effort | ETL-based pipelines |78| Query log analysis | Medium effort | SQL-based transformations |79| Automated scanners (OpenMetadata, DataHub, Atlan) | Low ongoing effort | Modern stacks |8081### Minimum Lineage Record82```83Source: [System/Table/Column]84Transformation: [Description of logic applied]85Destination: [System/Table/Column]86Frequency: [Real-time / Hourly / Daily / Weekly]87Owner: [Team responsible for the pipeline]88Last validated: [Date]89```9091## Data Ownership RACI9293| Activity | Data Owner | Data Steward | Data Consumer | Governance Team |94|----------|-----------|-------------|---------------|----------------|95| Define data policies | A | C | I | R |96| Maintain data dictionary | I | A/R | I | C |97| Approve access requests | A | R | I | C |98| Resolve data quality issues | A | R | I | C |99| Classify new data assets | A | R | I | C |100| Conduct data audits | I | R | I | A |101102R = Responsible, A = Accountable, C = Consulted, I = Informed103104## Data Quality Dimensions105106| Dimension | Definition | Measurement |107|-----------|-----------|-------------|108| Completeness | No missing required values | % non-null for required fields |109| Accuracy | Values correct and reflect reality | Spot check vs source system |110| Consistency | Same data, same value across systems | Cross-system reconciliation |111| Timeliness | Data available when needed | Lag from event to availability |112| Uniqueness | No unintended duplicates | Duplicate record count |113| Validity | Values conform to defined formats/ranges | % values passing validation rules |114115### Quality Threshold Example116| Dimension | Target | Alert threshold |117|-----------|--------|----------------|118| Completeness | ≥ 99% | < 95% |119| Accuracy | ≥ 99.5% | < 98% |120| Timeliness | ≤ 1 hour lag | > 4 hours |121| Uniqueness | 0 duplicates | Any duplicates |122123## Retention Policy by Data Type124125| Data type | Retention period | Basis | Disposal method |126|-----------|-----------------|-------|----------------|127| Customer PII | Duration of relationship + 7 years | Legal/contractual | Secure deletion |128| Financial records | 7 years | Tax/audit | Secure deletion |129| Employee records | Employment + 7 years | Employment law | Secure deletion |130| System logs | 12 months (hot) + 24 months (cold) | Security/PCI | Secure deletion |131| Marketing data | Until opt-out + 30 days | CCPA/GDPR | Secure deletion |132| Backup data | 90 days | Operational | Encrypted overwrite |133| Contracts | Term + 10 years | Legal | Secure archival |134135## Governance Committee Charter136137### Purpose138Oversee enterprise data governance program; resolve cross-domain data disputes; approve policies.139140### Membership141- Chief Data Officer (chair)142- Representatives from: Legal, Compliance, IT Security, Finance, Product, Analytics143- Data stewards from key domains (rotating)144145### Meeting Cadence146- Monthly governance committee meeting147- Quarterly data quality review148- Annual policy review149150### Decision Authority151| Decision | Authority level |152|----------|----------------|153| New data classification policy | Committee approval |154| Access to Restricted data | Data owner + Compliance |155| New data retention schedule | Committee approval |156| Data quality remediation plan | Steward (inform committee) |157| Cross-border data transfer | Legal + Compliance |158159## Data Incident Process160161### Severity Classification162| Severity | Definition | Response time |163|----------|-----------|---------------|164| P1 | Restricted data exposed externally | Immediate (< 1 hour) |165| P2 | Confidential data accessed without authorization | < 4 hours |166| P3 | Data quality issue affecting decisions | < 24 hours |167| P4 | Policy violation, no data exposure | < 1 week |168169### Response Steps1701. Detect and contain the incident1712. Assess scope — what data, how many records, who affected1723. Notify: Data Owner → Governance Team → Legal/Compliance (if P1/P2)1734. Preserve evidence (logs, access records)1745. Remediate root cause1756. Document lessons learned1767. Update controls to prevent recurrence
Mattakushi432