dataset-quality-audit
A data quality auditing tool that runs 12-dimension quality checks on tabular data, producing per-dimension scores (0–100), an overall grade, and actionable fix suggestions.
Capabilities
| Dimension |
Description |
| Missing Values |
Count and percentage of null/NaN values per column |
| Duplicate Rows |
Number and percentage of fully duplicated rows |
| Type Consistency |
Mixed types within a single column (e.g., numbers mixed with text) |
| Value Range / Outliers |
Outlier detection using the IQR method |
| Format Compliance |
Consistency of date, email, phone number, and other formatted fields |
| Uniqueness Constraints |
Whether ID-type columns contain duplicates |
| Whitespace Issues |
Leading/trailing spaces, empty strings, whitespace-only values |
| Constant Columns |
Columns with only a single unique value (zero information) |
| Distribution Skewness |
Whether numeric columns have excessive skewness |
| Column Naming |
Spaces, special characters, or inconsistent casing in column names |
| Cardinality Anomalies |
Unusually high or low number of unique values |
| Cross-Column Consistency |
Logical checks across columns (e.g., start date before end date) |
Quick Start
# Basic quality check
python3 scripts/data_quality_checker.py data.csv
# Save report as JSON
python3 scripts/data_quality_checker.py data.csv --output report.json
# Specify ID columns (for uniqueness checks)
python3 scripts/data_quality_checker.py users.csv --id-columns "user_id,email"
# Specify date columns (for format checks)
python3 scripts/data_quality_checker.py orders.csv --date-columns "created_at,updated_at"
Detailed Usage
Basic Invocation
python3 scripts/data_quality_checker.py <data-file> [options]
Parameters
| Parameter |
Short |
Required |
Default |
Description |
input |
— |
Yes |
— |
Path to input file (CSV/TSV/Excel/JSON) |
--output |
-o |
No |
stdout |
Path for the JSON report output |
--id-columns |
-id |
No |
Auto-detect |
Comma-separated column names that should be unique |
--date-columns |
-dc |
No |
Auto-detect |
Comma-separated column names containing dates |
--sample |
-s |
No |
All rows |
Number of rows to sample (useful for large files) |
--encoding |
-e |
No |
utf-8 |
File encoding |
Output Format (JSON)
{
"file": "data.csv",
"rows": 10000,
"columns": 15,
"overall_score": 78.5,
"grade": "B",
"dimensions": {
"missing_values": {
"score": 85.0,
"issues": [
{"column": "age", "missing_count": 150, "missing_pct": 1.5, "suggestion": "Fill with median or mode"}
]
},
"duplicates": {
"score": 95.0,
"issues": [...]
}
},
"top_suggestions": [
"Column 'age' has 1.5% missing values — consider filling with the median",
"Found 200 fully duplicated rows — consider deduplication"
]
}
Grading Scale
| Grade |
Score Range |
Meaning |
| A+ |
95–100 |
Excellent quality — ready for use as-is |
| A |
90–95 |
Good quality — minor issues only |
| B |
80–90 |
Moderate quality — recommended to fix before use |
| C |
60–80 |
Poor quality — significant cleaning required |
| D |
40–60 |
Very poor quality — many issues need attention |
| F |
0–40 |
Essentially unusable — requires re-collection or major cleanup |
Dependencies
pip install pandas numpy
Activation Triggers
Triggered when users ask to check data quality, find missing or duplicate values, detect outliers, validate formats, profile data, or clean data.
1---2name: dataset-quality-audit3description: "Run comprehensive quality checks on tabular data (CSV/Excel/TSV/JSON), detecting missing values, duplicates, outliers, format issues, and type inconsistencies to produce an overall score, grade, and actionable suggestions."4license: MIT5metadata:6 source: https://skillmd.com/skills/jantoniofc/dataset-quality-audit7---89# dataset-quality-audit1011A data quality auditing tool that runs 12-dimension quality checks on tabular data, producing per-dimension scores (0–100), an overall grade, and actionable fix suggestions.1213## Capabilities1415| Dimension | Description |16|-----------|-------------|17| Missing Values | Count and percentage of null/NaN values per column |18| Duplicate Rows | Number and percentage of fully duplicated rows |19| Type Consistency | Mixed types within a single column (e.g., numbers mixed with text) |20| Value Range / Outliers | Outlier detection using the IQR method |21| Format Compliance | Consistency of date, email, phone number, and other formatted fields |22| Uniqueness Constraints | Whether ID-type columns contain duplicates |23| Whitespace Issues | Leading/trailing spaces, empty strings, whitespace-only values |24| Constant Columns | Columns with only a single unique value (zero information) |25| Distribution Skewness | Whether numeric columns have excessive skewness |26| Column Naming | Spaces, special characters, or inconsistent casing in column names |27| Cardinality Anomalies | Unusually high or low number of unique values |28| Cross-Column Consistency | Logical checks across columns (e.g., start date before end date) |2930## Quick Start3132```bash33# Basic quality check34python3 scripts/data_quality_checker.py data.csv3536# Save report as JSON37python3 scripts/data_quality_checker.py data.csv --output report.json3839# Specify ID columns (for uniqueness checks)40python3 scripts/data_quality_checker.py users.csv --id-columns "user_id,email"4142# Specify date columns (for format checks)43python3 scripts/data_quality_checker.py orders.csv --date-columns "created_at,updated_at"44```4546## Detailed Usage4748### Basic Invocation4950```bash51python3 scripts/data_quality_checker.py <data-file> [options]52```5354### Parameters5556| Parameter | Short | Required | Default | Description |57|-----------|-------|----------|---------|-------------|58| `input` | — | Yes | — | Path to input file (CSV/TSV/Excel/JSON) |59| `--output` | `-o` | No | stdout | Path for the JSON report output |60| `--id-columns` | `-id` | No | Auto-detect | Comma-separated column names that should be unique |61| `--date-columns` | `-dc` | No | Auto-detect | Comma-separated column names containing dates |62| `--sample` | `-s` | No | All rows | Number of rows to sample (useful for large files) |63| `--encoding` | `-e` | No | utf-8 | File encoding |6465## Output Format (JSON)6667```json68{69 "file": "data.csv",70 "rows": 10000,71 "columns": 15,72 "overall_score": 78.5,73 "grade": "B",74 "dimensions": {75 "missing_values": {76 "score": 85.0,77 "issues": [78 {"column": "age", "missing_count": 150, "missing_pct": 1.5, "suggestion": "Fill with median or mode"}79 ]80 },81 "duplicates": {82 "score": 95.0,83 "issues": [...]84 }85 },86 "top_suggestions": [87 "Column 'age' has 1.5% missing values — consider filling with the median",88 "Found 200 fully duplicated rows — consider deduplication"89 ]90}91```9293## Grading Scale9495| Grade | Score Range | Meaning |96|-------|------------|---------|97| A+ | 95–100 | Excellent quality — ready for use as-is |98| A | 90–95 | Good quality — minor issues only |99| B | 80–90 | Moderate quality — recommended to fix before use |100| C | 60–80 | Poor quality — significant cleaning required |101| D | 40–60 | Very poor quality — many issues need attention |102| F | 0–40 | Essentially unusable — requires re-collection or major cleanup |103104## Dependencies105106- Python 3.8+107- pandas108- numpy109110```bash111pip install pandas numpy112```113114## Activation Triggers115116Triggered when users ask to check data quality, find missing or duplicate values, detect outliers, validate formats, profile data, or clean data.