majiayu000 CSV Analyzer
Overview
Comprehensive CSV data analysis and visualization engine. Run the script, then use this guide to interpret results and provide insights to users.
Quick Start
cd ~/.claude/skills/csv-analyzer/scripts
export $(grep -v '^#' /path/to/project/.env | xargs 2>/dev/null)
python3 analyze_csv.py /path/to/data.csv
Chart Selection Decision Tree
IMPORTANT: Choose charts based on what the user needs to understand:
What is the user trying to understand?
│
├── "What does my data look like?" (Overview)
│ └── Run with defaults → overview_dashboard.png
│
├── "Is my data clean?" (Quality)
│ └── Check: quality_score, missing_values, duplicates
│ └── Show: missing_values.png if problems exist
│
├── "What's the distribution?" (Single Variable)
│ ├── Numeric → numeric_distributions.png (histogram + KDE)
│ ├── Categorical → categorical_distributions.png (bar chart)
│ └── Time-based → time_series.png
│
├── "Are there outliers?" (Anomalies)
│ └── box_plots.png → points beyond whiskers are outliers
│
├── "How are variables related?" (Relationships)
│ ├── 2 numeric vars → correlation_heatmap.png
│ ├── 2-6 numeric vars → pairplot.png (scatter matrix)
│ ├── Numeric vs Categorical → violin_plot.png
│ └── All numeric → correlation_heatmap.png
│
└── "Can I predict X from Y?" (Predictive)
└── correlation_heatmap.png → |r| > 0.5 suggests predictive power
How to Interpret Results (For Claude)
Quality Score Interpretation
| Score | Grade | What to Tell User |
|---|---|---|
| 90-100 | A | "Your data is excellent quality - ready for analysis" |
| 80-89 | B | "Good quality data with minor issues worth noting" |
| 70-79 | C | "Moderate quality - address missing values before critical analysis" |
| 60-69 | D | "Significant quality issues - recommend data cleaning first" |
| <60 | F | "Critical issues - data needs substantial cleaning" |
Correlation Interpretation
| |r| Value | Strength | What to Say |
|---|---|---|
| 0.9 - 1.0 | Very Strong | "X and Y are very strongly related - almost deterministic" |
| 0.7 - 0.9 | Strong | "X and Y have a strong relationship - X could help predict Y" |
| 0.5 - 0.7 | Moderate | "X and Y are moderately correlated - some predictive value" |
| 0.3 - 0.5 | Weak | "X and Y have a weak relationship - limited predictive power" |
| 0.0 - 0.3 | Negligible | "X and Y appear unrelated" |
Sign matters:
- Positive: "As X increases, Y tends to increase"
- Negative: "As X increases, Y tends to decrease"
Skewness Interpretation
| Skewness | Distribution Shape | Recommendation |
|---|---|---|
| < -1 | Heavy left tail | "Most values are high, with some very low outliers" |
| -1 to -0.5 | Mild left skew | "Slightly more low outliers than high" |
| -0.5 to 0.5 | Symmetric | "Nicely balanced distribution - good for most analyses" |
| 0.5 to 1 | Mild right skew | "Slightly more high outliers than low" |
| > 1 | Heavy right tail | "Most values are low, with some very high outliers. Consider log transform for modeling." |
Outlier Assessment
When reporting outliers:
- Few outliers (<1%): "A few extreme values that may warrant investigation"
- Moderate outliers (1-5%): "Notable outliers - check if they're errors or genuine extremes"
- Many outliers (>5%): "High outlier rate suggests either data issues or a non-normal distribution"
Insight Generation Framework
After running analysis, provide insights in this order:
1. Data Overview (Always)
"Your dataset has [rows] records and [cols] columns:
- [n] numeric columns: [list top 3]
- [n] categorical columns: [list top 3]
- Data quality score: [score]/100 ([grade])"
2. Key Findings (Pick most relevant)
If quality issues exist:
"I noticed some data quality concerns:
- [X]% missing values in [column] - [recommend: drop/impute/investigate]
- [N] duplicate rows detected - [recommend: keep first/remove all/investigate]"
If strong correlations found:
"Interesting relationships I found:
- [col1] and [col2] are strongly correlated (r=[value]) - [interpretation]
- This suggests [actionable insight]"
If outliers detected:
"I detected outliers in [columns]:
- [column]: [n] values beyond normal range ([min outlier] to [max outlier])
- These could be [data errors / genuine extremes / worth investigating]"
If skewed distributions:
"[Column] has a [right/left]-skewed distribution:
- Most values cluster around [median]
- But there are extreme values up to [max]
- For modeling, consider [log transform / robust methods]"
3. Recommendations (Based on findings)
| Finding | Recommendation |
|---|---|
| Missing >20% in column | "Consider dropping this column or investigating why it's missing" |
| Missing <5% scattered | "Safe to impute with median (numeric) or mode (categorical)" |
| High correlation (>0.9) | "These columns may be redundant - consider keeping only one" |
| Many outliers | "Use robust statistics (median instead of mean) or investigate data collection" |
| Highly skewed | "Apply log transform before linear modeling" |
| Low quality score | "Prioritize data cleaning before analysis" |
Multi-Chart Dashboard Requests
When user asks for a "dashboard" or "comprehensive view":
# Generate all visualizations
python3 analyze_csv.py data.csv --format html --max-charts 10
Then present charts in this order:
- overview_dashboard.png - "Here's your data at a glance"
- correlation_heatmap.png - "Key relationships between variables"
- numeric_distributions.png - "How your numeric data is distributed"
- box_plots.png - "Outlier analysis"
- categorical_distributions.png - "Category breakdowns" (if applicable)
Command Reference
Basic Analysis
python3 analyze_csv.py data.csv
Full Report with All Charts
python3 analyze_csv.py data.csv --format markdown --max-charts 10
Quick Analysis (No Charts)
python3 analyze_csv.py data.csv --no-charts
Large Files (>100MB)
python3 analyze_csv.py huge.csv --sample 50000
Specific Date Columns
python3 analyze_csv.py data.csv --date-columns created_at updated_at
JSON for Programmatic Use
python3 analyze_csv.py data.csv --format json --no-charts
Custom Output Location
python3 analyze_csv.py data.csv --output-dir /path/to/project/.tmp/analysis
Chart Descriptions (For Explaining to Users)
| Chart | When to Show | How to Describe |
|---|---|---|
| overview_dashboard.png | Always for first look | "Here's a bird's eye view of your data" |
| missing_values.png | If missing data exists | "This shows where your data has gaps" |
| numeric_distributions.png | When exploring distributions | "This shows how your numeric values are spread out" |
| box_plots.png | When checking for outliers | "The dots outside the boxes are potential outliers" |
| correlation_heatmap.png | When exploring relationships | "Darker colors = stronger relationships" |
| categorical_distributions.png | For category analysis | "This shows the breakdown of your categories" |
| time_series.png | For temporal data | "Here's how your data changes over time" |
| pairplot.png | For multivariate exploration | "Each cell shows how two variables relate" |
| violin_plot.png | Comparing groups | "This shows how distributions differ across groups" |
Common User Questions → Actions
| User Says | Action |
|---|---|
| "Analyze this CSV" | Run full analysis, show overview + key insights |
| "Is my data clean?" | Focus on quality_score, missing values, duplicates |
| "Find patterns" | Show correlation_heatmap, highlight strong correlations |
| "Are there outliers?" | Show box_plots, list outlier counts per column |
| "Compare X across Y" | Generate violin_plot for numeric X vs categorical Y |
| "Show me trends" | Generate time_series if datetime column exists |
| "Create a dashboard" | Generate all charts, present organized summary |
| "What should I clean?" | List columns with missing >5%, duplicates, outliers |
Output Locations
Charts are saved to:
- Default:
~/.claude/skills/csv-analyzer/scripts/.tmp/csv_analysis/ - Custom: Use
--output-dir /path/to/project/.tmp/analysis
Always copy charts to user's project .tmp for visibility:
cp ~/.claude/skills/csv-analyzer/scripts/.tmp/csv_analysis/*.png /path/to/project/.tmp/csv_analysis/
Cost
Free - runs entirely locally using pandas, matplotlib, seaborn, scipy.
Dependencies
pip install pandas matplotlib seaborn scipy numpy