Data Analysis

Data Analysis — Full Data Analysis Pipeline

An agent team collaborates to perform the full data lifecycle: exploration → cleaning → analysis → visualization → reporting.

Execution Mode

Agent Team — 5 members communicate directly via SendMessage and cross-validate.

Agent Composition

Workflow

Phase 1: Preparation (Orchestrator performs directly)

1. Extract from user input:
   • Data Source: File path, format (CSV/Excel/JSON/DB), size
   • Analysis Purpose: Business questions, hypotheses, expected results
   • Constraints (optional): Time, specific analysis techniques, reporting audience
   • Domain Information (optional): Industry, variable meanings, business context
2. Create _workspace/ directory and _workspace/scripts/ subdirectory
3. Organize input and save to _workspace/00_input.md
4. Copy data files to _workspace/data/
5. If existing files are present, copy to _workspace/ and skip the corresponding Phase
6. Determine execution mode based on request scope

Phase 2: Team Assembly and Execution

Tasks 3a (analysis) and 3b (EDA visualization) run in parallel.

Inter-team communication flow:

• explorer completes → Sends cleaning recommendations to cleaner, analysis suggestions to analyst, distribution visualization targets to visualizer
• cleaner completes → Sends cleaned data location and transformation history to analyst
• analyst completes → Sends visualization requests to visualizer, insights to reporter
• visualizer completes → Sends visualization list to reporter
• reporter cross-validates all outputs. Sends correction requests when inconsistencies are found (up to 2 times)

Phase 3: Integration and Final Outputs

1. Check all files in _workspace/
2. Verify that all required corrections from reporter have been addressed
3. Report final summary to the user:
   • Exploration Report — 01_exploration_report.md
   • Cleaning Log — 02_cleaning_log.md
   • Analysis Results — 03_analysis_results.md
   • Visualizations — 04_visualizations.md
   • Final Report — 05_final_report.md
   • Reproducible Scripts — scripts/ directory

Scale-Based Modes

Leveraging existing files: If already cleaned data exists, skip explorer and cleaner. If analysis results exist, skip analyst and proceed with visualization and reporting only.

Data Transfer Protocol

File naming convention: {order}_{output}.{extension}

Error Handling

Test Scenarios

Normal Flow

Prompt: "Analyze this sales CSV file and find the cause of the sales decline" Expected Results:

• EDA: Variable profiling, missing/outlier analysis, sales-related variable identification
• Cleaning: Missing value treatment, outlier capping, type conversion log
• Analysis: Time-series decomposition, segment comparison (t-test/ANOVA), correlation analysis
• Visualization: Sales trend line chart, factor comparison bar chart, correlation heatmap
• Report: Top 3 decline causes + recommended actions + executive summary

Existing File Flow

Prompt: "I already have cleaned data. Just do statistical analysis and visualization" + cleaned data file attached Expected Results:

• Copy existing data to _workspace/data/
• Analysis mode: Skip explorer and cleaner, deploy analyst + visualizer + reporter
• Cleaning history recorded based on user-provided information

Error Flow

Prompt: "Analyze this Excel file" (many variables with >50% missing, numerous outliers) Expected Results:

• explorer reports data quality issues in detail
• cleaner presents per-variable treatment strategies with rationale, warning if rows decrease >30%
• analyst states data sufficiency limitations and performs power analysis
• reporter honestly records data quality issues in limitations section

Agent Extension Skills