Data Analysis Workflow
Use this skill for exploratory analysis, document review, dataset triage, metric investigation, or mixed-source fact finding inside this repo.
Default source rule: if the user does not name a path, start from workspace/input/.
Default Workflow
- Inventory the input files.
- Run
scripts/file_profile.py on the likely inputs.
- Split the work by source type:
- Tabular: CSV, TSV, XLSX, JSON, parquet-like exports
- Document: PDF, DOCX, PPTX, TXT, MD, HTML, XML
- Mixed folders: create a short source map first
- Create reproducible analysis:
- SQL in
workspace/sql/
- Notes in
workspace/notes/
- Charts and exports in
workspace/artifacts/
- End with findings, caveats, and suggested next drills.
Scripts
scripts/file_profile.py
Use first for profiling a file or folder. It prints a compact summary of structure, schema, sample content, and likely analysis directions.scripts/query_tabular.py
Use for SQL over one or more tabular files. It registers files as DuckDB tables and returns markdown, CSV, or JSON output.scripts/chart_from_query.py
Use when a chart should be produced from a repeatable SQL query rather than one-off notebook code.
Source-Type Guidance
- For CSV/XLSX/JSON tabular data, prefer SQL or pandas over manual reading.
- For PDFs and DOCX files, extract text first, then summarize sections and search for anchors before making claims.
- For PPTX files, treat slide titles and bullets as the first-pass structure.
- For unknown formats, inspect metadata and fall back to raw text or binary summary rather than guessing.
Reporting Standard
Every substantial analysis should leave behind at least one durable artifact:
- a note with the question, method, caveats, and findings
- a SQL file or Python snippet if the analysis is likely to be reused
- a chart or export when it materially helps interpretation
References
Read these only when needed:
references/workflow-recipes.md for concrete investigation patternsreferences/output-conventions.md for where to store results and how to name them
1---2name: data-analysis-workflow3description: Use this skill when the task is open-ended analysis of local files or mixed data sources in this repository, especially CSV, XLSX, PDF, DOCX, PPTX, JSON, text, HTML, XML, or folders of related evidence. It provides the default workflow, helper scripts, and output conventions for reproducible data exploration in Codex.4---56# Data Analysis Workflow78Use this skill for exploratory analysis, document review, dataset triage, metric investigation, or mixed-source fact finding inside this repo.910Default source rule: if the user does not name a path, start from `workspace/input/`.1112## Default Workflow13141. Inventory the input files.152. Run `scripts/file_profile.py` on the likely inputs.163. Split the work by source type:17 - Tabular: CSV, TSV, XLSX, JSON, parquet-like exports18 - Document: PDF, DOCX, PPTX, TXT, MD, HTML, XML19 - Mixed folders: create a short source map first204. Create reproducible analysis:21 - SQL in `workspace/sql/`22 - Notes in `workspace/notes/`23 - Charts and exports in `workspace/artifacts/`245. End with findings, caveats, and suggested next drills.2526## Scripts2728- `scripts/file_profile.py`29 Use first for profiling a file or folder. It prints a compact summary of structure, schema, sample content, and likely analysis directions.30- `scripts/query_tabular.py`31 Use for SQL over one or more tabular files. It registers files as DuckDB tables and returns markdown, CSV, or JSON output.32- `scripts/chart_from_query.py`33 Use when a chart should be produced from a repeatable SQL query rather than one-off notebook code.3435## Source-Type Guidance3637- For CSV/XLSX/JSON tabular data, prefer SQL or pandas over manual reading.38- For PDFs and DOCX files, extract text first, then summarize sections and search for anchors before making claims.39- For PPTX files, treat slide titles and bullets as the first-pass structure.40- For unknown formats, inspect metadata and fall back to raw text or binary summary rather than guessing.4142## Reporting Standard4344Every substantial analysis should leave behind at least one durable artifact:4546- a note with the question, method, caveats, and findings47- a SQL file or Python snippet if the analysis is likely to be reused48- a chart or export when it materially helps interpretation4950## References5152Read these only when needed:5354- `references/workflow-recipes.md` for concrete investigation patterns55- `references/output-conventions.md` for where to store results and how to name them
kernelshreyak