Data Analyse

Data Analyse

Take a dataset — a clean(ish) .xlsx/CSV, a pasted table, or the output of another toolkit skill — and deliver an insight brief: the headline findings, the key metrics for that type of data, and honest caveats. The design is intent-first (ask what question the data should answer before computing anything) and compute-then-interpret (every quoted number comes from the deterministic local engine; the narrative interprets, it never generates figures).

Self-sufficient & local engine. All computation runs on your machine via scripts/analyse.py (+ the shared toolkit engine) — no network calls. Note the AI agent driving the skill does send whatever it reads into its context to your AI provider; "never leaves" is not claimed. See ../../PRINCIPLES.md#data-handling--pii-policy.

Workflow

0 — Intent (ask up front)

Before computing anything, ask (a short AskUserQuestion):

1. What question should this data answer? ("are sales growing?", "where's the risk in receivables?", "which segment matters?") — or is it an open "tell me what you see"?
2. Who's it for and what decision does it feed? — a partner one-pager reads differently from a working analysis.
3. Anything known already? — targets, prior-period figures, or a hypothesis to test.

An open-ended "just analyse it" is fine — then the playbook (step 3) drives the metric selection, and the brief says so.

Done when: you have the question (or "open-ended"), the audience/decision, and any prior figures/hypotheses — or the user has explicitly said "just analyse it". A one-line restatement to the user confirms alignment.

1 — Ingest

Same shared engine as the other skills (../../scripts/):

import sys, pathlib
sys.path.insert(0, str(pathlib.Path("../../scripts").resolve()))
import ingest
header_rows, note = ingest.read_any(path)     # xlsx/csv/pdf/docx/msg; read_paste for pasted tables

Run from the skill directory (skills/data-analyse/). The ../../scripts path resolves to the toolkit-root scripts/ where ingest.py and dataclean.py live. The analysis engine (analyse.py) is in this skill's own scripts/ subdirectory. Multi-tab workbooks: read_any raises SheetSelectionRequired if several tabs hold data — list the sheets and ask, don't guess.

For files with 10k+ rows, use ingest.read_large instead of read_any to avoid OOM. See references/large-file-patterns.md for vectorised operation guidance.

Done when: you have (header_rows, note) from read_any (or read_large for 10k+ rows). If SheetSelectionRequired was raised, the user has chosen a sheet. note has been read for source-quality flags.

2 — Profile & quality gate

dataclean.profile_table(header, rows) / score_quality(...) — column types, missing %, duplicates. If the data is too messy to analyse honestly (columns that won't parse, heavy duplication, mixed junk), stop and route to data-tidy first — analysing dirty data produces confident-looking nonsense. A little noise is fine: the engine counts every skipped/unparseable cell and the brief must disclose those counts.

Currency gate: analyse.currency_mix(col) — pass the currency-bearing column: the amount column when codes ride inside the amounts (S$100, £1,000), or the separate ISO-code column when the layout is two columns (Amount + Currency of GBP/USD) — the common export shape. If it reports more than one code, never sum the amounts as-is (100 USD ≠ 100 SGD). Split by currency (breakdown(by=<currency col>)) or ask.

3 — Propose the analysis plan (confirm before computing)

analyse.suggest_playbook(header, rows) maps columns to roles (dates, amounts, categories, ids) and suggests analyses. Combine that with the intent and the data-type playbook (references/playbooks.md — transactions, receivables/ageing, pipeline, survey/categorical, task/ops list, general ledger, inventory, spend/AP, time series, cross-domain, generic) into a short plan: "I'll look at: monthly trend of Amount, breakdown by Customer with concentration, outliers, and ageing from Due date as of 12 Jul 2026 — sound right?" Show it; confirm.

Two datasets provided? If the user hands over two files that share a dimension (sales vs competitor prices, actual vs budget), that's the cross-domain playbook — join_on to relate them on the shared key (report coverage), then compare_series to read position and co-movement. It relates, it does not match line-by-line — reconciling two sets to find breaks is data-reconcile. Nail the join key and grain before computing.

4 — Compute (deterministic, engine only)

All metrics come from scripts/analyse.py — exact Decimal, dayfirst dates, reusing the shared parsers:

Run computations in a script (not mental arithmetic) so the analysis is reproducible; keep the script in the working folder as the audit trail. The engine ships tested — don't re-run its --self-test or envcheck.py as part of a normal analysis (that's setup overhead the user pays every run); reach for them only when an import fails or the environment is genuinely uncertain.

5 — Write the insight brief

ALWAYS this structure (markdown; British English, dates DD MMM YYYY, currency with code):

# [Dataset] — insight brief                      (as of [date], n = [rows])
## Headline            3–5 findings, most consequential first, each with its number
## Key metrics         the tables from the engine (render_md)
## Notable             outliers, concentration, anomalies — with the specific rows/values
## Caveats & quality   skipped cells, missing %, gap periods, currency notes, what this
                       data CANNOT answer
[footer: draft for review — descriptive analysis, not advice]

Deliver the brief as a .md (plus the optional metrics .xlsx). Keep it to one page of reading unless asked for depth.

Done when: the brief follows the exact structure (Headline / Key metrics / Notable / Caveats & quality / footer), every number in it came from the engine, and the .md file is written. If a metrics .xlsx was requested, that too is written.

6 — Offer the next step

• Visual one-pager wanted? → hand the computed series/breakdowns to data-visualise (its rows_from_xlsx reads the metrics workbook directly).
• Numbers look wrong vs another source? → that's data-reconcile, not this skill.

Done when: the next step is offered to the user (visualise, reconcile, or "done") and the brief has been delivered. The skill does not start the next step without the user's go-ahead.

Insight discipline (what makes the brief trustworthy)

• Every number is computed. If a figure appears in the brief, it came out of the engine or is an arithmetic step shown in the brief. No free-form estimates.
• Observation ≠ interpretation. "March revenue fell 40% MoM" is an observation; "likely the contract gap" is an interpretation — label it as a possible explanation and say what would confirm it. Never present correlation as cause.
• Calibrated language. "Verified" vs "likely" vs "couldn't confirm" (PRINCIPLES.md #4). A trend over 3 periods is "early"; over 2 it is not a trend.
• Disclose the denominators. Shares, averages and trends state n and what was excluded (skipped cells, unparsed dates, filled gap periods).
• Ratios and multiples are computed too. "20× the typical ticket" is only meaningful with the comparator named — compute it from engine figures and say what the base is (median, mean, next-largest). An eyeballed multiple is a free-form estimate in disguise.
• Never invent a conversion. No exchange rates, deflators or per-unit factors the user didn't supply — a blended figure built on an assumed rate is an invented number, however clearly footnoted. Report per-currency (per-unit) and, if a combined view is wanted, ask for the rate to use.
• Descriptive, not advisory. The brief describes what the data shows. It never recommends buying/selling/pricing/provisioning — flag the decision to the qualified owner (drafts, not advice).

Files

• scripts/analyse.py — the metric engine (see table above); reuses the shared parsers in ../../scripts/dataclean.py; python analyse.py --self-test.
• references/playbooks.md — per-data-type metric menus: which analyses matter for transactions, receivables, pipeline, survey, ops-list, time-series and generic tables.
• examples/sample_sales.csv — a synthetic sales export (no real data) to demo against.

Principles

Behavioural charter: ../../PRINCIPLES.md — drafts not advice, never invent, honesty and calibration, plain speech, action boundary.

Data handling

Metrics are computed on your machine — the data, the metrics and the brief stay on your synced or shared file store, and the engine uploads nothing. (The AI agent driving the skill does send whatever it reads into its context to your AI provider.) A brief that names individuals or quotes confidential figures is gated on any egress. Full rule: ../../PRINCIPLES.md#data-handling--pii-policy.

Feedback

Improvement or bug? Use the toolkit's shared format — ../../CONTRIBUTING.md#skill-feedback-format — save as feedback_data-analyse_[date].txt and hand it to the user to file; fix in scope if asked.

Requirements & mode

Portable: Python + openpyxl for .xlsx in/out; PDF/.docx/.msg inputs use the same optional libraries as data-tidy (degrade per-source). No network, no Office, no credentials. If an import fails or the environment is uncertain, pre-screen with python ../../scripts/envcheck.py and see ../../README.md#mode--environment-compatibility — otherwise just start analysing.