Get Stats
Use this skill whenever a conclusion depends on measured numeric data. Do not rely on selected examples, eyeballing, or averages alone.
Required Output
For every measured variable, report:
total: full population count before usability/filtering, stated separately from nn: number of usable samples- dropped/invalid sample count and filtering rule
- min, max, mean, median
- sample standard deviation when
n >= 2
- standard error of the mean when comparing means or estimating mean uncertainty
- p50/p75/p95/p99 for latency, queueing, size, or heavy-tailed distributions
- units and sign convention, especially for deltas
When answering the user, include the actual stats table in chat for the
measured quantities that drive the conclusion. Do not only say that an artifact
or JSON file was produced. A file can contain the full raw/reproducible output,
but the response itself must show the key rows with at least n, dropped/invalid
count, min, max, mean, median, stdev, stderr, p50, p75, p95, p99, units, and sign
convention. If there are many variables, show the decision-critical table in
chat and point to the artifact for the rest.
Always label total and n explicitly so a sample count cannot be mistaken for
the full number of records, cards, events, or groups. For grouped distributions,
show the overall population total and each group's total before showing usable
n; reconcile group totals to the overall total or explain any unclassified or
overlapping records.
When the raw observations are grouped into per-sample summaries, such as
per-route latency summaries, per-host windows, or per-experiment batches,
report both layers:
- within each group: the required stats for the raw observations in that group
- across groups: stats of each per-group statistic, e.g. min of per-group mins,
max of per-group maxes, mean and median of per-group means, stdev/stderr of
per-group medians, and p95/p99 of per-group maxima when tails matter
- group count, usable raw observation count, dropped group count, dropped raw
observation count, and the exact grouping key
- avoid averaging already-averaged groups without stating whether groups are
weighted by raw observation count or treated equally
When measuring an effect between two samples, also report:
- baseline and treatment
n
- baseline and treatment means and medians
- effect size as absolute delta and percent delta when meaningful
- Welch's t-test statistic, degrees of freedom, and p-value
- whether the effect is statistically meaningful and practically meaningful
- caveats for skew, outliers, censoring, non-independence, or confounders
Workflow
- Define the metric and unit before computing stats.
- State sample selection: source table/log/API, time range, filters, joins, and exclusions.
- Preserve raw observations or enough query/code to reproduce them.
- Compute descriptive stats for each sample before interpreting effects.
- For two-sample effects, prefer Welch's t-test unless the data shape makes it inappropriate.
- If the distribution is heavy-tailed, report quantiles and consider a non-parametric check; do not hide behind the mean.
- Separate measured facts from hypotheses.
Helper Script
Use scripts/compute_stats.py for quick deterministic stats from files or stdin.
Examples:
# One numeric value per line.
python3 scripts/compute_stats.py values.txt
# Compare two samples.
python3 scripts/compute_stats.py --a before.txt --b after.txt
# CSV grouped comparison, exactly two groups.
python3 scripts/compute_stats.py data.csv --csv --value-col latency_ms --group-col mode
The helper emits JSON so another script or notebook can consume it. If data is already in SQL, prefer computing raw rows in SQL and feeding the numeric columns to the helper, or compute equivalent aggregates directly in SQL and explicitly list the formulas.
Interpretation Rules
- Use
stdev for spread of observations; use stderr for uncertainty in the sample mean.
- Do not call a p-value meaningful without checking effect size and data quality.
- Do not use a t-test on repeated measurements that are obviously dependent without calling out the dependence.
- For latency, median and p95/p99 can matter more than mean; include both.
- For timing models, report model error in real units and confidence intervals, not only index positions.
1---2name: get-stats3description: Compute required descriptive and inferential statistics for measured samples. Use when the agent is analyzing latency, throughput, event timing, experiment baselines, A/B effects, benchmark results, production-vs-baseline comparisons, or any numeric measurement where outputs must include sample count, min, max, mean, median, standard deviation, standard error when relevant, confidence intervals when useful, and t-tests for measured effects.4---56# Get Stats78Use this skill whenever a conclusion depends on measured numeric data. Do not rely on selected examples, eyeballing, or averages alone.910## Required Output1112For every measured variable, report:1314- `total`: full population count before usability/filtering, stated separately from `n`15- `n`: number of usable samples16- dropped/invalid sample count and filtering rule17- min, max, mean, median18- sample standard deviation when `n >= 2`19- standard error of the mean when comparing means or estimating mean uncertainty20- p50/p75/p95/p99 for latency, queueing, size, or heavy-tailed distributions21- units and sign convention, especially for deltas2223When answering the user, include the actual stats table in chat for the24measured quantities that drive the conclusion. Do not only say that an artifact25or JSON file was produced. A file can contain the full raw/reproducible output,26but the response itself must show the key rows with at least `n`, dropped/invalid27count, min, max, mean, median, stdev, stderr, p50, p75, p95, p99, units, and sign28convention. If there are many variables, show the decision-critical table in29chat and point to the artifact for the rest.3031Always label `total` and `n` explicitly so a sample count cannot be mistaken for32the full number of records, cards, events, or groups. For grouped distributions,33show the overall population total and each group's total before showing usable34`n`; reconcile group totals to the overall total or explain any unclassified or35overlapping records.3637When the raw observations are grouped into per-sample summaries, such as38per-route latency summaries, per-host windows, or per-experiment batches,39report both layers:4041- within each group: the required stats for the raw observations in that group42- across groups: stats of each per-group statistic, e.g. min of per-group mins,43 max of per-group maxes, mean and median of per-group means, stdev/stderr of44 per-group medians, and p95/p99 of per-group maxima when tails matter45- group count, usable raw observation count, dropped group count, dropped raw46 observation count, and the exact grouping key47- avoid averaging already-averaged groups without stating whether groups are48 weighted by raw observation count or treated equally4950When measuring an effect between two samples, also report:5152- baseline and treatment `n`53- baseline and treatment means and medians54- effect size as absolute delta and percent delta when meaningful55- Welch's t-test statistic, degrees of freedom, and p-value56- whether the effect is statistically meaningful and practically meaningful57- caveats for skew, outliers, censoring, non-independence, or confounders5859## Workflow60611. Define the metric and unit before computing stats.622. State sample selection: source table/log/API, time range, filters, joins, and exclusions.633. Preserve raw observations or enough query/code to reproduce them.644. Compute descriptive stats for each sample before interpreting effects.655. For two-sample effects, prefer Welch's t-test unless the data shape makes it inappropriate.666. If the distribution is heavy-tailed, report quantiles and consider a non-parametric check; do not hide behind the mean.677. Separate measured facts from hypotheses.6869## Helper Script7071Use `scripts/compute_stats.py` for quick deterministic stats from files or stdin.7273Examples:7475```bash76# One numeric value per line.77python3 scripts/compute_stats.py values.txt7879# Compare two samples.80python3 scripts/compute_stats.py --a before.txt --b after.txt8182# CSV grouped comparison, exactly two groups.83python3 scripts/compute_stats.py data.csv --csv --value-col latency_ms --group-col mode84```8586The helper emits JSON so another script or notebook can consume it. If data is already in SQL, prefer computing raw rows in SQL and feeding the numeric columns to the helper, or compute equivalent aggregates directly in SQL and explicitly list the formulas.8788## Interpretation Rules8990- Use `stdev` for spread of observations; use `stderr` for uncertainty in the sample mean.91- Do not call a p-value meaningful without checking effect size and data quality.92- Do not use a t-test on repeated measurements that are obviously dependent without calling out the dependence.93- For latency, median and p95/p99 can matter more than mean; include both.94- For timing models, report model error in real units and confidence intervals, not only index positions.
dreamers-laboratory