# Artifact Data Loading

> Fit data into a self-contained page - pre-aggregating at the source, choosing a grain, embedding formats and their real byte costs, precision trimming, pagination versus virtualization, and offering a download for the full detail. Use when the data will not fit, the page is slow to open, or you are about to embed raw rows. Trigger on "too much data", "page is huge", "embed the data", "how do I get the rows in", "pagination", "virtualize", "download CSV", "aggregate".

- Skill: `lukehle/artifact-data-loading` (Agent Skill)
- Install (CLI): `npx skillmds@latest add lukehle/artifact-data-loading`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lukehle/artifact-data-loading/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- Author: Lukehle (https://skillmd.com/u/lukehle)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lukehle/artifact-data-loading

---


# Artifact data loading

A page is one file with a 16MB ceiling and no network. So the question is never "how do I ship all
the rows" — it is **"what is the smallest dataset that answers the question."**

Almost always, the answer is far smaller than the raw data, and the reduction happens *before*
embedding, not after.

---

## Aggregate at the source first

The highest-leverage step, and it happens in SQL, not in the page.

```sql
-- WRONG: 180,000 rows embedded so the page can group them
SELECT * FROM gl WHERE period = '2026-07'

-- RIGHT: 40 rows; the page renders, it does not compute
SELECT account, account_name,
       SUM(amount)                                    AS amount,
       COUNT(*)                                       AS txn_count,
       SUM(amount) - SUM(prior_amount)                AS variance
FROM   gl_with_prior
WHERE  period = '2026-07'
GROUP  BY account, account_name
HAVING ABS(SUM(amount) - SUM(prior_amount)) > 25000
```

Ship the grain the visual needs. A chart with 40 marks needs 40 rows — embedding 180,000 so the
browser can produce those 40 is pure cost.

**Ship a second, coarser aggregate for drill-down** rather than the detail: top 10 transactions per
account, not every transaction. Full detail belongs behind a download.

---

## Choosing the grain

Work backwards from the marks:

| Visual | Rows needed |
|---|---|
| KPI tile | 1 |
| Bridge / waterfall | one per bar (5-12) |
| Monthly time series, 3 years | 36 per series |
| Cohort heatmap, 24 × 24 | ≤ 576, and suppress small cells anyway |
| Detail table | the page size, not the table size |
| Scatter | consider binning above ~2,000 points |

If a grain exceeds a few thousand rows, ask what question needs that resolution. Usually the answer
is "none" — it was shipped because it was available.

---

## Format and real byte cost

Same 5,000 rows, six columns:

| Format | Approx size | Notes |
|---|---|---|
| Array of objects (keys repeated per row) | ~600KB | The default, and the most wasteful |
| Array of arrays + a header array | ~180KB | Usually a **3-4× win for one line of code** |
| Columnar (object of arrays) | ~170KB | Best when charts read one column at a time |
| CSV in a template literal, parsed on load | ~150KB | Smallest; costs a tiny parser |

```js
// Repeats every key 5,000 times
const rows = [{account:'6200', name:'Professional fees', amount:178000}, …];

// Same data, keys once
const cols = ['account','name','amount'];
const data = [['6200','Professional fees',178000], …];
const rows = data.map(r => Object.fromEntries(r.map((v,i) => [cols[i], v])));
```

Then trim what is inside:

- **Round before embedding.** `178000` not `178000.0000001`. Precision beyond what you display is
  pure weight — and per `ui-antipatterns`, displaying it would be a false-precision claim anyway.
- **Factor out repeated strings** into a lookup: `[0, 'Professional fees']` → `names[0]`.
- **Dates as short strings or offsets**, not ISO timestamps with milliseconds.
- **Drop columns the page never reads.** This is the one people skip; check honestly.

---

## Rendering large tables

Even when the data fits, the DOM may not. Above roughly **500 rows**, rendering everything makes the
page slow to open and janky to scroll.

Three options, in order of preference:

1. **Aggregate more.** Ask whether 5,000 rows is the deliverable or a failure to summarize. Usually
   the latter.
2. **Paginate.** Simple, predictable, printable, keyboard-friendly. Fine for a detail table.
3. **Virtualize.** Render only the visible window plus a buffer. Best for genuine exploration, but
   it breaks Ctrl+F, breaks print, and complicates accessibility — so use it deliberately.

If you virtualize, still provide a download for the full set, because search and print now only see
the window.

---

## Offer the detail as a download

When someone needs every row, give them a file rather than putting it in the DOM.

```js
const downloads = await claude.use('downloads');
if (!downloads) { hideExportButton(); return; }   // null means unavailable
await downloads.save({ filename: 'gl-detail-2026-07.csv', data: csv });
```

Two things to hold:

- **`null` means the capability is unavailable** — hide the affordance rather than showing a button
  that fails.
- **The viewer sees a confirmation and may decline.** A save is never silent or guaranteed, so offer
  it on explicit intent and handle rejection gracefully.

When the host provides `artifact-capabilities`, read it before wiring this up. If it is
unavailable, do not guess a downloads contract: hide export and keep the artifact usable
with its embedded data.

---

## Live data changes the calculus, not the discipline

With connector-backed data (`live-data-artifacts`) you are not embedding rows — but you are still
paying for them in transfer and render time, and the viewer waits.

Aggregate in the query, not in the handler. The same rule, one layer out.

Always ship a **baked snapshot alongside** the live path, sized by the rules above. It is what
renders when the capability is absent, and building it first guarantees the page works for everyone.

---

## A worked reduction

A GL detail file, 180,000 rows, 14 columns, 42MB raw. Over the ceiling and useless in a page.

1. Aggregate to account × period → **480 rows**
2. Filter to accounts over the materiality gate → **38 rows**
3. Add top-5 transactions per surviving account → **190 rows**
4. Array-of-arrays + rounded numbers → **~14KB embedded**
5. Full detail behind a download button

42MB → 14KB, and the page answers the question *better*, because the materiality gate is applied
mechanically instead of by eye.

---

## Related skills

- `artifact-architecture` — the budget and the tier decision
- `artifact-performance` — what to do once the data is in
- `live-data-artifacts` — the connector path
- `financial-tables` — designing the table the data lands in
- Host `artifact-capabilities` — optional downloads contract; hide export when absent