Data Programs
A data program is a named, stored, agent-authored JS script that fetches,
joins, filters, or aggregates data — provider APIs via providerFetch /
providerFetchAll / providerSearchAll, app data via appAction, or
Resources-backed workspace files — and emits a small { rows, schema }
result that gets cached in SQL. Any app can render that result through its
own table/chart components; nothing about the primitive is analytics-specific.
This ships automatically wherever run-code is available — there is nothing
to mount, wire, or configure per app. If your app already has run-code
enabled, save-data-program / preview-data-program / run-data-program /
list-data-programs / get-data-program / delete-data-program are already
registered actions.
Why this exists
Without it, "build a live view of X" tends to end in one of two bad places:
a hardcoded action per vendor/query shape (brittle, doesn't scale to every
customer's custom fields), or re-running the same fetch/join logic from
scratch on every page view (slow, and it re-spends the same provider quota
every time someone looks at a chart). A data program is the middle ground:
write the fetch/join/aggregate logic once as ordinary run-code, save it, and
let the cache + refresh policy handle "how often does this actually need to
re-run."
Authoring workflow
- Prototype the fetch/join/aggregate logic as a normal
run-code script and
confirm it returns the rows you expect.
- Change it to call
emit(rows, schema?) exactly once at the end instead of
returning or printing the result.
- Save it with
save-data-program({ name, title, description, code, defaultParams?, refreshMode?, refreshTtlMs?, background? }). This
dry-runs the code with defaultParams before persisting and rejects
the save with a structured error if it fails, so a broken program is never
stored. On success it returns { programId, rowCount, columns, sampleRows } — use that as proof the program produces the rows you
expect.
- Bind whatever UI needs the result. Your app's own read path calls
run-data-program({ programId, params?, forceRefresh?, includeRows? })
and gets back rows/schema/cache metadata to hand to your own table/chart
components — there is no dashboard-specific plumbing baked into the
primitive itself. (The analytics template's "program" dashboard panel
source is one such adoption, not a required pattern — see its
data-programs skill if you want a concrete UI wiring reference.)
- Iterate without persisting via
preview-data-program({ code, params? }) —
same dry-run path, no stored row.
- Read a result without running anything via
get-data-program({ programId, includeRows? }). Its lastRun is the
newest finished run, never one still queued; when that run failed,
lastGoodRun holds the newest successful one.
The sandbox surface
Exactly the run-code globals (providerFetch, providerFetchAll,
providerSearchAll, appAction, webFetch, workspace* Resources helpers),
plus:
params — a frozen global object holding the params for this run.
Read-only; mutating it throws.
emit(rows, schema?) — call exactly once, at the end. rows must be an
array of plain objects. schema, when provided, is
{ name: string, type: string }[]; when omitted it's inferred from the
first 50 rows (a column is "json" if rows disagree on primitive type,
otherwise "number" / "string" / "boolean"). console.log remains free
for debugging and never interferes with parsing the emitted result.
Caps enforced on every run:
| Limit |
Value |
| Max emitted rows |
10,000 (excess dropped, truncated: true) |
| Max emitted result size |
4 MiB (rows dropped from the end to fit; a single oversized row is a hard failure) |
| Max active programs per app |
200 |
| Minimum refresh TTL |
60,000 ms |
| Run rows kept per (program, params) |
5 most recent |
Truncation is always explicit (truncated: true) — never a silent drop.
Caching and refresh model
Every (programId, paramsHash) pair has its own run history:
| Situation |
Behavior |
Fresh cache (younger than refreshTtlMs, or refreshMode: "manual" with a prior success) |
Returns cached rows instantly, cacheHit: true |
| Stale cache |
Re-executes synchronously (25s budget for view-triggered reads, 120s for agent/manual calls), replacing the cache on success |
refreshMode: "manual" |
Only refreshes on explicit forceRefresh: true |
| No cache yet |
Executes synchronously like a normal first fetch |
background: true program with no fresh cache |
Serves the last good run with stale: true and enqueues a durable background execution; a later call finalizes it once complete |
| Background run has just finished |
The next read finalizes it, even while older rows are fresh, and serves its rows as a fresh hit, queueing nothing until the TTL lapses or a refresh is forced; a failed one answers reads with its failure for that same window |
| Background program still running, no prior success |
background_pending failure — surface an explicit "still computing" state, never a blank result |
| Execution fails |
Structured { code, message } failure; a previous success is attached as lastGoodRun for stale-serve. A script that throws reports program_error with its own message; sandbox_error means the sandbox itself failed |
| Program archived (soft-deleted) |
Explicit archived failure |
Use background: true only for scripts that routinely exceed the foreground
timeout (large multi-provider joins, deep pagination). Everything else should
stay foreground.
Security notes
- Credentials are always the calling viewer's, never the program author's.
providerFetch inside a program resolves auth using the caller's own
request context. A viewer without a configured provider key sees that
provider's normal auth error on that program's result — never another
user's data.
- Programs are shareable org-internal only — never public. Sharing a
program means teammates can view its cached result; because execution uses
the viewer's credentials, only bind to or view a program you trust.
- Raw provider tokens never reach the model or the browser. Only the
{ rows, schema } an emit() call produces ever leaves the sandbox.
- The mutating/executing actions (
save-data-program, preview-data-program,
run-data-program, delete-data-program) are not callable from a
sandboxed extension/iframe bridge — only from the agent or a server-side
read path.
Related skills
actions for run-code conventions and the sandbox globals this primitive
is built on.
sharing for the org-internal, never-public sharing model programs use.
security for credential-handling and access-scoping invariants.
1---2name: data-programs3description: >-4 Save a run-code fetch/join/aggregate script as a stored, refreshable data5 source any app's own charts/tables can render, instead of a hardcoded6 provider action or a per-view re-fetch. Use when an ad-hoc run-code or7 provider-api-request analysis should become a live, cached data source other8 users or panels can reuse.9metadata:10 source: https://skillmd.com/skills/builderio/data-programs11---1213# Data Programs1415A **data program** is a named, stored, agent-authored JS script that fetches,16joins, filters, or aggregates data — provider APIs via `providerFetch` /17`providerFetchAll` / `providerSearchAll`, app data via `appAction`, or18Resources-backed workspace files — and emits a small `{ rows, schema }`19result that gets cached in SQL. Any app can render that result through its20own table/chart components; nothing about the primitive is analytics-specific.2122This ships automatically wherever `run-code` is available — there is nothing23to mount, wire, or configure per app. If your app already has run-code24enabled, `save-data-program` / `preview-data-program` / `run-data-program` /25`list-data-programs` / `get-data-program` / `delete-data-program` are already26registered actions.2728## Why this exists2930Without it, "build a live view of X" tends to end in one of two bad places:31a hardcoded action per vendor/query shape (brittle, doesn't scale to every32customer's custom fields), or re-running the same fetch/join logic from33scratch on every page view (slow, and it re-spends the same provider quota34every time someone looks at a chart). A data program is the middle ground:35write the fetch/join/aggregate logic once as ordinary `run-code`, save it, and36let the cache + refresh policy handle "how often does this actually need to37re-run."3839## Authoring workflow40411. Prototype the fetch/join/aggregate logic as a normal `run-code` script and42 confirm it returns the rows you expect.432. Change it to call `emit(rows, schema?)` exactly once at the end instead of44 returning or printing the result.453. Save it with `save-data-program({ name, title, description, code,46 defaultParams?, refreshMode?, refreshTtlMs?, background? })`. This47 **dry-runs the code with `defaultParams` before persisting** and rejects48 the save with a structured error if it fails, so a broken program is never49 stored. On success it returns `{ programId, rowCount, columns,50 sampleRows }` — use that as proof the program produces the rows you51 expect.524. Bind whatever UI needs the result. Your app's own read path calls53 `run-data-program({ programId, params?, forceRefresh?, includeRows? })`54 and gets back rows/schema/cache metadata to hand to your own table/chart55 components — there is no dashboard-specific plumbing baked into the56 primitive itself. (The analytics template's `"program"` dashboard panel57 source is one such adoption, not a required pattern — see its58 `data-programs` skill if you want a concrete UI wiring reference.)595. Iterate without persisting via `preview-data-program({ code, params? })` —60 same dry-run path, no stored row.616. Read a result without running anything via62 `get-data-program({ programId, includeRows? })`. Its `lastRun` is the63 newest finished run, never one still queued; when that run failed,64 `lastGoodRun` holds the newest successful one.6566## The sandbox surface6768Exactly the `run-code` globals (`providerFetch`, `providerFetchAll`,69`providerSearchAll`, `appAction`, `webFetch`, `workspace*` Resources helpers),70plus:7172- `params` — a **frozen** global object holding the params for this run.73 Read-only; mutating it throws.74- `emit(rows, schema?)` — call exactly once, at the end. `rows` must be an75 array of plain objects. `schema`, when provided, is76 `{ name: string, type: string }[]`; when omitted it's inferred from the77 first 50 rows (a column is `"json"` if rows disagree on primitive type,78 otherwise `"number"` / `"string"` / `"boolean"`). `console.log` remains free79 for debugging and never interferes with parsing the emitted result.8081Caps enforced on every run:8283| Limit | Value |84| --- | --- |85| Max emitted rows | 10,000 (excess dropped, `truncated: true`) |86| Max emitted result size | 4 MiB (rows dropped from the end to fit; a single oversized row is a hard failure) |87| Max active programs per app | 200 |88| Minimum refresh TTL | 60,000 ms |89| Run rows kept per (program, params) | 5 most recent |9091Truncation is always explicit (`truncated: true`) — never a silent drop.9293## Caching and refresh model9495Every `(programId, paramsHash)` pair has its own run history:9697| Situation | Behavior |98| --- | --- |99| Fresh cache (younger than `refreshTtlMs`, or `refreshMode: "manual"` with a prior success) | Returns cached rows instantly, `cacheHit: true` |100| Stale cache | Re-executes synchronously (25s budget for view-triggered reads, 120s for agent/manual calls), replacing the cache on success |101| `refreshMode: "manual"` | Only refreshes on explicit `forceRefresh: true` |102| No cache yet | Executes synchronously like a normal first fetch |103| `background: true` program with no fresh cache | Serves the last good run with `stale: true` and enqueues a durable background execution; a later call finalizes it once complete |104| Background run has just finished | The next read finalizes it, even while older rows are fresh, and serves its rows as a fresh hit, queueing nothing until the TTL lapses or a refresh is forced; a failed one answers reads with its failure for that same window |105| Background program still running, no prior success | `background_pending` failure — surface an explicit "still computing" state, never a blank result |106| Execution fails | Structured `{ code, message }` failure; a previous success is attached as `lastGoodRun` for stale-serve. A script that throws reports `program_error` with its own message; `sandbox_error` means the sandbox itself failed |107| Program archived (soft-deleted) | Explicit `archived` failure |108109Use `background: true` only for scripts that routinely exceed the foreground110timeout (large multi-provider joins, deep pagination). Everything else should111stay foreground.112113## Security notes114115- **Credentials are always the calling viewer's, never the program author's.**116 `providerFetch` inside a program resolves auth using the caller's own117 request context. A viewer without a configured provider key sees that118 provider's normal auth error on that program's result — never another119 user's data.120- **Programs are shareable org-internal only — never public.** Sharing a121 program means teammates can view its cached result; because execution uses122 the *viewer's* credentials, only bind to or view a program you trust.123- Raw provider tokens never reach the model or the browser. Only the124 `{ rows, schema }` an `emit()` call produces ever leaves the sandbox.125- The mutating/executing actions (`save-data-program`, `preview-data-program`,126 `run-data-program`, `delete-data-program`) are not callable from a127 sandboxed extension/iframe bridge — only from the agent or a server-side128 read path.129130## Related skills131132- `actions` for `run-code` conventions and the sandbox globals this primitive133 is built on.134- `sharing` for the org-internal, never-public sharing model programs use.135- `security` for credential-handling and access-scoping invariants.