Objectstack Data

Data Modeling — ObjectStack Data Protocol

Skill Boundaries

Quick Reference — Detailed Rules

For comprehensive documentation with incorrect/correct examples:

• Naming Conventions — snake_case rules, option values, config properties
• Field Types — All 49 field types and configs
• Relationships — lookup vs master_detail, junction patterns, delete behaviors
• Validation Rules — All validation types, script inversion, severity levels
• Index Strategy — btree/gin/gist/fulltext, composite indexes, partial indexes
• Data Lifecycle & Retention — lifecycle classes (record/audit/telemetry/transient/event), retention/TTL/rotation/archive policies; ❗ append-only objects must declare one (distinct from lifecycle hooks below)
• Lifecycle Hooks — the 8 lifecycle events, handler vs sandboxed body (ctx + capability contract), registration, canonical patterns
• Datasources & Federation — defineDatasource, external/federated objects (remoteName/columnMap), auto-connect gating, credentials; ❌ no field.columnName on external objects
• Security & Access Control — permission sets, assignment rows, RLS policies, secret / requiredPermissions, tenancy, platform-global posture

Core Concepts

Object Definition

An Object is the fundamental data entity in ObjectStack. It maps to a database table and exposes automatic CRUD APIs.

Required properties:

sharingModel — the four canonical values (ADR-0090 D4; legacy aliases removed). A custom object that omits it resolves to private at runtime, but the publish door rejects it before that:

Important optional properties:

Object Capabilities (enable)

Toggle system behaviours per object:

Search Fields (searchableFields)

searchableFields on the object is the canonical set $search scans (ADR-0061). Leave it unset and search auto-defaults to the nameField plus the object's short-text and enum columns (text / email / phone / url / autonumber / textarea / markdown / select / status); declare it to pin the set explicitly. Views may narrow it, never widen it.

⛔ Searching by a related record's title — mirror the value, always

$search scans the queried object's own columns. A dotted path is never a search target: unlike fields / sort / filters, the search axis does not resolve traversal, and project_id.name in searchableFields (or in a $searchFields override) is refused, not silently dropped.

This is the one prescription — emit it every time. To search by a related record's title, copy that title into a stored field on this object and declare that field searchable. A task list searched by project name:

// `project_name` is a stored, denormalized mirror of the parent's title.
{
  name: 'task',
  enable: { searchable: true },
  fields: {
    name:         { type: 'text', required: true },
    project_id:   { type: 'lookup', reference: 'project' },
    project_name: { type: 'text', label: 'Project Name' },   // ← the mirror
  },
  searchableFields: ['name', 'project_name'],
}

?search=apollo expands to name $contains 'apollo' OR project_name $contains 'apollo' — one single-table scan, every driver, no traversal. (A text mirror also lands in the auto-default set when the object declares no searchableFields.)

❌ Never mirror onto a formula field. A formula field is virtual — no driver materializes a column for it, so a $contains predicate against one has nothing to scan (the SQL driver would emit a WHERE over a column that does not exist). CEL also only reads this record's own fields (record.<field>), so a formula cannot fetch the related title in the first place. The mistake is refused, not silent: a formula entry in any searchableFields — the object's own set included — is an os validate error (searchable-field-unsearchable), and a request naming one is 400 INVALID_FIELD. It used to clear both and then never match.

Mirror maintenance is the trade-off — a mirror is denormalized data, only as fresh as whatever writes it. Cover both write paths:

Rows written by a path that bypasses hooks (bulk import, direct SQL) need a one-off backfill. See Lifecycle Hooks.

Both spellings are refused loudly: os validate reports searchable-field-unknown, and a request naming the dotted path is 400 INVALID_FIELD. Each message carries its own prescription.

Cross-object search paths are rejected by design, not pending. Do not invent a per-project convention for this — the mirror field is the answer.

Field Groups (MVP)

Organize fields into logical groups (e.g., "Contact Information", "Billing", "System") for forms, detail pages, and editors.

• Declare groups on ObjectSchema.fieldGroups — array order is the display order.
• Assign each field to a group via Field.group, which references an ObjectFieldGroup.key. In-group display order equals the traversal order of fields.
• Group keys must be snake_case; group labels are human-readable.
• Optional per-group: icon, description, and collapse ('none' always open · 'expanded' collapsible, starts open · 'collapsed' collapsible, starts closed — replaces the deprecated defaultExpanded flag, ADR-0085). Groups render identically on forms, modals, and detail pages; for a bespoke single-page layout assign a custom Page instead.

import { ObjectSchema } from '@objectstack/spec/data';

export default ObjectSchema.create({
  name: 'account',
  label: 'Account',
  sharingModel: 'private',

  fieldGroups: [
    { key: 'contact_info', label: 'Contact Information', icon: 'user' },
    { key: 'billing',      label: 'Billing', collapse: 'collapsed' },
    { key: 'system',       label: 'System' },
  ],

  fields: {
    name:       { type: 'text',  required: true, group: 'contact_info' },
    email:      { type: 'email',                  group: 'contact_info' },
    phone:      { type: 'phone',                  group: 'contact_info' },
    vat_id:     { type: 'text',                   group: 'billing' },
    billing_address: { type: 'address',           group: 'billing' },
    created_at: { type: 'datetime', readonly: true, group: 'system' },
    created_by: { type: 'lookup', reference: 'user', readonly: true, group: 'system' },
  },
});

Supported migrations at this layer: add / rename / delete / reorder groups (edit the fieldGroups array), assign a field to a group (edit Field.group). Explicit per-field in-group ordering is deferred to a future iteration.

Conditional Field Rules

Put conditional UI/data-entry rules on the field definition when the rule belongs to the data model and should apply everywhere the field is edited: default forms, Studio-authored forms, inline master-detail grids, public forms, and API-backed writes.

import { P } from '@objectstack/spec';
import { ObjectSchema, Field } from '@objectstack/spec/data';

export const Invoice = ObjectSchema.create({
  name: 'invoice',
  sharingModel: 'private',
  fields: {
    status: Field.select({
      options: [
        { label: 'Draft', value: 'draft' },
        { label: 'Sent', value: 'sent' },
        { label: 'Paid', value: 'paid' },
        { label: 'Void', value: 'void' },
      ],
    }),
    paid_at: Field.datetime({
      visibleWhen: P`record.status == 'paid'`,
      requiredWhen: P`record.status == 'paid'`,
    }),
    locked_total: Field.currency({
      readonlyWhen: P`record.status == 'paid'`,
    }),
  },
});

• Use visibleWhen to hide irrelevant fields in ObjectUI forms.
• Use readonlyWhen for state-locked fields; the ObjectQL write path ignores incoming changes when the predicate is TRUE.
• readonly: true governs the end-user surface, not trusted system writers. A non-system write (REST/UI, and any runAs:'user' flow — the default) has the field stripped from any non-system write; the write reports success but the value never lands. System-context writes — runAs:'system' flows, system hooks, seeds, imports, migrations — are exempt and DO write it. So the pattern "users can't edit this, but automation maintains it" is expressed by declaring the field readonly and running the maintaining flow runAs:'system' (see objectstack-automation), not by removing readonly. Writing a readonly field from a runAs:'user' update_record node is a build-time error (os validate / os build).
• Use requiredWhen for conditional requiredness; the ObjectQL validator enforces it on submit. The conditionalRequired alias was REMOVED in protocol 17 — emitting it is a parse error.
• Choose by intent — invariant or transition gate. A fact true of every stored record is an invariant: a validations[] script rule. A row that violates it is refused on any edit until repaired. A transition condition ("required once the record reaches paid") is requiredWhen / field bounds, which judge the write, not the stored row. "Required when X" reads like an invariant and is not one; an invariant written as a gate never enforces itself.
• For inline master_detail grids, predicates are evaluated row-by-row against the child row's record, so line-item rules should live on child fields.
• For complex predicates, load objectstack-formula and emit CEL via P\...`; do not use Salesforce-style AND, IN (...), or {field}` syntax.

Quick-Start Template

import { ObjectSchema } from '@objectstack/spec/data';

export default ObjectSchema.create({
  name: 'support_case',
  label: 'Support Case',
  pluralLabel: 'Support Cases',
  description: 'A customer-reported issue tracked to resolution.',
  icon: 'life-buoy',
  sharingModel: 'private',                 // required in practice — see above
  highlightFields: ['subject', 'status', 'priority'],
  enable: {
    trackHistory: true,
    feeds: true,
    activities: true,
  },
  fields: {
    subject:     { type: 'text', required: true, maxLength: 255 },
    description: { type: 'richtext' },
    status:      { type: 'select', required: true, options: [
      { label: 'New',       value: 'new', default: true },
      { label: 'Open',      value: 'open' },
      { label: 'Escalated', value: 'escalated', color: '#e74c3c' },
      { label: 'Resolved',  value: 'resolved',  color: '#2ecc71' },
      { label: 'Closed',    value: 'closed' },
    ]},
    priority:    { type: 'select', options: [
      { label: 'Low',    value: 'low' },
      { label: 'Medium', value: 'medium', default: true },
      { label: 'High',   value: 'high',   color: '#e67e22' },
      { label: 'Urgent', value: 'urgent',  color: '#e74c3c' },
    ]},
    account:     { type: 'lookup', reference: 'account', required: true },
    contact:     { type: 'lookup', reference: 'contact' },
    assigned_to: { type: 'lookup', reference: 'user' },
    due_date:    { type: 'datetime' },
  },
  validations: [
    {
      name: 'status_flow',
      type: 'state_machine',
      field: 'status',
      transitions: {
        new:       ['open'],
        open:      ['escalated', 'resolved'],
        escalated: ['open', 'resolved'],
        resolved:  ['open', 'closed'],
        closed:    [],
      },
      message: 'Invalid status transition.',
    },
  ],
});

Declared indexes are a separate decision — see Index Strategy.

Schema evolution on an existing database

The metadata→DB sync is additive-only: new tables/columns are created on boot, but existing columns are never altered or dropped. A non-additive change to an object that already has data silently diverges from the physical schema, and the database column wins at write time:

required is not the NOT NULL dial (ADR-0113). required is the write contract — the engine refuses an insert that omits the value — and it implies nothing about the column. The physical constraint is a separate explicit opt-in, storage: { notNull: true }, and it is what drift detection compares against. So tightening required on a deployed object is safe (existing null rows stay readable), while declaring storage.notNull over null rows is a destructive migration. The two cannot be combined with requiredWhen — a conditional contract cannot be an unconditional column constraint.

Tell-tale: /meta reports a field optional (no required, no storage.notNull) but writes that omit it fail with a raw driver error rather than a clean validation 400 — that is a stale NOT NULL column, not a validator bug. Run os migrate plan to preview and os migrate apply to reconcile, or ratify the column by declaring storage: { notNull: true }. CLI details: see objectstack-platform.

Common Patterns

Naming Rules Summary

See rules/naming.md for incorrect/correct examples.

Field Type Selection

49 types available. Quick categories:

• Text: text, textarea, email, url, phone, password, markdown, html, richtext — ⚠️ password on a generic object is plaintext at rest (masked on read, never hashed); prefer secret for credentials
• Secret: secret — reversible, encrypted-at-rest credential (DB password, API key, token) via the registered ICryptoProvider; masked on read, fail-closed (ADR-0100). The recommended type for credentials
• Numbers: number, currency, percent
• Date/Time: date, datetime, time
• Logic: boolean, toggle
• Selection: select, multiselect, radio, checkboxes
• Relational: lookup, master_detail, tree, user — user is a person picker (a lookup specialized to sys_user; stored identically to lookup)
• Media: image, file, avatar, video, audio
• Calculated: formula, summary, autonumber — formula fields take a CEL expression in expression (use F\...`from@objectstack/spec`); see objectstack-formula skill
• Embedded: composite, repeater, record — embedded JSON sub-objects stored on the parent row (no separate table / FK)
• Enhanced: location, address, code, json, color, rating, slider, signature, qrcode, progress, tags, vector

See rules/field-types.md for full reference.

Relationship Patterns

See rules/relationships.md for detailed examples.

multiple: true lookup ≠ junction object. A multi-value lookup ({ type: 'lookup', reference: 'x', multiple: true }) is stored and read as an array of ids on the record — reference elements positionally ({record.tags.0} in flow values). It is NOT a junction table. Reach for a junction object (two lookups) only when the relationship itself carries attributes (role, added_at, …).

Validation Patterns

⚠️ Script validation is inverted: Validation fails when expression is true.

On insert, an optional field omitted from the payload reads as null in a validation predicate — so record.due_date == null matches an omitted field the same as an explicit null. (On update, the prior record supplies it.)

The complete set of validation types (ValidationRuleSchema discriminators):

• script — Formula expression (inverted logic)
• state_machine — Legal state transitions
• format — Regex or built-in format
• cross_field — Compare values across fields
• json_schema — Validate a JSON field against a JSON Schema
• conditional — Apply a nested rule only when a predicate holds

There is NO unique validation type (removed from the spec). Enforce uniqueness — including composite — with a unique index, and state its scope (ADR-0120): indexes: [{ fields: ['department', 'email'], unique: 'organization' }].

See rules/validation.md for all types and examples.

Index Patterns

The whole declaration surface is fields / unique / name. unique defaults to false; omit it when that is what you mean.

indexes: [
  { fields: ['status', 'created_at'] },                // composite
  { fields: ['email'], unique: 'organization' },       // unique per organization
  { fields: ['hostname'], unique: 'global' },          // unique platform-wide
  { name: 'idx_acct_status', fields: ['status'] },     // custom name
]

type and partial were retired at protocol 17: no driver ever read either, so an authored type chose no access method and an authored partial produced a full index with the predicate discarded. Both are now a tsc error and a parse error; os migrate meta --from 16 strips them. Access methods and partial predicates are database-layer migrations.

A unique index must state its scope — 'organization' (one holder per organization, NULL-safe) or 'global' (one holder across the installation). On a declared index bare unique: true is the deprecated spelling of 'global': it reads like "per organization" and does the opposite, so os lint warns and protocol 18 rejects it. On a FIELD, unique: true means 'organization' and stays valid.

See rules/indexing.md for composite indexes, unique scope, and how to build partial / gin / gist indexes at the database layer.

Lifecycle Hooks

Implement business logic at data operation lifecycle points:

import { defineHook, HookContext } from '@objectstack/spec/data';

export default defineHook({
  name: 'account_defaults',
  object: 'account',
  events: ['beforeInsert'],
  handler: async (ctx: HookContext) => {
    if (!ctx.input.industry) {
      ctx.input.industry = 'Other';
    }
    ctx.input.created_at = new Date().toISOString();
  },
});

The handler above is the inline (in-process) form. The preferred, metadata-native form is a sandboxed body — { language: 'js', source, capabilities } run in an isolated VM, the shape that AI/Studio-authored hooks and every build artifact carry. See references/data-hooks.md for all 8 lifecycle events, both registration forms, the sandboxed body ctx + capability contract, and the canonical patterns.

Object Extension Model

To add fields/validations/indexes to an object you do not own, author defineObjectExtension({ extend, fields, priority }) and register it on defineStack({ objectExtensions: [...] }). priority sets merge order (default 200, range 0–999); an extension can add but never remove. ⛔ Not the same key as object-level ownership (the record-ownership enum 'user' | 'business_unit' | 'org' | 'none'). Schema: node_modules/@objectstack/spec/src/data/object.zod.ts (ObjectExtensionSchema).

Security & Access Control

Per-object access control is authored in permission sets, not on the object schema. There is no object-level permissions key (and no hooks key either) — ObjectSchema.create() rejects both as unknown keys.

Full rules: Security & Access Control.

Access depth (scope-depth) — the ERP "see my unit / my unit and below" axis

On an owner-scoped (private) object, a per-object grant in a permission set may carry readScope / writeScope to widen the owner match declaratively instead of hand-writing an RLS policy (ADR-0057 D1): own (default) · own_and_reports · unit · unit_and_below · org. It resolves at request time into an owner_id IN (…) set and AND-injects like RLS. ⚠️ Open-core boundary (ADR-0016): only own and org work in open source — the hierarchy-relative three need the paid @objectstack/security-enterprise plugin, fail closed to own without it, and defineStack errors unless the grant declares requires: ['hierarchy-security']. Schema: PermissionSetSchema.objects.*.

Metadata Protection (protection)

Package authors can lock shipped metadata against Studio edits / overlays / deletes.

The protection block is declared on the source schema (*.object.ts, *.app.ts, *.view.ts, …) and stripped at load time — it never appears in the runtime envelope. The runtime instead populates _lock, _lockReason, _lockDocsUrl, _lockSource, and _packageId, which REST returns to Studio and the lock banner reads.

Schema

protection?: {
  /** Lock level — controls what Studio can do to this item. */
  lock: 'none' | 'no-overlay' | 'no-delete' | 'full';
  /** REQUIRED — reason shown in the Studio lock banner (1–500 chars). */
  reason: string;
  /** Optional doc URL — renders as a "View docs" link in the banner. */
  docsUrl?: string;
}

The block is .strict(): reason is required (min 1 / max 500 chars) and unknown keys are rejected.

Example

// src/objects/sys-user.object.ts
import { ObjectSchema } from '@objectstack/spec/data';

export const SysUserObject = ObjectSchema.create({
  name: 'sys_user',
  label: 'User',
  sharingModel: 'public_read',
  protection: {
    lock: 'full',
    reason: 'Core identity object',
    docsUrl: 'https://objectstack.ai/docs/references/shared/protection',
  },
  fields: { username: { type: 'text', required: true } },
});

The same block works on non-object metadata (apps, views, dashboards, flows, agents, tools, skills, reports, email-templates). Enforcement: PUT/DELETE on /api/v1/meta/:type/:name return 403 item_locked, and an artifact lock overrides a package lock. Default to no protection block for tenant-authored metadata.

Seed Data & Fixtures (defineSeed())

Object definition and its seed data live together — writing a *.object.ts almost always goes with a *.seed.ts (test fixtures, reference rows, bootstrap data). defineSeed() is type-safe: pass the object definition and TypeScript checks every record's field keys at compile time.

The factory is named defineSeed — not defineDataset. The dataset name is reserved for the unrelated ADR-0021 analytics semantic layer (defineDataset from @objectstack/spec/ui), which is not a seed factory.

Quick start

// src/data/index.ts
import { defineSeed } from '@objectstack/spec/data';
import { Status } from '../objects/status.object';
import { Category } from '../objects/category.object';

// Reference data — every environment
export const statusSeed = defineSeed(Status, {
  externalId: 'code',
  mode: 'upsert',
  records: [
    { code: 'active',   label: 'Active',   color: '#2ecc71' },
    { code: 'inactive', label: 'Inactive', color: '#95a5a6' },
  ],
});

// Demo data — dev/test only
export const categorySeed = defineSeed(Category, {
  externalId: 'slug',
  mode: 'upsert',
  env: ['dev', 'test'],
  records: [
    { slug: 'electronics', name: 'Electronics' },
  ],
});

export const SeedData = [statusSeed, categorySeed];   // parents first

Seed fields

Full Zod shape: node_modules/@objectstack/spec/src/data/seed.zod.ts.

Import modes

externalId selection

Pick a stable natural business key. Never use id — UUIDs differ across environments.

Relationship references

For lookup fields, supply the natural key of the target record (not its UUID). The seed runner resolves at load time. Order seeds so parents appear before children in the exported array:

If a lookup value matches no natural key, the loader now falls back to resolving it as the target's id — so a reference to a real existing record by internal id resolves instead of dangling to null. Natural keys remain the portable default; rely on the id fallback only for records you didn't seed (e.g. a system user).

const contacts = defineSeed(Contact, {
  externalId: 'email',
  records: [{
    email: 'john@acme.example.com',
    first_name: 'John',
    account: 'Acme Corporation',   // natural key of an Account record
  }],
});

Dynamic values (CEL)

Any field value may be a CEL expression evaluated at install time against a single per-load pinned now. This is the only correct way to author time-based or identity-derived seed values — new Date() ships the package author's clock to every customer and breaks build determinism.

import { defineSeed } from '@objectstack/spec/data';
import { cel } from '@objectstack/spec';

defineSeed(Opportunity, {
  records: [{
    name:            'Acme Q3 Renewal',
    close_date:      cel`daysFromNow(45)`,
    created_at:      cel`now()`,
    owner_id:        cel`os.user.id`,   // installer
    organization_id: cel`os.org.id`,
  }],
});

Stdlib in seed context: now(), today(), daysFromNow(n), daysAgo(n), isBlank(v), coalesce(v, fallback). Scope: os.user, os.org, os.env. See objectstack-formula for the full contract.

Determinism gate: two consecutive os build runs with no source changes must produce byte-identical dist/objectstack.json. CEL + pinned now is what guarantees that — using Date.now() will fail CI.

Seed best practices

Linting & Generation Quality

os lint checks the data model against the conventions in this skill — not just naming/labels but the relationship/master-detail/roll-up patterns. Run it after authoring or generating metadata. Severities: error (structural, fails the command), warning (likely-wrong choice), suggestion (nudge).

Data-model rules (in addition to naming/label/i18n):

code counts for R9, but is NOT a title-derivation key. R9's name-like list above is the looser of two "name-like" sets, and the difference is deliberate. R9 asks "will records be anonymous?" — is there any readable face at all — and a code clears that bar. ADR-0079's title derivation (resolveDisplayField) asks the narrower "what IS the title?", and its name-ish set is name/title/subject/label/full_name/display_name without code — an identifier is not a title. So an object whose only name-ish field is code is R9-clean, yet its title is derived by the lower-priority "first title-eligible field by declaration order" tier rather than by name. Nothing user-visible turns on this (R9 is suggestion, and the Record #<id> floor guarantees a title regardless), but do not read the R9 list as the derivation contract — set nameField explicitly when the title matters.

These same rules are the rubric for AI-generated metadata — a generation is "good" exactly when it is schema-valid and lint-clean:

• os lint --score — print a 0–100 metadata-quality score (+ letter grade and severity breakdown) for the current project. Schema errors and lint errors weigh most; suggestions barely move it.
• os lint --eval — run the generation eval over a bundled golden corpus (invoice+lines, project+tasks, blog+comments, expense+lines, account+contacts) offline; each case must clear the pass bar (--eval-min, default 75). Deterministic, no API key.

When generating object metadata, target a lint-clean model: master_detail (with required + deleteBehavior + inlineEdit for line items), roll-up summaries on parents, select options, and a name/title field per object.

Verify your work

After authoring or editing any *.object.ts / *.seed.ts, run the author-time gate before reporting done:

os validate     # Zod schema + CEL predicates (record.<field> existence) + bindings
# or: os build  # the same gates, plus emits dist/

It catches what otherwise fails silently at runtime: a bare field ref in a requiredWhen / readonlyWhen / visibleWhen, a validation rule, a formula, or a row-level-security/sharing predicate (done instead of record.done) that evaluates to null and never fires. os lint is a separate pass that additionally checks the data model against the conventions in this skill (relationships, master-detail, roll-ups) — run it too, but it does not replace os validate. (Reminder: two consecutive os build runs with no source change must be byte-identical — see the determinism gate above.) In a scaffolded project the gate is npm run validate.

References

See references/_index.md for the full list of Zod schemas (with one-line descriptions) — pointers into node_modules/@objectstack/spec/src/. Always Read the source for exact field shapes; do not rely on memory of property names.