Schema Projections

Projections

Add projections to collections to rename fields or map nested JSON paths to flat column names.

Docs: https://docs.estuary.dev/concepts/advanced/projections/

When to Use

• Rename fields to match destination naming conventions
• Flatten nested JSON paths to column names (e.g., /user/name → user_name)
• Extract array elements as named fields

Procedure

Step 1: Pull the Collection Spec

flowctl catalog pull-specs --name <collection-name>

Step 2: Add Projections

Add a projections stanza to the collection spec (top-level, alongside key and schema):

collections:
  tenant/your/collection:
    schema: schema.yaml
    key: [/id]
    projections:
      # Rename a top-level field
      user_id: /id

      # Flatten nested paths
      customer_name: /customer/name
      customer_email: /customer/email

      # Extract array elements
      primary_tag: /tags/0

Values are JSON Pointers: / separated, array indices numeric (/tags/0).

Fields from your schema already show up as columns in materializations automatically. You only need projections when you want to rename a field or expose a nested path as a different column name.

Step 3: Publish Changes

flowctl catalog publish --source flow.yaml

New field names become available in materialization field selection. Add them to fields.require in the materialization binding to include them.

Notes

• Projections are also used for logical partitions (partition: true) — see docs for that use case
• If the underlying field is pruned by the complexity limit, the projection survives but its type may widen (e.g., to JSON) — define the field in the readSchema to pin the type

Troubleshooting

Projection Not Appearing in Materialization

• Ensure JSON pointer path exists in the schema
• Add projected field to fields.require in materialization binding
• Projections are defined on the collection, not the materialization

Invalid JSON Pointer

• Uses / not . for path separation
• Array indices are numeric: /tags/0 not /tags[0]
• Escape ~ as ~0 and / as ~1 in field names

Related Skills

• schema-field-selection — Choose which projected fields to materialize
• schema-custom-types-ddl — Customize column types for projected fields
• schema-field-redaction — Hash or block fields at capture time