Hex Reference Architecture
Architecture
┌────────────────────────────────────────┐
│ Orchestration Layer │
│ (Airflow, Dagster, GitHub Actions, │
│ Cron, Custom API) │
├────────────────────────────────────────┤
│ Hex API Client │
│ (Run, Poll, Cancel, List) │
├────────────────────────────────────────┤
│ Hex Platform │
│ ┌──────────┐ ┌───────────────────┐ │
│ │ Projects │ │ Data Connections │ │
│ │ (SQL, │ │ (Snowflake, │ │
│ │ Python, │ │ BigQuery, │ │
│ │ R) │ │ Postgres, etc.) │ │
│ └──────────┘ └───────────────────┘ │
└────────────────────────────────────────┘
Project Structure
hex-orchestrator/
├── src/hex/
│ ├── client.ts # API client
│ ├── orchestrator.ts # Pipeline runner
│ ├── scheduler.ts # Cron-based triggers
│ └── types.ts # TypeScript interfaces
├── src/notify/
│ └── slack.ts # Completion notifications
├── tests/
├── config/
│ └── pipelines.json # Pipeline definitions
└── .env.example
Integration Patterns
| Pattern |
When |
Tool |
| CI-triggered refresh |
On deploy |
GitHub Actions |
| Scheduled pipeline |
Daily/weekly reports |
Cron, Airflow |
| On-demand run |
User-triggered analysis |
API endpoint |
| Orchestrated pipeline |
Multi-step ETL |
Airflow, Dagster |
Overview
This architecture separates project triggers, parameter validation, scoped execution, bounded orchestration, aggregate observability, and cancellation/rollback. Workspace data remains inside approved project boundaries and never becomes architectural telemetry.
Prerequisites
- A project/data owner for each source, environment allowlist, secret manager, and owner for every execution edge.
- Separate sandbox/staging/production configuration plus rollback for project, schedule, queue, and client controls.
Instructions
- Map every trigger-to-project edge with owner, data class, allowed parameters, authorization scope, retry behavior, observability, and rollback.
- Start with a sandbox project and fail closed on unknown project, parameter schema, destination, or response shape.
- Make asynchronous execution idempotent and bounded; quarantine uncertainty rather than resubmitting or exporting output for debugging.
- Canary one project with aggregate signals before promotion and retain the prior revision.
- Re-evaluate controls whenever credentials, project ownership, schedules, or parameters change.
Output
Produce an architecture record with owners, opaque project IDs, policy revisions, idempotency/retry behavior, observability, test evidence, and rollback revision. Exclude SQL, output, credentials, and identities.
Error Handling
Stop on unknown scope/destination, failed redaction, non-idempotent retry, or authorization drift. Quarantine the event and restore the prior controlled path instead of adding a broad fallback.
Examples
source=scheduled-sandbox; project=proj-sandbox-12; params=r4; trigger=approved; probe=pass; rollback=arch-r17 is a reviewable architecture receipt.
Resources
1---2name: hex-reference-architecture3description: Implement Hex reference architecture with best-practice project layout. Use when designing new Hex integrations, reviewing project structure, or establishing architecture standards for Hex applications. Trigger with phrases like "hex architecture", "hex best practices", "hex project structure", "how to organize hex", "hex layout".4license: MIT5---6# Hex Reference Architecture
7
8## Architecture
9
10```
11┌────────────────────────────────────────┐
12│ Orchestration Layer │
13│ (Airflow, Dagster, GitHub Actions, │
14│ Cron, Custom API) │
15├────────────────────────────────────────┤
16│ Hex API Client │
17│ (Run, Poll, Cancel, List) │
18├────────────────────────────────────────┤
19│ Hex Platform │
20│ ┌──────────┐ ┌───────────────────┐ │
21│ │ Projects │ │ Data Connections │ │
22│ │ (SQL, │ │ (Snowflake, │ │
23│ │ Python, │ │ BigQuery, │ │
24│ │ R) │ │ Postgres, etc.) │ │
25│ └──────────┘ └───────────────────┘ │
26└────────────────────────────────────────┘
27```
28
29## Project Structure
30
31```
32hex-orchestrator/
33├── src/hex/
34│ ├── client.ts # API client
35│ ├── orchestrator.ts # Pipeline runner
36│ ├── scheduler.ts # Cron-based triggers
37│ └── types.ts # TypeScript interfaces
38├── src/notify/
39│ └── slack.ts # Completion notifications
40├── tests/
41├── config/
42│ └── pipelines.json # Pipeline definitions
43└── .env.example
44```
45
46## Integration Patterns
47
48| Pattern | When | Tool |
49|---------|------|------|
50| CI-triggered refresh | On deploy | GitHub Actions |
51| Scheduled pipeline | Daily/weekly reports | Cron, Airflow |
52| On-demand run | User-triggered analysis | API endpoint |
53| Orchestrated pipeline | Multi-step ETL | Airflow, Dagster |
54
55## Overview
56
57This architecture separates project triggers, parameter validation, scoped execution, bounded orchestration, aggregate observability, and cancellation/rollback. Workspace data remains inside approved project boundaries and never becomes architectural telemetry.
58
59## Prerequisites
60
61- A project/data owner for each source, environment allowlist, secret manager, and owner for every execution edge.
62- Separate sandbox/staging/production configuration plus rollback for project, schedule, queue, and client controls.
63
64## Instructions
65
661. Map every trigger-to-project edge with owner, data class, allowed parameters, authorization scope, retry behavior, observability, and rollback.
672. Start with a sandbox project and fail closed on unknown project, parameter schema, destination, or response shape.
683. Make asynchronous execution idempotent and bounded; quarantine uncertainty rather than resubmitting or exporting output for debugging.
694. Canary one project with aggregate signals before promotion and retain the prior revision.
705. Re-evaluate controls whenever credentials, project ownership, schedules, or parameters change.
71
72## Output
73
74Produce an architecture record with owners, opaque project IDs, policy revisions, idempotency/retry behavior, observability, test evidence, and rollback revision. Exclude SQL, output, credentials, and identities.
75
76## Error Handling
77
78Stop on unknown scope/destination, failed redaction, non-idempotent retry, or authorization drift. Quarantine the event and restore the prior controlled path instead of adding a broad fallback.
79
80## Examples
81
82`source=scheduled-sandbox; project=proj-sandbox-12; params=r4; trigger=approved; probe=pass; rollback=arch-r17` is a reviewable architecture receipt.
83
84## Resources
85
86- [Hex API](https://learn.hex.tech/docs/api/api-overview)
87- [Airflow Provider](https://github.com/hex-inc/airflow-provider-hex)
88- [Orchestration Blog](https://hex.tech/blog/announcing-orchestration-public-api/)
gabrielmoreira