Hex Prod Checklist

Hex Production Checklist

Overview

Hex is a collaborative data analytics platform where teams build notebooks, dashboards, and scheduled data pipelines. A production integration triggers project runs, retrieves results, and monitors pipeline health via the Hex API. Failures mean stale dashboards, broken scheduled reports, or data pipelines that silently stop producing output for downstream consumers.

Authentication & Secrets

• HEX_API_KEY stored in secrets manager (not config files)
• Token scoped to "Run projects" permission only
• Token expiration set > 90 days with calendar renewal reminder
• Separate tokens for dev/staging/prod workspaces
• Key rotation schedule documented (quarterly cycle)

API Integration

• Production base URL configured (https://app.hex.tech/api/v1)
• Rate limits respected (20 requests/min, 60/hour)
• All orchestrated projects published before triggering runs
• Input parameters documented and validated before submission
• Run status polling with configurable interval (default: 5s)
• Result retrieval handles large datasets (pagination/streaming)
• Project version pinning prevents unexpected behavior changes

Error Handling & Resilience

• Circuit breaker configured for Hex API outages
• Retry with exponential backoff for 429/5xx responses
• ERRORED and KILLED run states handled with alert + re-queue
• Run timeout detection (alert if run exceeds 2x typical duration)
• Database connection errors in Hex projects surface to caller
• Graceful degradation serves cached results when API is down

Monitoring & Alerting

• API latency tracked per project run
• Error rate alerts set (any ERRORED run = notification)
• Run duration regression tracked (>50% increase = warning)
• Scheduled run completion rate monitored daily
• API quota usage tracked against plan limits

Validation Script

async function checkHexReadiness(): Promise<void> {
  const checks: { name: string; pass: boolean; detail: string }[] = [];
  // API connectivity
  try {
    const res = await fetch('https://app.hex.tech/api/v1/projects', {
      headers: { Authorization: `Bearer ${process.env.HEX_API_KEY}` },
    });
    checks.push({ name: 'Hex API', pass: res.ok, detail: res.ok ? 'Connected' : `HTTP ${res.status}` });
  } catch (e: any) { checks.push({ name: 'Hex API', pass: false, detail: e.message }); }
  // Credentials present
  checks.push({ name: 'API Key Set', pass: !!process.env.HEX_API_KEY, detail: process.env.HEX_API_KEY ? 'Present' : 'MISSING' });
  // Rate limit check
  try {
    const res = await fetch('https://app.hex.tech/api/v1/projects', {
      headers: { Authorization: `Bearer ${process.env.HEX_API_KEY}` },
    });
    const remaining = res.headers.get('x-ratelimit-remaining');
    checks.push({ name: 'Rate Limit', pass: Number(remaining) > 5, detail: `${remaining} remaining` });
  } catch (e: any) { checks.push({ name: 'Rate Limit', pass: false, detail: e.message }); }
  for (const c of checks) console.log(`[${c.pass ? 'PASS' : 'FAIL'}] ${c.name}: ${c.detail}`);
}
checkHexReadiness();

Error Handling

Prerequisites

• Protected production approval, named incident/rollback owners, and a pinned project/configuration revision proven in sandbox and staging.
• Baselines for run health, latency, quota, output shape, and authorized access plus a canary project.

Instructions

1. Confirm environment, secret reference, project scope, artifact revision, and approval before enabling production execution.
2. Run bounded canary parameters and verify health, aggregate output assertions, and least-privilege access without exporting results.
3. Monitor errors, latency, quota, and cancellation signals through the observation window; halt on a scope or output regression.
4. Promote in stages or cancel and restore the previous revision, recording the decision and verified rollback.
5. Close only after the owner accepts the redacted receipt and temporary access is revoked.

Output

Create a readiness receipt with revision, canary project, health/latency/quota outcomes, aggregate assertion, approvals, outcome, and rollback reference. Exclude SQL, output, credentials, and identities.

Examples

revision=r44; canary=proj-sandbox-12; health=pass; quota=within-budget; assertions=pass; outcome=promote; rollback=r43 is a complete canary decision.

Resources

• Hex API Overview
• Hex Status

Next Steps

See hex-security-basics for data connection security and access control.