Scaffold Task

Scaffold a New Background Task

You add a complete Trigger.dev background task: task file, orchestrator, trigger script, test, and config registration.

Process

Step 1: Gather Task Details

If $ARGUMENTS provides a task name, use it as a starting point. Ask the user:

1. Task name (e.g., "enrich-contacts", "check-relevance", "fetch-events")
2. Data source: Which table does this task read from? Which column gates eligibility?
3. Processing type: External API call, LLM call, SQL-only transform, or combination?
4. Rate limit: If external API — what are the provider's rate limits? (requests/second, requests/minute)
5. Batch size: How many items per batch? (suggest defaults based on type — see below)

Read ${CLAUDE_SKILL_DIR}/../../imp_doc/trigger-dev/task-patterns.md for task architecture patterns. Read ${CLAUDE_SKILL_DIR}/../../imp_doc/trigger-dev/pipeline-config.md for config conventions.

Suggested batch sizes by processing type:

Step 2: Generate Task File

Copy from ${CLAUDE_SKILL_DIR}/../../templates/trigger-task.ts and customize:

Create trigger/<task-name>.ts with:

• Task ID: <task-name> (must be globally unique across all task files)
• retry.maxAttempts based on type: API=3, LLM=2, SQL=1
• queue.concurrencyLimit: API=5, LLM=5, SQL=10
• Module-level Pool + drizzle() (never import shared db in trigger files)
• Rate limiter initialization via initRateLimiter() if external API
• Batch query: SELECT eligible rows with status gate, LIMIT to batch size
• Processing loop with error handling per item
• Status column updates: set processing before work, done/error after

Step 3: Generate Orchestrator

Copy from ${CLAUDE_SKILL_DIR}/../../templates/orchestrator.ts and customize:

Create trigger/run-<task-name>.ts with:

• Task ID: run-<task-name>
• Import and trigger the batch task from Step 2
• Convergence loop: run batches until 0 items processed
• Consecutive zero threshold (default: 3)
• Max rounds safety cap (default: 200)
• Pipeline run tracking via startPipelineRun / completePipelineRun

Step 4: Generate Test File

Create tests/unit/<task-name>.test.ts:

import { describe, it, expect, vi } from "vitest";

describe("<task-name>", () => {
  it("should process a valid payload", async () => {
    // Test that a well-formed payload processes correctly
  });

  it("should handle empty batch gracefully", async () => {
    // Test that 0 eligible rows returns { processed: 0 }
  });

  it("should handle API/processing errors per item", async () => {
    // Test that one item failing doesn't abort the batch
  });
});

Step 5: Register in Config

Edit src/lib/pipeline-config.ts:

1. Add batch size to BATCH_DEFAULTS:

   <taskName>: <batchSize>,  // <reasoning>
   

2. If external API, add rate limit to RATE_LIMITS:

   <apiName>: {
     tokensPerSecond: <N>,
     maxBurst: <N>,
     windowMaxRequests: <N>,
     windowSeconds: 60,
   },
   

3. If external API, add timeout to FETCH_TIMEOUTS:

   <apiName>: <milliseconds>,
   

Step 6: Generate Trigger Script

Create scripts/trigger-<task-name>.ts:

import { tasks } from "@trigger.dev/sdk/v3";

async function main() {
  const tenantId = process.argv[2];
  if (!tenantId) {
    console.error("Usage: npx tsx --env-file=.env scripts/trigger-<task-name>.ts <tenantId>");
    process.exit(1);
  }

  const handle = await tasks.trigger("run-<task-name>", { tenantId });
  console.log(`Triggered run-<task-name>: ${handle.id}`);
  console.log(`Monitor: https://cloud.trigger.dev`);
}

main().catch(console.error);

Step 7: Report

Task "<task-name>" scaffolded:

  trigger/<task-name>.ts            — Batch processing task (<batchSize> items/round)
  trigger/run-<task-name>.ts        — Orchestrator (convergence loop)
  scripts/trigger-<task-name>.ts    — Trigger script
  tests/unit/<task-name>.test.ts    — Unit tests

Config updates:
  src/lib/pipeline-config.ts         — Added BATCH_DEFAULTS.<taskName>: <batchSize>
  [if API] src/lib/pipeline-config.ts — Added RATE_LIMITS.<apiName>
  [if API] src/lib/pipeline-config.ts — Added FETCH_TIMEOUTS.<apiName>

Next steps:
  1. Implement the processing logic in trigger/<task-name>.ts
  2. Run tests: npm test -- tests/unit/<task-name>.test.ts
  3. Test locally: swap to dev key, npm run dev, trigger via script
  4. Deploy: npm run deploy

Step 8: Record Learnings

If you encountered errors, workarounds, or non-obvious behaviors, record them to .outbound-builder-plugin-memory.json.

For each learning:

1. Category: error-fix, api-quirk, schema-pattern, config-gotcha, build-pattern, or domain-insight
2. Deterministic ID: mem- + first 8 chars of MD5 hash of category:title
3. Read .outbound-builder-plugin-memory.json (create with {"version":1,"entries":[],"proven_patterns":[]} if missing)
4. Dedup by ID — increment success_count if exists, otherwise append new entry
5. Rebuild proven_patterns from entries with success_count >= 2

Skip if nothing noteworthy happened.

Rules

• Task IDs must be globally unique — check existing files in trigger/ before naming
• Always include rate limiting for external APIs
• Always include per-item error handling — one failed item must not abort the batch
• Status gate columns prevent double-processing — always use them
• Module-level Pool + drizzle() in task files — never import shared db
• Orchestrators track runs via pipeline_runs table for observability
• Reference ${CLAUDE_SKILL_DIR}/../../imp_doc/trigger-dev/task-patterns.md for task patterns
• Reference ${CLAUDE_SKILL_DIR}/../../imp_doc/trigger-dev/pipeline-config.md for config conventions