1098 Turborepo Pipelines 64279184

Turborepo Task Pipelines

Task orchestration, dependencies, and parallel execution strategies.

Pipeline Configuration

Define tasks in turbo.json:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**", ".next/**"]
    },
    "test": {
      "dependsOn": ["build"],
      "outputs": ["coverage/**"]
    },
    "lint": {},
    "dev": {
      "cache": false,
      "persistent": true
    }
  }
}

Task Dependencies

Topological Dependencies (^)

^ means "run this task in dependencies first":

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"]
    }
  }
}

Example flow:

packages/ui (dependency)
  ↓ builds first
apps/web (depends on @repo/ui)
  ↓ builds second

Internal Dependencies

Run tasks in same package first:

{
  "pipeline": {
    "deploy": {
      "dependsOn": ["build", "test"]
    }
  }
}

Execution order in same package:

1. Run build
2. Run test
3. Run deploy

Combined Dependencies

Mix topological and internal:

{
  "pipeline": {
    "test": {
      "dependsOn": ["^build", "lint"]
    }
  }
}

Execution order:

1. Build all dependencies (^build)
2. Lint current package (lint)
3. Run tests (test)

Task Configuration Options

outputs

Define what gets cached:

{
  "build": {
    "outputs": [
      "dist/**",           // All files in dist
      ".next/**",          // Next.js build
      "!.next/cache/**",   // Exclude Next.js cache
      "build/**",          // Build directory
      "public/dist/**"     // Public assets
    ]
  }
}

cache

Enable/disable caching:

{
  "dev": {
    "cache": false        // Don't cache dev server
  },
  "build": {
    "cache": true         // Cache build (default)
  }
}

persistent

Keep task running (for dev servers):

{
  "dev": {
    "cache": false,
    "persistent": true    // Don't kill after completion
  }
}

env

Environment variables affecting output:

{
  "build": {
    "env": [
      "NODE_ENV",
      "NEXT_PUBLIC_API_URL",
      "DATABASE_URL"
    ]
  }
}

passThroughEnv

Pass env vars without affecting cache:

{
  "build": {
    "passThroughEnv": [
      "DEBUG",            // Pass through but don't invalidate cache
      "LOG_LEVEL"
    ]
  }
}

inputs

Override default input detection:

{
  "build": {
    "inputs": [
      "src/**/*.ts",
      "!src/**/*.test.ts", // Exclude test files
      "package.json"
    ]
  }
}

outputMode

Control output display:

{
  "build": {
    "outputMode": "full"        // Show all output
  },
  "dev": {
    "outputMode": "hash-only"   // Show cache hash only
  },
  "test": {
    "outputMode": "new-only"    // Show new output only
  },
  "lint": {
    "outputMode": "errors-only" // Show errors only
  }
}

Running Tasks

Basic Execution

# Run build in all packages
turbo run build

# Run multiple tasks
turbo run build test lint

# Run with specific package manager
pnpm turbo run build

Filtering

Run tasks in specific packages:

# Single package
turbo run build --filter=web
turbo run build --filter=@repo/ui

# Multiple packages
turbo run build --filter=web --filter=api

# All apps
turbo run build --filter='./apps/*'

# Pattern matching
turbo run test --filter='*-api'

Dependency Filtering

# Package and its dependencies
turbo run build --filter='...web'

# Package's dependencies only (exclude package itself)
turbo run build --filter='...^web'

# Package and its dependents
turbo run test --filter='ui...'

# Package's dependents only
turbo run test --filter='^ui...'

Git-Based Filtering

Run only on changed packages:

# Changed since main branch
turbo run build --filter='[main]'

# Changed since HEAD~1
turbo run build --filter='[HEAD~1]'

# Changed in working directory
turbo run test --filter='...[HEAD]'

# Package and dependencies, only if changed
turbo run build --filter='...[origin/main]'

Concurrency Control

Parallel Execution (Default)

Turborepo runs tasks in parallel when safe:

# Run with default parallelism
turbo run build

Limit Concurrency

# Max 3 tasks at once
turbo run build --concurrency=3

# 50% of CPU cores
turbo run build --concurrency=50%

# No parallelism (sequential)
turbo run build --concurrency=1

Continue on Error

# Don't stop on first error
turbo run test --continue

Task Execution Order

Example monorepo:

apps/
├── web (depends on @repo/ui, @repo/utils)
└── docs (depends on @repo/ui)
packages/
├── ui (depends on @repo/utils)
└── utils (no dependencies)

With config:

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"]
    }
  }
}

Execution order for turbo run build:

1. Wave 1 (parallel): @repo/utils (no dependencies)
2. Wave 2 (parallel): @repo/ui (depends on utils)
3. Wave 3 (parallel): web and docs (both depend on ui)

Complex Pipeline Examples

Full-Stack Application

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".next/**", "dist/**"]
    },
    "test": {
      "dependsOn": ["^build"],
      "outputs": ["coverage/**"]
    },
    "lint": {
      "dependsOn": ["^build"]
    },
    "typecheck": {
      "dependsOn": ["^build"]
    },
    "dev": {
      "cache": false,
      "persistent": true
    },
    "deploy": {
      "dependsOn": ["build", "test", "lint", "typecheck"]
    }
  }
}

Monorepo with Code Generation

{
  "pipeline": {
    "generate": {
      "cache": false,
      "outputs": ["src/generated/**"]
    },
    "build": {
      "dependsOn": ["^build", "generate"],
      "outputs": ["dist/**"]
    },
    "test": {
      "dependsOn": ["generate"],
      "outputs": ["coverage/**"]
    }
  }
}

Database-Dependent Pipeline

{
  "pipeline": {
    "db:generate": {
      "cache": false
    },
    "db:migrate": {
      "cache": false
    },
    "build": {
      "dependsOn": ["^build", "db:generate"],
      "outputs": ["dist/**"]
    },
    "test:unit": {
      "dependsOn": ["build"]
    },
    "test:integration": {
      "dependsOn": ["db:migrate"],
      "cache": false
    }
  }
}

Dry Run

Preview execution without running:

# See what would run
turbo run build --dry-run

# JSON output for scripts
turbo run build --dry-run=json

# Show full task graph
turbo run build --graph

Force Execution

Ignore cache and run tasks:

# Force rebuild everything
turbo run build --force

# Force specific package
turbo run build --filter=web --force

Output Control

# Show only errors
turbo run build --output-logs=errors-only

# Show new logs only
turbo run build --output-logs=new-only

# Show cache hash only
turbo run build --output-logs=hash-only

# Show full output
turbo run build --output-logs=full

Best Practices

1. Use topological dependencies - ^build ensures correct build order
2. Cache build outputs - Define outputs for faster rebuilds
3. Disable cache for dev - Set cache: false for dev servers
4. Mark persistent tasks - Use persistent: true for long-running tasks
5. Filter strategically - Use filters to run only affected tasks
6. Control concurrency - Limit parallelism for resource-intensive tasks
7. Configure env vars - Include vars that affect output in env
8. Use dry-run - Preview execution plan before running
9. Continue on error in CI - Use --continue to see all errors
10. Leverage git filtering - Run only on changed packages in CI

Common Patterns

CI/CD Pipeline

# .github/workflows/ci.yml
jobs:
  build:
    steps:
      - run: turbo run build test lint --filter='...[origin/main]'

Only build/test/lint changed packages and their dependents.

Development Workflow

# Start all dev servers
turbo run dev

# Start specific app with dependencies
turbo run dev --filter=web...

Pre-commit Hook

// package.json
{
  "scripts": {
    "pre-commit": "turbo run lint test --filter='...[HEAD]'"
  }
}

Only lint/test changed packages.

Deployment

# Build and test specific app
turbo run build test --filter=web...

# Deploy if successful
turbo run deploy --filter=web

Build app and its dependencies, then deploy.