Optimize Performance

Make a dlt pipeline faster or lighter on memory. Use when the user says a pipeline is slow, takes too long, runs out of memory, uses too much RAM, or wants to optimize, speed up, parallelize, or increase throughput. Covers source-agnostic levers (parallelism, workers, buffers, file rotation); for source-specific tuning use the pipeline toolkit's own optimize skill. Also decides whether a dltHub platform job genuinely needs a larger instance (more memory/CPU) — the last resort after tuning.

dlt-hub Updated 65 repo stars

File contents

dlt-hub/dlthub-ai-harness commit 292a2d393f

Frequently asked questions

npx get-skill add dlt-hub/optimize-performance