Results for “data-lineage”

Design leakage-safe feature engineering strategies for tabular/time-series datasets. Use when: (1) preparing model-ready features, (2) selecting transformations and encodings, (3) documenting feature lineage. NOT for: model serving or infra provisioning.

Wire ETL, ingestion, cron, edge-function, and queue jobs correctly. Use for "build a pipeline", "sync X into Y", "nightly aggregation", "cron double-counts", "dedupe", "backfill", "the numbers are wrong after a retry". Bakes in idempotency, atomic writes, data contracts, dead-letter, and observability.

Build scalable data pipelines, modern data warehouses, and real-time streaming architectures. Implements Apache Spark, dbt, Airflow, and cloud-native data platforms. Use PROACTIVELY for data pipeline design, analytics infrastructure, or modern data stack implementation.

Use for the search-planning stage of a literature review, when database lanes and language layers must be chosen, when DOI coverage policy is unclear, or when inclusion and exclusion criteria need drafting before any source is read; not for running the screening and PRISMA pipeline of a formal review, which is prisma-scoping-review-pipeline.

Use this skill when conditioning, loading, preparing, or labeling signal data for analysis or ML training. Covers: cleaning a single signal (fill gaps, remove drift, deoutlier, denoise, resample/align a time base) BEFORE analysis; building a `signalDatastore` pipeline; creating a `labeledSignalSet` for Signal Labeler; deriving labels (filename, folder, in-file, ROI, time-frequency ROI); stratified train/val/test splits; framing long signals; parallel processing; and shaping datastore output for `trainnet`. Triggers include "clean up this signal", "remove drift / detrend", "fill gaps", "remove spikes / outliers", "denoise", "resample to a uniform rate", "align channels", "labels from filenames", "stratified split", "prepare for Signal Labeler", and function names like `fillgaps`, `fillmissing`, `detrend`, `filloutliers`, `smoothdata`, `resample`, `synchronize`, `signalDatastore`, `labeledSignalSet`, `filenames2labels`, `folders2labels`, `splitlabels`, `framesig`, `framelbl`, `createDatastores`.

Data pipeline specialist for ML experiments. Use when the user asks to talk to Gekko, requests the data engineer, or needs DataLoader optimization.

Data Pipeline Architect

Data pipeline patterns — ETL/ELT, batch vs streaming, idempotency, orchestration. Use when designing a data pipeline, choosing between batch and streaming, implementing ingestion or transformation, setting up orchestration, or debugging pipeline failures.

Manages and monitors dbt projects, model runs, and test results via dbt CLI and dbt Cloud API, covering run status, test failures, source freshness, and manifest analysis.