How do I install the Pipeline skill?

Run npx skillmds add majiayu000/pipeline-2 in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Pipeline safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Pipeline?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Pipeline free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Pipeline?

majiayu000 (@majiayu000) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Pipeline by majiayu000 — Agent Skill

Name: Pipeline
Rating: 4.2 (5 reviews)
Author: majiayu000

File contents

Pipeline Operations

Available Sources

Source	Description	Schedule
`everflow`	Affiliate conversions/revenue	Hourly 7am-11pm ET
`redtrack`	Ad spend tracking	Hourly 7am-11pm ET
`s3_exports`	CSV files from S3 (Sticky.io)	Daily 6am ET
`posthog`	PostHog analytics	Stubbed
`mautic`	Mautic contacts/campaigns	Stubbed
`google_sheets`	Google Sheets data	Stubbed

API Reference: See docs/API_REFERENCE.md for live docs, auth, and request/response examples.

Run Pipeline Directly (Local Testing)

# Activate environment
source .venv/bin/activate

# Run a specific source
python scripts/run_pipeline.py everflow
python scripts/run_pipeline.py redtrack
python scripts/run_pipeline.py s3_exports

# Dry run (see what would happen)
python scripts/run_pipeline.py everflow --dry-run

Trigger via Temporal (Production)

# Trigger and return immediately
python scripts/trigger_workflow.py everflow

# Trigger and wait for completion
python scripts/trigger_workflow.py everflow -w

# Trigger with notification on success
python scripts/trigger_workflow.py everflow -w --notify

Check Pipeline Status

# View worker logs (local)
make logs-worker

# View Fly.io logs (production)
fly logs

# Temporal Cloud UI
# https://cloud.temporal.io/namespaces/signalroom-713.nzg5u/workflows

Active Schedules

View in Temporal Cloud UI or:

python scripts/setup_schedules.py --list

Current schedules:

hourly-sync-everflow-redtrack - Hourly 7am-11pm ET
daily-sync-s3 - Daily 6am ET

Troubleshooting

"Unknown source" error: Source not registered in src/signalroom/pipelines/runner.py

Credentials error: Check .env has correct API keys

Database connection failed:

Verify Supabase pooler settings (port 6543)
User must be postgres.{project_ref}

Activity timeout: Pipeline took >30 min. Filter resources or increase timeout.

---name: pipeline-2description: Run dlt pipelines and Temporal workflows for SignalRoom. Use when syncing data sources, triggering workflows, checking pipeline status, or debugging data ingestion issues.--- # Pipeline Operations ## Available Sources | Source | Description | Schedule | |--------|-------------|----------| | `everflow` | Affiliate conversions/revenue | Hourly 7am-11pm ET | | `redtrack` | Ad spend tracking | Hourly 7am-11pm ET | | `s3_exports` | CSV files from S3 (Sticky.io) | Daily 6am ET | | `posthog` | PostHog analytics | Stubbed | | `mautic` | Mautic contacts/campaigns | Stubbed | | `google_sheets` | Google Sheets data | Stubbed | **API Reference**: See `docs/API_REFERENCE.md` for live docs, auth, and request/response examples. ## Run Pipeline Directly (Local Testing) ```bash # Activate environment source .venv/bin/activate # Run a specific source python scripts/run_pipeline.py everflow python scripts/run_pipeline.py redtrack python scripts/run_pipeline.py s3_exports # Dry run (see what would happen) python scripts/run_pipeline.py everflow --dry-run ``` ## Trigger via Temporal (Production) ```bash # Trigger and return immediately python scripts/trigger_workflow.py everflow # Trigger and wait for completion python scripts/trigger_workflow.py everflow -w # Trigger with notification on success python scripts/trigger_workflow.py everflow -w --notify ``` ## Check Pipeline Status ```bash # View worker logs (local) make logs-worker # View Fly.io logs (production) fly logs # Temporal Cloud UI # https://cloud.temporal.io/namespaces/signalroom-713.nzg5u/workflows ``` ## Active Schedules View in Temporal Cloud UI or: ```bash python scripts/setup_schedules.py --list ``` Current schedules: - `hourly-sync-everflow-redtrack` - Hourly 7am-11pm ET - `daily-sync-s3` - Daily 6am ET ## Troubleshooting **"Unknown source" error:** Source not registered in `src/signalroom/pipelines/runner.py` **Credentials error:** Check `.env` has correct API keys **Database connection failed:** - Verify Supabase pooler settings (port 6543) - User must be `postgres.{project_ref}` **Activity timeout:** Pipeline took >30 min. Filter resources or increase timeout.

Frequently asked questions