DevOps & Infra

ivanshamaev Skill

dbt Core (adapter-agnostic) — project structure, models (SQL/Python), sources and refs, materializations (table/view/incremental/ephemeral), incremental strategies (append/merge/delete+insert), is_incremental() macro, snapshots (SCD Type 2), seeds, tests (generic/singular/severity), Jinja macros, dbt-utils package, node selection syntax (graph operators), hooks, exposures, metrics (MetricFlow), CI/CD (slim CI with state:modified+), dbt_project.yml, profiles.yml, multi-adapter support (Spark/PostgreSQL/ClickHouse/BigQuery)

ivanshamaev Skill

Redpanda Kafka-compatible streaming — cluster setup, topic config, rpk CLI, producer/consumer tuning, tiered storage, Schema Registry, Kafka Connect compatibility, monitoring, Docker/Kubernetes deployment

ivanshamaev Skill

SQLFluff SQL linter — .sqlfluff config, dialect selection (ansi/bigquery/clickhouse/duckdb/hive/postgres/snowflake/sparksql/trino), rule sets, templating (Jinja/dbt), fix command, VS Code integration, pre-commit hook, CI/CD GitHub Actions, custom rules, noqa inline suppression

ivanshamaev Skill

Use when writing, configuring, or optimizing dbt projects targeting Trino or Starburst — covering profiles.yml setup, all authentication methods, materializations (table/view/incremental/materialized_view/ephemeral), incremental strategies (append/merge/delete+insert), table properties (format/partitioning/sorted_by), on_schema_change, seeds, snapshots, grants, session properties, data modeling patterns (Kimball/staging/intermediate/mart), dbt project structure, tests, and CI/CD.

ivanshamaev Skill

Root cause analysis for DataOps incidents — 5-Why analysis for pipeline failures, failure taxonomy (infrastructure/data/logic/dependency/config/concurrency), Airflow diagnosis (task state SQL/scheduler heartbeat/DagBag errors), Spark diagnosis (OOM/skew/FetchFailed/serialization), Kafka consumer lag spike RCA, data quality anomaly investigation (volume/freshness/distribution shift), log correlation across components, timeline reconstruction, impact quantification SQL

ivanshamaev Skill

Docker best practices — multi-stage builds (builder/runtime separation), layer cache optimization (dependency install before source copy), minimal base images (distroless/alpine/slim), security hardening (non-root USER, read-only FS, no SUID binaries), .dockerignore patterns, BuildKit secrets for private registries, image vulnerability scanning (Trivy), COPY vs ADD, CMD vs ENTRYPOINT patterns, data engineering Dockerfiles (dbt/Spark/Python ETL)

ivanshamaev Skill

Kafka production platform review — broker configuration (replication factor/min.insync.replicas/rack awareness/KRaft mode), topic design (partition count formula/compaction/retention), consumer group management (lag monitoring/rebalance tuning/cooperative sticky), producer tuning (acks/idempotence/compression/batching), security (SASL_SSL/ACLs/mTLS), JMX metrics to Prometheus (kafka-exporter/JMX exporter), alert rules (under-replicated partitions/ISR shrink/consumer lag/disk), capacity planning, Strimzi Kubernetes operator

ivanshamaev Skill

DataOps CI/CD pipeline review — pipeline stages for data projects (lint/test/build/scan/deploy), trunk-based vs GitFlow branching, environment promotion (dev→staging→prod), artifact versioning (Docker image tags/dbt manifest), deployment gating (DQ checks/smoke tests/manual approval), rollback strategy, pipeline observability (DORA metrics), reusable workflow patterns, monorepo vs polyrepo CI strategy

ivanshamaev Skill

Multi-cloud governance — cloud-agnostic data platform patterns, federated identity (OIDC/SAML between AWS/GCP/Azure), Terraform multi-cloud modules, cross-cloud data replication (S3↔GCS/Azure), unified cost management (FinOps Foundation framework), cloud-agnostic observability (OpenTelemetry), policy enforcement (OPA Gatekeeper across clouds), disaster recovery cross-cloud, vendor lock-in avoidance (open formats Iceberg/Parquet), centralized secrets management (HashiCorp Vault)

ivanshamaev Skill

Blue/green deployment for data platforms — Kubernetes blue/green with service selector swap, Argo Rollouts blue-green strategy, database schema migration compatibility (backward-compatible DDL), dbt blue/green schema swap (swap_schema macro), Airflow DAG versioning (dag_id with version suffix), traffic cutover and rollback procedure, smoke tests before cutover, Canary vs blue/green decision guide

ivanshamaev Skill

Jenkins modernization for DataOps — migration from Freestyle to Declarative Pipeline, Jenkinsfile best practices, shared libraries (vars/src structure), parallel stages, Kubernetes agent pods (docker-in-docker vs kaniko), credential management (withCredentials/Jenkins Credential Store), pipeline-as-code with Multibranch Pipeline, Blue Ocean UI, migration path to GitHub Actions, JCasC (Jenkins Configuration as Code)

ivanshamaev Skill

Self-healing data platform — auto-restart failed Airflow DAGs (idempotency required), Kafka consumer group auto-resume after partition rebalance, circuit breaker for flapping pipelines, automatic partition backfill on gap detection, auto-scale Kubernetes resources on OOM detection, stale statistics auto-ANALYZE, DQ gate auto-quarantine pattern, dead letter queue reprocessing, watchdog agents for heartbeat monitoring, self-healing Airflow sensor (RoutineLoadLagSensor pattern)

ivanshamaev Skill

Alert fatigue reduction — SLO-based alerting (burn rate vs threshold), multi-window burn rate alerts (short/long window), AlertManager inhibition and silencing, alert deduplication (group_by), routing by severity (PagerDuty for critical / Slack for warning), alert ownership labels, runbook links in annotations, alert review process (weekly noise audit), flapping detection (for/pending period tuning), absent() vs rate() alerting patterns, ticket-based escalation for warning-level noise

ivanshamaev Skill

Network security review for data platforms — Kubernetes NetworkPolicy (default-deny + allow patterns), VPC/subnet design (private subnets for all data services), security group rules audit (0.0.0.0/0 detection), TLS everywhere (Kafka mTLS/Trino HTTPS/DB SSL), service mesh (Istio mTLS), DNS security (private hosted zones), egress filtering (no unrestricted outbound), VPC peering vs PrivateLink, network flow logs analysis, firewall rules review

ivanshamaev Skill

Prometheus optimization — recording rules (level:metric:operations naming), cardinality explosion diagnosis (tsdb status API), scrape interval tuning, remote write configuration (batch size/queue capacity), Thanos/Mimir for long-term retention, federation vs remote write, AlertManager routing trees (inhibition/silencing), absent() alerts for missing series, relabeling to drop expensive labels, TSDB compaction, ServiceMonitor vs PodMonitor patterns

ivanshamaev Skill

Terraform security scanning — tfsec static analysis (AWS/GCP/Azure misconfigurations), Checkov IaC policy checks (750+ rules, CIS benchmarks), tflint security rules, KICS scanner, S3 encryption/public access/versioning checks, IAM least-privilege patterns, security group rule review (0.0.0.0/0), KMS key rotation, VPC flow logs, CloudTrail enabled, pre-commit hooks, SARIF output for GitHub Security tab, policy-as-code with OPA/Sentinel

ivanshamaev Skill

Airflow cost optimization — KubernetesPodOperator right-sizing (request vs actual CPU/memory), spot node tolerations for batch tasks, task consolidation (reduce pod-per-task overhead), pool-based concurrency control, idle worker cleanup, log retention policy (S3 lifecycle), metadata DB right-sizing, CeleryExecutor worker autoscaling (KEDA Kafka/queue depth), DAG run frequency analysis (oversceduled DAGs), cloud cost attribution per DAG

ivanshamaev Skill

AWS data platform review — S3 data lake (lifecycle/replication/encryption/access points), EMR vs Glue ETL trade-offs, MSK Kafka configuration, RDS/Aurora for metadata, Redshift vs Athena for analytics, MWAA (managed Airflow), EKS for containerized pipelines, IAM roles (IRSA for EKS/EMREC2 instance profiles), Lake Formation row/column security, AWS Glue catalog, VPC data platform networking, cost optimization (S3 Intelligent-Tiering/Spot instances)

ivanshamaev Skill

GCP data platform review — BigQuery (dataset IAM/column-level security/partitioned tables/clustering/reservations), GCS data lake (uniform bucket-level access/lifecycle/CMEK), Dataproc vs Dataflow vs Spark on GKE, Pub/Sub streaming, Cloud Composer (managed Airflow), Workload Identity for GKE, VPC Service Controls, Data Catalog, Dataplex for data governance, BigQuery cost optimization (slot reservations vs on-demand), Cloud Logging/Monitoring

ivanshamaev Skill

GitOps deployment review — ArgoCD Application/AppProject CRDs, app-of-apps pattern, sync policies (automated+self-heal+prune), drift detection and remediation, FluxCD HelmRelease/Kustomization, git repository structure for GitOps (app code vs config repo separation), image update automation, progressive delivery (Argo Rollouts canary/blue-green), GitOps for data platform components (Airflow/Kafka/Spark)

ivanshamaev Skill

MCP (Model Context Protocol) server development — FastMCP/Python SDK, tools/resources/prompts primitives, STDIO and Streamable HTTP transports, Claude Desktop/Claude Code client config, MCP Inspector testing, security best practices (input validation/OAuth2/confused deputy prevention), production Docker/Kubernetes deployment, agentic data platform integration

ivanshamaev Skill

Apache Flink — streaming architecture (JobManager/TaskManager/slots/parallelism), PyFlink Table API and SQL (Kafka/filesystem DDL, windowing TUMBLE/HOP/SESSION, aggregations), DataStream API (KeyedStream, stateful operations ValueState/MapState, process functions), event-time vs processing-time, watermark strategies, checkpointing (exactly-once, RocksDB state backend), Kafka source/sink (delivery guarantees), backpressure, savepoints, deployment (standalone/YARN/Kubernetes)

ivanshamaev Skill

Use when designing or implementing Data Vault 2.0 data models — Hubs, Links, Satellites, Reference Tables, Same-As Links, Multi-Active Satellites, Effectivity Satellites, Point-in-Time tables, Bridge tables, Business Vault patterns, hash key generation, DDL for all entity types, insert-only DML load patterns, pipeline architecture, and constructing Information Marts from the vault.

ivanshamaev Skill

Use when designing, building, or debugging a RAG (Retrieval-Augmented Generation) data pipeline — document ingestion, chunking strategies (fixed/recursive/semantic/structure-aware), embedding models (OpenAI/Cohere/sentence-transformers), vector stores (pgvector/Chroma/Qdrant/Weaviate), incremental refresh, hybrid retrieval (dense + BM25 + RRF), re-ranking with cross-encoders, metadata filtering, and production monitoring.

taipt1504 Bundle

Unified testing workflow — TDD (RED/GREEN/REFACTOR), blackbox integration testing with F8A Summer Test (JSON test cases, WireMock, Testcontainers), and 7-phase verification pipeline. Use when writing unit or integration tests, generating test scaffolds, configuring JaCoCo coverage thresholds, using StepVerifier for reactive tests, MockMvc/WebTestClient for API tests, or setting up Testcontainers. Includes scripts/generate-test-scaffold.sh.

ivanshamaev Skill

Grafana dashboard review — panel types (timeseries/stat/gauge/table/heatmap), variable templating (datasource/label_values/query), dashboard linking and drilldown, data platform dashboards (pipeline overview/DAG health/Kafka throughput/Spark performance), alerting from panels, annotation markers for deployments, dashboard-as-code (Grafonnet/Terraform grafana provider), performance optimization (query caching/recording rules), dark/light theme standards

ivanshamaev Skill

Terraform cost estimation — Infracost breakdown and diff commands (monthly cost per resource), PR cost comments (before/after change delta), budget thresholds (block PRs over cost limit), usage files for variable quantities (S3 GB/requests), AWS pricing data sources, cost tagging strategy, OpenTofu/Terraform cost attribution, FinOps tagging policy enforcement, multi-environment cost comparison (dev vs prod)

ivanshamaev Skill

Airflow CDC orchestration for StarRocks — watermark-based incremental sync, Routine Load health check DAG, Flink job submission via REST API, offset lag alerting, dead letter queue reprocessing, multi-table incremental sync with dependency graph, schema change detection and pipeline pause/resume

ivanshamaev Skill

dbt + StarRocks production readiness — slim CI with state:modified+, manifest.json artifact storage and --defer for dev, dbt Cloud vs dbt Core deployment, RBAC for dbt users in StarRocks, secrets management for profiles.yml, dbt run/test exit codes in CI, breaking change detection, scheduled dbt runs via Airflow/dbt Cloud, dbt docs generation and hosting, model versioning

ivanshamaev Skill

Kubernetes cluster health assessment — node status and pressure conditions (disk/memory/PID), pod failure diagnosis (CrashLoopBackOff/OOMKilled/Pending/Evicted), control plane health (API server/etcd/scheduler/controller-manager), resource quota utilization, scheduling failures (taints/affinity/insufficient resources), kubectl diagnostic commands, node eviction policies, kubelet troubleshooting

ivanshamaev Skill

Kubernetes cost optimization — idle workload detection (zero-replica deployments, always-pending jobs), overprovisioning analysis (request vs actual CPU/memory), namespace-level cost attribution, node bin-packing via requests tuning, spot instance strategy, LimitRange defaults, right-sizing with VPA recommendations, Goldilocks tool, resource efficiency metrics, namespace resource quotas for chargeback

ivanshamaev Skill

Kubernetes security audit — RBAC review (ClusterRoleBindings/RoleBindings/ServiceAccount permissions), Pod Security Standards (privileged/hostPID/hostNetwork/runAsRoot), NetworkPolicy enforcement, Secret management (encryption at rest, external-secrets), image vulnerability scanning (Trivy/Grype), admission controllers (OPA Gatekeeper/Kyverno), audit logging, CIS Kubernetes Benchmark, supply chain security (SBOM/image signing)

ivanshamaev Skill

Kubernetes storage review — PVC lifecycle (provisioning/binding/reclaim), StorageClass selection (SSD/HDD/NVMe), dynamic vs static provisioning, StatefulSet volume management, volume performance tuning (ReadWriteMany vs ReadWriteOnce), CSI drivers (AWS EBS/GCP PD/Ceph/Longhorn), PVC resize, snapshot and backup, storage class access modes, orphaned PVs detection, shared storage patterns (NFS/CephFS)

ivanshamaev Skill

Secrets management review — HashiCorp Vault (KV v2/dynamic credentials/lease renewal/audit log), External Secrets Operator (Vault/AWS SSM/GCP Secret Manager → K8s secrets), secret rotation strategies (database dynamic credentials/TLS cert-manager), detecting secrets in code (gitleaks/truffleHog/Semgrep), Airflow connections from Vault backend, dbt profiles without hardcoded credentials, CI/CD secrets (GitHub OIDC/GitLab CI variables), no plaintext secrets in logs/configs

ivanshamaev Skill

Disaster recovery review for data platforms — RTO/RPO definitions, DR strategy selection (cold/warm/hot standby), Airflow metadata DB backup and restore (pg_dump/WAL), Kafka topic replication (MirrorMaker2), data lake replication (S3 CRR/GCS Transfer), Kubernetes cluster backup (Velero), runbook for DR failover, DR testing (game day exercises), backup validation, cross-region Terraform, recovery time measurement

ivanshamaev Skill

GitHub Actions optimization for DataOps — path filters to skip unchanged components, concurrency groups (cancel-in-progress), job parallelization, caching strategies (pip/npm/Maven/Docker layers), self-hosted runners for heavy workloads, matrix strategy for multi-environment tests, reusable workflows to reduce duplication, OIDC for cloud credentials (no static keys), artifact retention, workflow timing analysis