databricks-solutions
- 266 skills
- 0 followers
- 6 repo stars
- 6 days ago last updated
- ▌ Gold Layer Design · databricks-solutions bundleEnd-to-end orchestrator for designing complete Gold layer schemas with ERDs, YAML files, lineage tracking, and comprehensive business documentation. Guides users through dimensional modeling, ERD creation (master/domain/summary based on table count), YAML schema generation, column-level lineage documentation, business onboarding guide creation, source table mapping, and design validation. Orchestrates design-workers (05-erd-diagrams, 06-table-documentation, 01-grain-definition, 07-design-validation, 02-dimension-patterns, 03-fact-table-patterns, 04-conformed-dimensions, 08-industry-alignment). Use when designing a Gold layer from scratch, creating dimensional models, documenting business processes, aligning to Databricks Industry Vibe Data Models, or preparing for Gold layer implementation.
- ▌ Semantic Layer Setup · databricks-solutions bundleEnd-to-end orchestrator for building the Databricks semantic layer including Metric Views, Table-Valued Functions (TVFs), and Genie Spaces. Guides users through metric view creation, TVF development, Genie Space setup, and API-driven deployment. Orchestrates mandatory dependencies on semantic-layer skills (metric-views-patterns, databricks-table-valued-functions, genie-space-patterns, genie-space-export-import-api) and common skills (databricks-asset-bundles, databricks-expert-agent, databricks-python-imports). Use when building the semantic layer end-to-end, creating Metric Views and TVFs for Genie, or setting up Genie Spaces.
- ▌ Databricks Autonomous Operations · databricks-solutions bundleEnd-to-end autonomous deployment and operations skill. Deploys Databricks Asset Bundles, runs jobs and pipelines, polls for completion, diagnoses failures, applies fixes, redeploys, and verifies — all without human intervention. Also serves as SDK/CLI/REST API reference. Use when deploying bundles, running jobs, monitoring pipelines, troubleshooting ANY failure, or operating in a self-healing deploy-fix-redeploy cycle. Triggers on "deploy", "bundle deploy", "run job", "run pipeline", "make it work", "job failed", "troubleshoot", "fix and redeploy".
- ▌ Metric Views Patterns · databricks-solutions bundleStandard patterns for creating Databricks Metric Views with semantic metadata for Genie and AI/BI. Use when creating metric views, troubleshooting metric view creation errors, validating schema references before deployment, implementing joins (including snowflake schema patterns), or optimizing metric views for Genie natural language queries.
- ▌ Vibecoding State · databricks-solutions bundleOwns the end-to-end state, gating, pathway-applicability, and retrospective contract for the `Instructions.md` / `example/<use_case>/WALKTHROUGH.md` vibecoding workshops. Every workshop prompt invokes this skill at the top (`enter`) and bottom (`exit`) so the state file is read, gates are enforced, non-applicable steps are skipped, and the per-step log is appended — without each prompt repeating the contract as prose. Also owns the `bootstrap` initializer, the bootstrap → canonical `migrate_canonical` move, and the async `retrospective.per_prompt` / `retrospective.rollup` passes. Invoke BEFORE acting on any workshop prompt.
- ▌ Genie Code Environment · databricks-solutions bundleSession-start behavioral manifest for running this workshop inside Databricks Genie Code. Read FIRST when the client is Genie Code (the coding agent embedded in the Databricks workspace) so you begin knowing how it behaves — surface/page tool-scoping, the three execution paths (runDatabricksCli → SDK → native tools) and the "blocked ≠ impossible, try the next path" discipline, the runDatabricksCli allow-list tiers, bundle-deploy reality (--target dev mandatory, CWD pinned to the page's bundle root, FUSE create-then-validate gap), AppKit/Node reality (apps init --output-dir, no local npm but server-side build, SDK app deploy), agent-skills git-clone install, the Genie-Space deploy tiers, and how to verify a deployed app (3-hop OAuth session). Detection of WHICH client is active lives in vibecoding-state; this skill explains HOW the Genie Code client behaves. Not needed for the IDE+CLI client.
- ▌ Databricks Asset Bundles · databricks-solutions bundleStandard patterns for Databricks Asset Bundles configuration files for serverless jobs, DLT pipelines, dashboards, alerts, apps, and workflows. Use when creating, configuring, or deploying DABs for infrastructure-as-code deployments. Covers mandatory serverless environment configuration, hierarchical job architecture (atomic/composite/orchestrator), DLT pipeline patterns, dashboard resources with dataset_catalog/dataset_schema, SQL Alerts v2 API schema, Apps lifecycle, Python notebook parameter passing (dbutils.widgets.get vs argparse), deployment error prevention, and pre-deployment validation.
- ▌ 02 Appkit Build · databricks-solutions bundleBuild full-stack UI and backend features on a Databricks AppKit project from a PRD or feature spec. Covers SQL query design, type generation, React frontend with AppKit UI components, backend plugin wiring, and distinctive visual design. Use when asked to implement a UI, build features from a PRD, create pages or dashboards, add components, develop the frontend/backend of an existing AppKit app, build with mock data, or use static data for visual prototyping. Triggers on "build UI", "implement PRD", "create dashboard", "add page", "build features", "implement design", "create components", "build app from PRD", "develop frontend", "mock data", "static data", "two-phase data".
- ▌ 03 Appkit Deploy · databricks-solutions bundleDeploy a Databricks AppKit application to Databricks Apps. Covers config validation, build verification, deployment, UI verification, error diagnosis with fix loop, and workspace app limit handling. Use when asked to deploy an AppKit app, push to production, ship the app, or troubleshoot a failed deploy. Triggers on "deploy app", "push to databricks", "ship app", "deploy appkit", "databricks apps deploy", "fix deploy error", "app won't start".
- ▌ 05 Appkit Lakebase Wiring · databricks-solutions bundleWire a Lakebase PostgreSQL backend into an existing AppKit project. Covers database schema design from a PRD, idempotent DDL, Express API routes with mock fallback, React data hooks, and local testing. PRD-independent patterns that apply to any AppKit + Lakebase app. Use after registering the Lakebase plugin via 04-appkit-plugin-add. Triggers on "wire lakebase", "lakebase backend", "CRUD API", "lakebase tables", "DDL", "database schema design", "useLakebaseData", "mock fallback", "ConnectionStatus", "replace mock data", "connect frontend to backend", "API-backed data", "replace static data with database".
- ▌ 00 Uc Resources Foundation · databricks-solutionsUse when any agent or downstream skill needs Unity Catalog schemas and managed volumes (knowledge_sources, agent_outputs, memory tables, benchmark tables, etc.) provisioned idempotently. Foundation Step 0 — runs before MLflow tracing setup and before any track-specific skill. Owned by no track. Reads the resolved spec to discover which volumes a use case needs and creates them alongside the canonical agent + ops UC schemas.
- ▌ 07 Design Validation · databricks-solutions bundleCross-validation of Gold layer design artifacts during the design phase. Use when validating that YAML schemas, ERDs, lineage CSVs, and PK/FK references are internally consistent before handing off to implementation. Catches design-time inconsistencies (e.g., column in ERD but not in YAML, FK referencing non-existent table) that would otherwise surface as runtime bugs.
- ▌ Databricks Aibi Dashboards · databricks-solutions bundleProduction-grade patterns for Databricks AI/BI (Lakeview) dashboards. Prevents visualization errors, deployment failures, and maintenance issues through widget-query alignment, number formatting, parameter configuration, monitoring table patterns, chart scale properties, and automated deployment workflows. Includes pivot tables with hierarchy drill-down and ratio metrics, point/choropleth maps, sankey diagrams, waterfall and histogram charts, cross-filtering and drill-through patterns, filter defaultSelection/disallowAll configuration, disaggregated vs aggregated query modes, and complete JSON templates for all widget types.
- ▌ Genie Space Export Import API · databricks-solutions bundleComprehensive patterns for Databricks Genie Space Export/Import API - JSON schema, serialization format, and programmatic deployment. Use when programmatically creating, exporting, or importing Genie Spaces via REST API, troubleshooting API deployment errors, or implementing CI/CD for Genie Spaces. Includes complete GenieSpaceExport schema, API endpoints (List, Get, Create, Update, Delete), JSON format requirements, ID generation, variable substitution, inventory-driven generation patterns, and production deployment checklists.
- ▌ Lakebase Cdf · databricks-solutionsLakebase Change Data Feed (CDF) — stream INSERT/UPDATE/DELETE from Lakebase Postgres via the WAL (wal2delta) into Unity Catalog lb_<table>_history Delta tables. Use for the sync_from_lakebase step when enable_lakebase_cdf is true: prerequisites (Postgres 17 autoscaling, non-default-storage catalog, REPLICA IDENTITY FULL), schema-level enablement, verifying STREAMING in wal2delta.tables, consuming _pg_* columns, and disabling.
- ▌
- ▌ Databricks Aibi Custom Viz Vega · databricks-solutions bundleBuild high-fidelity custom charts inside Databricks AI/BI (Lakeview) dashboards using Vega-Lite custom visualizations (Public Preview). Covers the documented contract — the `databricks_query` data source, Fields-section Name references, theme-aware signals (`colors`, `mode`, `dashboardTheme`), and cross-filtering via the reserved `databricks_mark_selection` parameter. Use when the built-in widget types in 02-databricks-aibi-dashboards cannot express the chart you need, but you still want a documented, per-widget, cross-filter-capable visualization. For full-page bespoke React layouts (composite cards, KPI tiles, in-cell sparklines), see 06-databricks-aibi-custom-pages-react.
- ▌ Databricks Aibi Custom Pages React · databricks-solutions bundleBuild full-page, high-fidelity React "custom pages" inside a Databricks AI/BI (Lakeview) dashboard — the mechanism behind pixel-perfect, custom-app-like dashboards (composite KPI cards, in-cell sparklines, custom filter chips, status pills, email-preview surfaces). A custom page is a single widget whose `multilineTextboxSpec.lines` carry a `<!-- @custom-page -->` marker followed by a JSON envelope `{code, manifest, datasetMap}`, where `code` is CommonJS React that reads dashboard datasets through the injected `viz.useCustomPageQuery` hook. Use ONLY when neither the built-in widgets (02-databricks-aibi-dashboards) nor Vega-Lite custom viz (05-databricks-aibi-custom-viz-vega) can express the layout. WARNING — this mechanism is NOT in official Databricks docs; treat as high-volatility and verify in your workspace before relying on it.
- ▌ Databricks Unity Catalog 3 · databricks-solutionsManage Unity Catalog resources including catalogs, schemas, and tables. Handles discovery, creation, updates, and deletions with proper naming conventions and governance. Use when exploring catalogs, creating schemas, managing tables, or setting up data governance.
- ▌ Databricks Apps 2 · databricks-solutions bundleBuild apps on Databricks Apps platform. Use when asked to create dashboards, data apps, analytics tools, or visualizations. Evaluates data access patterns (analytics vs Lakebase synced tables) before scaffolding. Invoke BEFORE starting implementation.
- ▌ Databricks Lakebase 2 · databricks-solutions bundleDatabricks Lakebase Postgres: projects, scaling, connectivity, Lakebase synced tables, and Data API. Use when asked about Lakebase databases, OLTP storage, or connecting apps to Postgres on Databricks.
- ▌ Databricks Model Serving 3 · databricks-solutions bundleManage Databricks Model Serving endpoints via CLI. Use when asked to create, configure, query, or manage model serving endpoints for LLM inference, custom models, or external models.
- ▌ Add Tools 2 · databricks-solutions bundleAdd tools to your agent and grant required permissions in databricks.yml. Use when: (1) Adding MCP servers, Genie spaces, vector search, or UC functions to agent, (2) Permission errors at runtime, (3) User says 'add tool', 'connect to', 'grant permission', (4) Configuring databricks.yml resources.
- ▌ Run Locally 2 · databricks-solutionsRun and test the agent locally. Use when: (1) User says 'run locally', 'start server', 'test agent', or 'localhost', (2) Need curl commands to test API, (3) Troubleshooting local development issues, (4) Configuring server options like port or hot-reload.
- ▌ Create Tools 2 · databricks-solutions bundleCreate Databricks resources that agents connect to as tools. Use when: (1) User needs to create a Genie space, vector search index, UC function, or UC connection, (2) User says 'create tool', 'set up genie', 'create vector search', 'register MCP server', (3) Before add-tools when the resource doesn't exist yet, (4) User asks 'what do I need to create before adding this tool'.
- ▌ Modify Agent 2 · databricks-solutionsModify agent code, add tools, or change configuration. Use when: (1) User says 'modify agent', 'add tool', 'change model', or 'edit agent.py', (2) Adding MCP servers to agent, (3) Changing agent instructions, (4) Understanding SDK patterns.
- ▌ Discover Tools 2 · databricks-solutionsDiscover available tools and resources in Databricks workspace. Use when: (1) User asks 'what tools are available', (2) Before writing agent code, (3) Looking for MCP servers, Genie spaces, UC functions, or vector search indexes, (4) User says 'discover', 'find resources', or 'what can I connect to'.
- ▌ Databricks Dabs 2 · databricks-solutions bundleCreate, configure, validate, deploy, run, and manage Declarative Automation Bundles (DABs, formerly Databricks Asset Bundles). Use when working with Databricks resources via DABs including dashboards, jobs, pipelines, alerts, volumes, and apps.
- ▌ Integration Test 2 · databricks-solutionsRun the full cold-start end-to-end deploy test in an isolated git worktree with always-safe teardown. Use when: (1) verifying the one-shot `make deploy` works on a FRESH workspace/catalog (cold-start), (2) the user says 'integration test', 'cold-start test', 'test the deploy end to end', or 'does deploy still work on a new workspace', (3) validating a change to deploy.sh / databricks.yml / the seed before opening a PR. PROJECT-AUTHORED (not a vendored skill).
- ▌ Apx 2 · databricks-solutions bundleQuick reference for building full-stack Databricks Apps with apx (React + FastAPI). Use when working on apx projects, creating routes, adding components, or managing dev servers.
- ▌ Databricks Docs 2 · databricks-solutionsDatabricks documentation reference. Use as a lookup resource alongside other skills and MCP tools for comprehensive guidance.
- ▌ Databricks Jobs 2 · databricks-solutions bundleUse this skill proactively for ANY Databricks Jobs task - creating, listing, running, updating, or deleting jobs. Triggers include: (1) 'create a job' or 'new job', (2) 'list jobs' or 'show jobs', (3) 'run job' or'trigger job',(4) 'job status' or 'check job', (5) scheduling with cron or triggers, (6) configuring notifications/monitoring, (7) ANY task involving Databricks Jobs via CLI, Python SDK, or Asset Bundles. ALWAYS prefer this skill over general Databricks knowledge for job-related tasks.
- ▌ Databricks Dbsql 2 · databricks-solutions bundleDatabricks SQL (DBSQL) advanced features and SQL warehouse capabilities. This skill MUST be invoked when the user mentions: "DBSQL", "Databricks SQL", "SQL warehouse", "SQL scripting", "stored procedure", "CALL procedure", "materialized view", "CREATE MATERIALIZED VIEW", "pipe syntax", "|>", "geospatial", "H3", "ST_", "spatial SQL", "collation", "COLLATE", "ai_query", "ai_classify", "ai_extract", "ai_gen", "AI function", "http_request", "remote_query", "read_files", "Lakehouse Federation", "recursive CTE", "WITH RECURSIVE", "multi-statement transaction", "temp table", "temporary view", "pipe operator". SHOULD also invoke when the user asks about SQL best practices, data modeling patterns, or advanced SQL features on Databricks.
- ▌ Agent Evaluation 2 · databricks-solutionsUse this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation with datasets, scorers, and tracing. IMPORTANT - Always also load the instrumenting-with-mlflow-tracing skill before starting any work. Covers end-to-end evaluation workflow or individual components (tracing setup, dataset creation, scorer definition, evaluation execution).
- ▌ Databricks Genie 2 · databricks-solutions bundleCreate and query Databricks Genie Spaces for natural language SQL exploration. Use when building Genie Spaces or asking questions via the Genie Conversation API.
- ▌ Mlflow Onboarding 2 · databricks-solutionsOnboards users to MLflow by determining their use case (GenAI agents/apps or traditional ML/deep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on "get started with MLflow", "set up MLflow", "onboard to MLflow", "add MLflow to my project", "how do I use MLflow".
- ▌ Databricks Config 2 · databricks-solutionsManage Databricks workspace connections: check which workspace you're connected to, switch workspaces, list available workspaces, or authenticate to a new workspace.
- ▌ Databricks App Apx 2 · databricks-solutions bundleBuild full-stack Databricks applications using APX framework (FastAPI + React).
- ▌ Databricks Iceberg 2 · databricks-solutions bundleApache Iceberg tables on Databricks — Managed Iceberg tables, External Iceberg Reads (fka Uniform), Compatibility Mode, Iceberg REST Catalog (IRC), Iceberg v3, Snowflake interop, PyIceberg, OSS Spark, external engine access and credential vending. Use when creating Iceberg tables, enabling External Iceberg Reads (uniform) on Delta tables (including Streaming Tables and Materialized Views via compatibility mode), configuring external engines to read Databricks tables via Unity Catalog IRC, integrating with Snowflake catalog to read Foreign Iceberg tables
- ▌ Databricks Parsing 2 · databricks-solutionsParse documents (PDF, DOCX, PPTX, images) using ai_parse_document, or build custom RAG pipelines. Use when the user asks to parse documents or build a custom RAG.
- ▌ Databricks Python Sdk 2 · databricks-solutions bundleDatabricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.
- ▌ Databricks App Python 2 · databricks-solutions bundleBuilds Python-based Databricks applications using Dash, Streamlit, Gradio, Flask, FastAPI, or Reflex. Handles OAuth authorization (app and user auth), app resources, SQL warehouse and Lakebase connectivity, model serving integration, and deployment. Use when building Python web apps, dashboards, ML demos, or REST APIs for Databricks, or when the user mentions Streamlit, Dash, Gradio, Flask, FastAPI, Reflex, or Databricks app.
- ▌ Databricks Metric Views 2 · databricks-solutions bundleUnity Catalog metric views: define, create, query, and manage governed business metrics in YAML. Use when building standardized KPIs, revenue metrics, order analytics, or any reusable business metrics that need consistent definitions across teams and tools.
- ▌ Databricks Agent Bricks 2 · databricks-solutions bundleCreate and manage Databricks Agent Bricks: Knowledge Assistants (KA) for document Q&A, Genie Spaces for SQL exploration, and Supervisor Agents (MAS) for multi-agent orchestration. Use when building conversational AI applications on Databricks.
- ▌ Databricks Asset Bundles 2 · databricks-solutions bundleCreate and configure Databricks Asset Bundles (DABs) with best practices for multi-environment deployments. Use when working with: (1) Creating new DAB projects, (2) Adding resources (dashboards, pipelines, jobs, alerts), (3) Configuring multi-environment deployments, (4) Setting up permissions, (5) Deploying or running bundle resources
- ▌ Spark Python Data Source 2 · databricks-solutions bundleUse when building custom Spark data source connectors for external systems (databases, APIs, message queues), implementing batch/streaming readers/writers, or creating data source plugins for systems without native Spark support. Triggers - "build Spark data source", "create Spark connector", "implement Spark reader/writer", "connect Spark to [system]", "streaming data source"
- ▌ Databricks Model Serving 2 · databricks-solutions bundleDeploy and query Databricks Model Serving endpoints. Use when (1) deploying MLflow models or AI agents to endpoints, (2) creating ChatAgent/ResponsesAgent agents, (3) integrating UC Functions or Vector Search tools, (4) querying deployed endpoints, (5) checking endpoint status. Covers classical ML models, custom pyfunc, and GenAI agents.
- ▌ Databricks Unity Catalog 2 · databricks-solutions bundleUnity Catalog system tables and volumes. Use when querying system tables (audit, lineage, billing) or working with volume file operations (upload, download, list files in /Volumes/).
- ▌ Databricks Vector Search 2 · databricks-solutions bundlePatterns for Databricks Vector Search: create endpoints and indexes, query with filters, manage embeddings. Use when building RAG applications, semantic search, or similarity matching. Covers both storage-optimized and standard endpoints.
- ▌ Databricks Zerobus Ingest 2 · databricks-solutions bundleBuild Zerobus Ingest clients for near real-time data ingestion into Databricks Delta tables via gRPC. Use when creating producers that write directly to Unity Catalog tables without a message bus, working with the Zerobus Ingest SDK in Python/Java/Go/TypeScript/Rust, generating Protobuf schemas from UC tables, or implementing stream-based ingestion with ACK handling and retry logic.
- ▌ Databricks Aibi Dashboards 2 · databricks-solutionsCreate Databricks AI/BI dashboards. CRITICAL: You MUST test ALL SQL queries via execute_sql BEFORE deploying. Follow guidelines strictly.
- ▌ Databricks Mlflow Evaluation 2 · databricks-solutions bundleMLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement.
- ▌ Databricks Lakebase Autoscale 2 · databricks-solutions bundlePatterns and best practices for using Lakebase Autoscaling (next-gen managed PostgreSQL) with autoscaling, branching, scale-to-zero, and instant restore.
- ▌ Databricks Synthetic Data Gen 2 · databricks-solutions bundleGenerate realistic synthetic data using Spark + Faker (strongly recommended). Supports serverless execution, multiple output formats (Parquet/JSON/CSV/Delta), and scales from thousands to millions of rows. For small datasets (<10K rows), can optionally generate locally and upload to volumes. Use when user mentions 'synthetic data', 'test data', 'generate data', 'demo dataset', 'Faker', or 'sample data'.
- ▌ Databricks Lakebase Provisioned 2 · databricks-solutions bundlePatterns and best practices for using Lakebase Provisioned (Databricks managed PostgreSQL) for OLTP workloads.
- ▌ Instrumenting With Mlflow Tracing 2 · databricks-solutions bundleInstruments Python and TypeScript code with MLflow Tracing for observability. Must be loaded when setting up tracing as part of any workflow including agent evaluation. Triggers on adding tracing, instrumenting agents/LLM apps, getting started with MLflow tracing, tracing specific frameworks (LangGraph, LangChain, OpenAI, DSPy, CrewAI, AutoGen), or when another skill references tracing setup. Examples - "How do I add tracing?", "Instrument my agent", "Trace my LangChain app", "Set up tracing for evaluation"
- ▌ Databricks Spark Structured Streaming 2 · databricks-solutions bundleComprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, implementing real-time data processing, handling stateful operations, or optimizing streaming performance.
- ▌ Databricks Spark Declarative Pipelines 2 · databricks-solutions bundleCreates, configures, and updates Databricks Lakeflow Spark Declarative Pipelines (SDP/LDP) using serverless compute. Handles streaming tables, materialized views, CDC, SCD Type 2, and Auto Loader ingestion patterns. Use when building data pipelines, working with Delta Live Tables, ingesting streaming data, implementing change data capture, or when the user mentions SDP, LDP, DLT, Lakeflow pipelines, streaming tables, or bronze/silver/gold medallion architectures.
- ▌ Databricks Unstructured PDF Generation 2 · databricks-solutionsGenerate synthetic PDF documents for RAG and unstructured data use cases. Use when creating test PDFs, demo documents, or evaluation datasets for retrieval systems.
- ▌ Databricks Docs · databricks-solutionsDatabricks documentation reference. Use as a lookup resource alongside other skills and MCP tools for comprehensive guidance.
- ▌ Databricks Jobs · databricks-solutions bundleUse this skill proactively for ANY Databricks Jobs task - creating, listing, running, updating, or deleting jobs. Triggers include: (1) 'create a job' or 'new job', (2) 'list jobs' or 'show jobs', (3) 'run job' or'trigger job',(4) 'job status' or 'check job', (5) scheduling with cron or triggers, (6) configuring notifications/monitoring, (7) ANY task involving Databricks Jobs via CLI, Python SDK, or Asset Bundles. ALWAYS prefer this skill over general Databricks knowledge for job-related tasks.
- ▌ Databricks Dbsql · databricks-solutions bundleDatabricks SQL (DBSQL) advanced features and SQL warehouse capabilities. This skill MUST be invoked when the user mentions: "DBSQL", "Databricks SQL", "SQL warehouse", "SQL scripting", "stored procedure", "CALL procedure", "materialized view", "CREATE MATERIALIZED VIEW", "pipe syntax", "|>", "geospatial", "H3", "ST_", "spatial SQL", "collation", "COLLATE", "ai_query", "ai_classify", "ai_extract", "ai_gen", "AI function", "http_request", "remote_query", "read_files", "Lakehouse Federation", "recursive CTE", "WITH RECURSIVE", "multi-statement transaction", "temp table", "temporary view", "pipe operator". SHOULD also invoke when the user asks about SQL best practices, data modeling patterns, or advanced SQL features on Databricks.
- ▌ Mlflow Onboarding · databricks-solutionsOnboards users to MLflow by determining their use case (GenAI agents/apps or traditional ML/deep learning) and guiding them through relevant quickstart tutorials and initial integration. If an experiment ID is available, it should be supplied as input to help determine the use case. Use when the user asks to get started with MLflow, set up tracking, add observability, or integrate MLflow into their project. Triggers on "get started with MLflow", "set up MLflow", "onboard to MLflow", "add MLflow to my project", "how do I use MLflow".
- ▌ Databricks App Apx · databricks-solutions bundleBuild full-stack Databricks applications using APX framework (FastAPI + React).
- ▌ Databricks Iceberg · databricks-solutions bundleApache Iceberg tables on Databricks — Managed Iceberg tables, External Iceberg Reads (fka Uniform), Compatibility Mode, Iceberg REST Catalog (IRC), Iceberg v3, Snowflake interop, PyIceberg, OSS Spark, external engine access and credential vending. Use when creating Iceberg tables, enabling External Iceberg Reads (uniform) on Delta tables (including Streaming Tables and Materialized Views via compatibility mode), configuring external engines to read Databricks tables via Unity Catalog IRC, integrating with Snowflake catalog to read Foreign Iceberg tables
- ▌ Databricks Metric Views · databricks-solutions bundleUnity Catalog metric views: define, create, query, and manage governed business metrics in YAML. Use when building standardized KPIs, revenue metrics, order analytics, or any reusable business metrics that need consistent definitions across teams and tools.
- ▌ Spark Python Data Source · databricks-solutions bundleUse when building custom Spark data source connectors for external systems (databases, APIs, message queues), implementing batch/streaming readers/writers, or creating data source plugins for systems without native Spark support. Triggers - "build Spark data source", "create Spark connector", "implement Spark reader/writer", "connect Spark to [system]", "streaming data source"
- ▌ Databricks Mlflow Evaluation · databricks-solutions bundleMLflow 3 GenAI agent evaluation. Use when writing mlflow.genai.evaluate() code, creating @scorer functions, using built-in scorers (Guidelines, Correctness, Safety, RetrievalGroundedness), building eval datasets from traces, setting up trace ingestion and production monitoring, aligning judges with MemAlign from domain expert feedback, or running optimize_prompts() with GEPA for automated prompt improvement.
- ▌ Databricks Lakebase Autoscale · databricks-solutions bundlePatterns and best practices for using Lakebase Autoscaling (next-gen managed PostgreSQL) with autoscaling, branching, scale-to-zero, and instant restore.
- ▌ Databricks Lakebase Provisioned · databricks-solutions bundlePatterns and best practices for using Lakebase Provisioned (Databricks managed PostgreSQL) for OLTP workloads.
- ▌ Databricks Spark Structured Streaming · databricks-solutions bundleComprehensive guide to Spark Structured Streaming for production workloads. Use when building streaming pipelines, implementing real-time data processing, handling stateful operations, or optimizing streaming performance.
- ▌ Databricks Config · databricks-solutionsManage Databricks workspace connections: check which workspace you're connected to, switch workspaces, list available workspaces, or authenticate to a new workspace.
- ▌ Deploy 2 · databricks-solutionsDeploy agent to Databricks Apps using DAB (Databricks Asset Bundles). Use when: (1) User says 'deploy', 'push to databricks', or 'bundle deploy', (2) 'App already exists' error occurs, (3) Need to bind/unbind existing apps, (4) Debugging deployed apps, (5) Querying deployed app endpoints.
- ▌ Quickstart 2 · databricks-solutionsSet up Databricks agent development environment. Use when: (1) First time setup, (2) Configuring Databricks authentication, (3) User says 'quickstart', 'set up', 'authenticate', or 'configure databricks', (4) No .env file exists.
- ▌ Agent Memory 2 · databricks-solutions bundleAdd memory capabilities to your agent. Use when: (1) User asks about 'memory', 'state', 'remember', 'conversation history', (2) Want to persist conversations or user preferences, (3) Adding checkpointing or long-term storage.
- ▌ Load Testing 2 · databricks-solutions bundleLoad test a Databricks App to find its maximum QPS. Use when: (1) User says 'load test', 'benchmark', 'QPS', 'throughput', or 'performance test', (2) User wants to find how many queries per second their app can handle, (3) User wants to set up load testing scripts for their agent, (4) User wants to view load test results/dashboard.
- ▌ Lakebase Setup 2 · databricks-solutionsConfigure Lakebase for agent memory storage. Use when: (1) Adding memory capabilities to the agent, (2) 'Failed to connect to Lakebase' errors, (3) Permission errors on checkpoint/store tables, (4) User says 'lakebase', 'memory setup', or 'add memory'.
- ▌ Databricks Core 2 · databricks-solutions bundleDatabricks CLI operations: auth, profiles, data exploration, and bundles. Contains up-to-date guidelines for Databricks-related CLI tasks.
- ▌ Sync Architecture Docs 2 · databricks-solutionsKeep this repo's architecture diagrams and prose docs in sync with the code as it evolves. Use this whenever the user asks to update/refresh the architecture diagrams, sync docs with the code, audit docs for staleness, or check that docs/architecture.md still matches the implementation — and proactively after any significant change (a new graph node, tool, Lakebase schema, data-pipeline step, auth change, or frontend swap), since the diagrams and READMEs drift silently. Covers refreshing the four Mermaid diagrams in docs/architecture.md, reconciling the known stale-doc patterns, and verifying with greps + Mermaid validity.
- ▌ Migrate From Model Serving 2 · databricks-solutionsMigrate an MLflow ResponsesAgent from Databricks Model Serving to Databricks Apps. Use when: (1) User wants to migrate from Model Serving to Apps, (2) User has a ResponsesAgent with predict()/predict_stream() methods, (3) User wants to convert to @invoke/@stream decorators.
- ▌ Apx · databricks-solutions bundleQuick reference for building full-stack Databricks Apps with apx (React + FastAPI). Use when working on apx projects, creating routes, adding components, or managing dev servers.
- ▌ Rust · databricks-solutionsStrict set of rules in terms of codebase development, design patterns, and best practices. Use when the user wants to develop a new feature or refactor existing code.
- ▌ Asyncio · databricks-solutionsUse when debugging event loop hangs, task scheduling issues, call_soon vs call_soon_threadsafe confusion, Future callback timing, _enter_task/_leave_task conflicts, GIL contention patterns, per-step vs per-drive task context, uvloop compatibility problems, sniffio/anyio backend detection failures, streaming backpressure deadlocks, or native runtime context issues on the asyncio thread. Also use when verifying asyncio assumptions via quick Python one-liners.
- ▌ Commiter · databricks-solutionsCreate git commits with emoji prefixes and conventional commit messages. Use when the user wants to commit changes, push code, or asks to use "just pm" command.
- ▌ Consort · databricks-solutions bundleConsort, a spec-first, test-driven agent framework for building transactional applications on Lakebase-paired projects: a deterministic orchestrator drives role agents through a spec-first design lane (Spec Driven Development) and a Test Driven Development build lane (RED-GREEN-REFACTOR) that runs against a live copy-on-write branch of a real, governed Lakebase database (Databricks' serverless Postgres/OLTP database, not the Delta Lakehouse), composed with paired-branch primitives (cheap experiments, parent-aware schema diff, real per-branch databases). The controls are code the agent cannot edit: gates fail closed, tests are immutable within a unit of work, and a green result is a real test run against real data. Spec-first means the spec is drafted, reviewed, and frozen at gates before any build cycle runs; the spec, the architecture, and the database all evolve increment over increment. Use when planning a new feature, running design-spec gates, running TDD cycles, comparing parallel experiments, or detect
- ▌ UI UX Design Principles · databricks-solutions bundleExperience-level engineering canon, the UI/UX counterpart to software-design-principles (code-level) and architectural-design-principles (system-level). Usability heuristics, visual hierarchy, accessibility, interaction and feedback, information architecture, design systems and tokens, and testable UI. Imported by the consort UX Designer (authoring design-guide.{md,json} + ia.md and the adherence gate) and by the Driver building UI. Use when: shaping a design guide or information architecture, reviewing a user-facing surface, choosing a UI framework, or making the UI testable.
- ▌ Software Design Principles · databricks-solutions bundleFoundational engineering canon – SOLID, DRY, clean code, layered architecture, cross-cutting concerns, NFRs. Imported by workflow skills (consort, lakebase-scm-workflows). Use when designing a module, reviewing a PR, planning a refactor, mapping cross-cutting concerns to layers, or arguing about API shape.
- ▌ Architectural Design Principles · databricks-solutions bundleSystem-level engineering canon, the architecture counterpart to software-design-principles. Layered architecture + dependency direction, ports and adapters (repository / ORM-as-adapter), the twelve-factor app for cloud-native development, evolutionary architecture with fitness functions, and evolutionary database design (schema that evolves by migration on paired branches). Imported by workflow skills (consort, lakebase-scm-workflows). Use when: shaping a system boundary, deciding what is a backing service, mapping config to the environment, or authoring the fitness functions that keep the architecture honest as it evolves.
- ▌ Databricks Architecture · databricks-solutions bundleCreate or edit a Databricks solution-architecture diagram — a Lucidchart-style platform diagram (sources → Lakeflow/Genie pipeline → lakehouse/lakebase → dashboards, Genie, apps → end user, on a governed platform). Use when someone wants to draw, generate, or edit an architecture diagram, visualize a Databricks solution's components and data flow, or produce an architecture image. Produces a self-contained HTML you render to a PNG and iterate on.
- ▌ Databricks Solution Builder · databricks-solutions bundleGenerate comprehensive specification files for building Databricks assets, demos or end 2 end projects. Use when users want to create a new demo, design a demo story, or need help structuring demo components, create an entire project. This skill creates prompts that another agent will execute to build the actual demo.
- ▌
- ▌ Loyalty Segmentation Adaptation · databricks-solutionsConfigure and run the Harvestly Co. loyalty segmentation demo in the user's Databricks workspace. Use when the user is working in (or has imported) the loyalty-segmentation project AND says any of "run in my workspace", "set this up", "configure for my workspace", "configure for my catalog/schema", "deploy this demo", "make this work in my workspace", OR asks about adapting the demo's structure, data contract, swapping synthetic data for real data, or how to point it at their own customer tables.
- ▌ Spec Audit · databricks-solutionsAudit and improve spec coverage for a given spec. Use when (1) a spec has low or 0% requirement coverage, (2) tests exist but lack @req tags, (3) code behaviors have drifted from the spec's success criteria, (4) you need to identify unspecified behaviors in the codebase. Covers the full audit loop: analyze coverage -> tag existing tests -> identify spec gaps -> propose spec updates.
- ▌ Brainstorming · databricks-solutionsYou MUST use this before any creative work — creating features, building components, adding functionality, or modifying behavior. Starts from existing specs rather than scratch. Use when the user asks to build, add, change, or design anything, even if it seems simple. Covers the full loop: find governing spec -> explore intent -> design within spec constraints -> transition to planning.
- ▌ Writing Plans · databricks-solutionsUse when you have a spec or requirements for a multi-step task, before touching code. Creates spec-linked implementation plans with TDD steps, exact file paths, and spec coverage tracking. Use this after brainstorming, when a user says 'plan this', 'how should we implement', or when you're about to start a multi-file feature. Covers the full loop: spec review -> file mapping -> task decomposition -> TDD steps -> coverage verification.
- ▌ Grill Me To Docs · databricks-solutionsUse when the user wants their firsthand knowledge of building, deploying, or running this project with real customers, users, or developers turned into published documentation — e.g. "interview me for docs", "extract what I know", "help me document this so others can run it without me".
- ▌ Verification Testing · databricks-solutions bundleCode verification and testing for the Human Evaluation Workshop. Use when (1) running tests after code changes, (2) writing new unit tests (pytest/vitest), (3) writing E2E tests with Playwright/TestScenario, (4) debugging test failures, (5) understanding what to mock in E2E tests, (6) verifying a feature implementation. Covers the full test pyramid: unit tests -> integration tests -> E2E tests.
- ▌ Databricks Testing · databricks-solutionsExecute code on Databricks clusters using MCP Command Execution API. Supports stateless quick validation and stateful iterative development. Use when testing Python/SQL code on clusters, debugging pipelines, or validating transformations.
- ▌ Databricks Ml Pipeline · databricks-solutionsEnd-to-end machine learning pipelines on Databricks including data exploration, feature engineering, model training with hyperparameter optimization, MLflow experiment tracking, model registration to Unity Catalog, and deployment as DABs. Use when building ML workflows, training models, or deploying ML pipelines.
- ▌ Databricks Bundle Deploy · databricks-solutionsPackage and deploy Databricks Asset Bundles with proper parameterization, multi-environment support, and serverless compute. Handles project structure, databricks.yml generation, validation, and deployment. Use when packaging tested code for production, deploying pipelines, or managing multi-environment deployments.