Rill Connector

Instructions for developing a connector in Rill

Introduction

Connectors are resources that contain credentials and settings for connecting to external systems. They are typically found at the root of the project's DAG, providing access to data sources and services that power downstream resources like models and metrics views.

Connectors are usually lightweight resources. When reconciled, they validate the connection to the external system but do not move or process data. The main exception is managed OLAP connectors (with managed: true), which trigger database provisioning.

Driver capabilities

Each connector uses a driver that implements one or more capabilities:

• OLAP database: Can power metrics views and dashboards (e.g., duckdb, clickhouse, druid, pinot, starrocks, snowflake, bigquery, databricks as live connectors)
• SQL database: Can run SQL queries as model inputs (e.g., postgres, bigquery, snowflake)
• Information schema: Can list tables and their schemas (e.g., duckdb, bigquery)
• Object store: Can list, read, and write flat files (e.g., s3, gcs)
• Notifier: Can send notifications and alerts (e.g., slack)
• AI: Can generate embeddings or responses (e.g., openai, claude, gemini)

Core Concepts

Naming conventions

Connectors are typically named after their driver (e.g., a file connectors/duckdb.yaml creates a connector named duckdb). Use descriptive names when you have multiple connectors of the same type:

• connectors/s3_data.yaml and connectors/s3_staging.yaml
• connectors/clickhouse_prod.yaml and connectors/clickhouse_dev.yaml

Secrets management

Always store sensitive credentials in .env and reference them using template syntax:

type: connector
driver: s3
aws_access_key_id: "{{ .env.aws_access_key_id }}"
aws_secret_access_key: "{{ .env.aws_secret_access_key }}"

NOTE: Some legacy projects use the deprecated .vars instead of .env.

Managed connectors

OLAP connectors can be provisioned automatically by Rill using managed: true. This is supported for duckdb and clickhouse drivers:

type: connector
driver: duckdb
managed: true

When a managed connector is reconciled, Rill provisions the database infrastructure. The user is billed for the CPU, memory, and disk usage of the provisioned database.

Access modes

Control read/write access using the mode property:

• mode: read: Prevents Rill models from writing to this connector
• mode: readwrite: Allows both reading and writing (default for managed connectors)

Use mode: read when connecting to external databases with pre-existing tables to prevent unintended modifications.

Dev/Prod configuration

Use dev: and prod: blocks for environment-specific settings:

type: connector
driver: clickhouse
mode: readwrite

# Use a local database in development to prevent overwriting data in the production cluster when iterating on changes.
dev:
  managed: true

prod:
  host: "{{ .env.clickhouse_host }}"
  port: 9440
  username: "{{ .env.clickhouse_user }}"
  password: "{{ .env.clickhouse_password }}"
  ssl: true

Driver-Specific Notes

DuckDB

DuckDB is Rill's default embedded OLAP database. Key properties:

• managed: true: Rill provisions and manages the database
• init_sql: SQL to run at startup (install extensions, create secrets, attach databases)

For MotherDuck (cloud DuckDB), use the path property with md: prefix:

type: connector
driver: duckdb
path: "md:my_database"
token: "{{ .env.motherduck_token }}"

ClickHouse

ClickHouse can be user-managed or Rill-managed. Key properties:

• managed: true: Rill provisions and manages an empty Clickhouse cluster. If set, don't set any other connector properties.
• host, port, username, password: Connection credentials
• database: Target database name
• ssl: true: Required for ClickHouse Cloud
• cluster: Cluster name for multi-node Clickhouse clusters
• dsn: Alternative connection string format (format: clickhouse://host:port?username=<username>&...)

Common ports:

• 8443: HTTPS native protocol (ClickHouse Cloud)
• 9440: Secure native protocol
• 9000: Native protocol (non-SSL)

S3

AWS S3 and S3-compatible storage. Key properties:

• aws_access_key_id, aws_secret_access_key: AWS credentials
• region: AWS region
• endpoint: Custom endpoint for S3-compatible services (R2, MinIO, GCS interop)
• path_prefixes: A list of bucket paths that the connector can access, e.g. [s3://my-bucket]; useful for improving bucket introspection

GCS

Google Cloud Storage. Key properties:

• google_application_credentials: Service account JSON (must be a literal JSON string value)
• key_id: HMAC key ID to use instead of a service account JSON; required for direct use with DuckDB and Clickhouse through S3 compatibility
• secret: HMAC secret to use instead of a service account JSON; required for direct use with DuckDB and Clickhouse through S3 compatibility

BigQuery

Google BigQuery. Key properties:

• project_id: GCP project ID
• google_application_credentials: Service account JSON

Snowflake

Snowflake data warehouse. Key properties:

• account, user, privateKey, database, schema, warehouse, role: Connection parameters
• dsn: Connection string to use instead of separate connection parameters

Postgres

PostgreSQL database. Key properties:

• host, port, user, password, dbname: Connection parameters
• sslmode: SSL mode (disable, require, verify-full, etc.)

Druid

Apache Druid. Can be configured via host/port or DSN:

• host, port, username, password, ssl: Direct connection
• dsn: Full connection string

Redshift

Amazon Redshift. Key properties:

• aws_access_key_id, aws_secret_access_key: AWS credentials
• workgroup: Redshift Serverless workgroup name
• region: AWS region
• database: Database name

Athena

Amazon Athena. Key properties:

• aws_access_key_id, aws_secret_access_key: AWS credentials
• workgroup: Redshift Serverless workgroup name
• region: AWS region
• output_location: S3 path in format s3://bucket/path to store temporary query results in (Athena only)

StarRocks

StarRocks live OLAP connector (query in place, no ingestion). Key properties:

• host, port, username, password, database, catalog: Connection parameters
• ssl: true: Enable a secure connection
• dsn: Connection string in place of separate parameters

Databricks

Databricks, as a live OLAP connector or a data source for ingestion. Key properties:

• host, http_path, token: SQL warehouse hostname, HTTP path, and access token
• catalog, schema: Unity Catalog name and schema (optional; default to workspace defaults)
• dsn: Connection string in place of separate parameters

Other drivers

• Slack: Use bot_token for alert notifications
• OpenAI, Claude, or Gemini: Use api_key for AI-powered features; optionally set model to override the default
• HTTPS: Simple connector for public HTTP sources
• Pinot: Use broker_host, controller_host, username, password

Examples

DuckDB: Managed

Explicit:

# connectors/duckdb.yaml
type: connector
driver: duckdb
managed: true

or relying on defaults:

# connectors/duckdb.yaml
type: connector
driver: duckdb

DuckDB: With init_sql for S3 secrets

# connectors/duckdb.yaml
type: connector
driver: duckdb

init_sql: |
  CREATE SECRET IF NOT EXISTS s3 (
    TYPE S3,
    KEY_ID '{{ .env.aws_access_key_id }}',
    SECRET '{{ .env.aws_secret_access_key }}',
    REGION 'us-east-1'
  )

This is now deprecated in favor of creating a dedicated s3.yaml connector file, which Rill will automatically load and create as a secret in DuckDB.

DuckDB: With extensions

# connectors/duckdb.yaml
type: connector
driver: duckdb

init_sql: |
  INSTALL spatial;
  LOAD spatial;

DuckDB: MotherDuck database with existing tables

# connectors/motherduck.yaml
type: connector
driver: duckdb
path: "md:my_database"
token: "{{ .env.motherduck_token }}"
schema_name: main
mode: read

ClickHouse: Cloud with SSL

# connectors/clickhouse.yaml
type: connector
driver: clickhouse
host: "abc123.us-east-1.aws.clickhouse.cloud"
port: 8443
username: "default"
password: "{{ .env.clickhouse_password }}"
database: "default"
ssl: true

ClickHouse: Readwrite with cluster

# connectors/clickhouse.yaml
type: connector
driver: clickhouse
mode: readwrite

host: "{{ .env.clickhouse_host }}"
port: 9440
username: "{{ .env.clickhouse_user }}"
password: "{{ .env.clickhouse_password }}"
database: "default"
cluster: "my_cluster"
ssl: true

ClickHouse: Dev/prod configuration

# connectors/clickhouse.yaml
type: connector
driver: clickhouse
mode: readwrite

dev:
  managed: true

prod:
  host: "{{ .env.clickhouse_host }}"
  port: 9440
  username: "{{ .env.clickhouse_user }}"
  password: "{{ .env.clickhouse_password }}"
  database: "default"
  cluster: "{{ .env.clickhouse_cluster }}"
  ssl: true

S3: Basic with credentials and region

# connectors/s3.yaml
type: connector
driver: s3
aws_access_key_id: "{{ .env.aws_access_key_id }}"
aws_secret_access_key: "{{ .env.aws_secret_access_key }}"
region: us-west-2

S3: Cloudflare R2 (S3-compatible)

# connectors/r2.yaml
type: connector
driver: s3
aws_access_key_id: "{{ .env.r2_access_key_id }}"
aws_secret_access_key: "{{ .env.r2_secret_access_key }}"
endpoint: "https://{{ .env.r2_account_id }}.r2.cloudflarestorage.com"
region: auto

GCS: Minimal (uses default credentials)

# connectors/gcs.yaml
type: connector
driver: gcs

GCS: With explicit credentials

# connectors/gcs.yaml
type: connector
driver: gcs
google_application_credentials: "{{ .env.gcs_service_account_json }}"

BigQuery

# connectors/bigquery.yaml
type: connector
driver: bigquery
project_id: "my-gcp-project"
google_application_credentials: "{{ .env.bigquery_service_account_json }}"

Snowflake: Basic with DSN

# connectors/snowflake.yaml
type: connector
driver: snowflake
dsn: "{{ .env.snowflake_dsn }}"

Postgres

# connectors/postgres.yaml
type: connector
driver: postgres
host: "{{ .env.postgres_host }}"
port: 5432
user: "{{ .env.postgres_user }}"
password: "{{ .env.postgres_password }}"
dbname: "analytics"
sslmode: require

Druid: Host-based

# connectors/druid.yaml
type: connector
driver: druid
host: "{{ .env.druid_host }}"
port: 8888
username: "{{ .env.druid_user }}"
password: "{{ .env.druid_password }}"
ssl: true

Redshift: Serverless

# connectors/redshift.yaml
type: connector
driver: redshift
aws_access_key_id: "{{ .env.aws_access_key_id }}"
aws_secret_access_key: "{{ .env.aws_secret_access_key }}"
workgroup: "my-workgroup"
region: us-east-1
database: "analytics"

StarRocks

# connectors/starrocks.yaml
type: connector
driver: starrocks
host: "{{ .env.starrocks_host }}"
port: 9030
username: "{{ .env.starrocks_user }}"
password: "{{ .env.starrocks_password }}"
database: "analytics"

Databricks

# connectors/databricks.yaml
type: connector
driver: databricks
host: "dbc-xxxxxxxx-xxxx.cloud.databricks.com"
http_path: "/sql/1.0/warehouses/xxxxxxxxxxxxxxxx"
token: "{{ .env.databricks_token }}"
catalog: "main"
schema: "default"

DuckLake

DuckLake lakehouse. Uses the DuckDB driver with an attach: clause pointing at the catalog and data path; queried live (no ingestion):

# connectors/ducklake.yaml
type: connector
driver: duckdb
attach: "'ducklake:metadata.ducklake' (DATA_PATH 's3://my-bucket/ducklake/')"

OpenAI

# connectors/openai.yaml
type: connector
driver: openai
api_key: "{{ .env.openai_api_key }}"

Claude

# connectors/claude.yaml
type: connector
driver: claude
api_key: "{{ .env.claude_api_key }}"

Gemini

# connectors/gemini.yaml
type: connector
driver: gemini
api_key: "{{ .env.gemini_api_key }}"

Slack

# connectors/slack.yaml
type: connector
driver: slack
bot_token: "{{ .env.slack_bot_token }}"

Pinot

# connectors/pinot.yaml
type: connector
driver: pinot
broker_host: "{{ .env.pinot_broker_host }}"
controller_host: "{{ .env.pinot_controller_host }}"
username: "{{ .env.pinot_user }}"
password: "{{ .env.pinot_password }}"
ssl: true

Reference documentation

Here is a full JSON schema for the connector syntax:

allOf:
    - properties:
        type:
            const: connector
            description: Refers to the resource type and must be `connector`
            type: string
      required:
        - type
      title: Properties
      type: object
    - $ref: '#/definitions/common_properties'
    - oneOf:
        - examples:
            - allow_host_access: true
              aws_access_key_id: '{{ .env.AWS_ACCESS_KEY_ID }}'
              aws_access_token: '{{ .env.AWS_ACCESS_TOKEN }}'
              aws_secret_access_key: '{{ .env.AWS_SECRET_ACCESS_KEY }}'
              driver: athena
              external_id: MyExternalID
              output_location: s3://my-bucket/athena-output/
              region: us-east-1
              role_arn: arn:aws:iam::123456789012:role/MyRole
              role_session_name: MySession
              type: connector
              workgroup: primary
          properties:
            allow_host_access:
                description: Allow the Athena client to access host environment configurations such as environment variables or local AWS credential files. Defaults to true, enabling use of credentials and settings from the host environment unless explicitly disabled.
                type: boolean
            aws_access_key_id:
                description: AWS Access Key ID used for authentication. Required when using static credentials directly or as base credentials for assuming a role.
                type: string
            aws_access_token:
                description: AWS session token used with temporary credentials. Required only if the Access Key and Secret Key are part of a temporary session credentials.
                type: string
            aws_secret_access_key:
                description: AWS Secret Access Key paired with the Access Key ID. Required when using static credentials directly or as base credentials for assuming a role.
                type: string
            driver:
                const: athena
                description: Refers to the driver type and must be driver `athena`
                type: string
            external_id:
                description: External ID required by some roles when assuming them, typically for cross-account access. Used only if 'role_arn' is specified and the role's trust policy requires it.
                type: string
            output_location:
                description: S3 URI where Athena query results should be stored (e.g., s3://your-bucket/athena/results/). Optional if the selected workgroup has a default result configuration.
                type: string
            region:
                description: AWS region where Athena and the result S3 bucket are located (e.g., us-east-1). Defaults to 'us-east-1' if not specified.
                type: string
            role_arn:
                description: ARN of the IAM role to assume. When specified, the SDK uses the base credentials to call STS AssumeRole and obtain temporary credentials scoped to this role.
                type: string
            role_session_name:
                description: Session name to associate with the STS AssumeRole session. Used only if 'role_arn' is specified. Useful for identifying and auditing the session.
                type: string
            workgroup:
                description: Athena workgroup to use for query execution. Defaults to 'primary' if not specified.
                type: string
          required:
            - driver
          title: Athena
          type: object
        - examples:
            - azure_storage_account: mystorageaccount
              azure_storage_key: '{{ .env.AZURE_STORAGE_KEY }}'
              driver: azure
              type: connector
          properties:
            allow_host_access:
                description: Allow access to host environment configuration
                type: boolean
            azure_storage_account:
                description: Azure storage account name
                type: string
            azure_storage_connection_string:
                description: Optional azure connection string for storage account
                type: string
            azure_storage_key:
                description: Azure storage access key
                type: string
            azure_storage_sas_token:
                description: Optional azure SAS token for authentication
                type: string
            driver:
                const: azure
                description: Refers to the driver type and must be driver `azure`
                type: string
            path_prefixes:
                description: |
                    A list of container or virtual directory prefixes that this connector is allowed to access.
                    Useful when different containers or paths use different credentials, allowing the system
                    to route access through the appropriate connector based on the blob path.
                    Example: `azure://my-bucket/`, ` azure://my-bucket/path/` ,`azure://my-bucket/path/prefix`
                type:
                    - string
                    - array
          required:
            - driver
            - azure_storage_account
            - azure_storage_key
          title: Azure
          type: object
        - examples:
            - allow_host_access: true
              driver: bigquery
              google_application_credentials: '{{ .env.GOOGLE_APPLICATION_CREDENTIALS }}'
              project_id: my-project-id
              type: connector
          properties:
            allow_host_access:
                description: Enable the BigQuery client to use credentials from the host environment when no service account JSON is provided. This includes Application Default Credentials from environment variables, local credential files, or the Google Compute Engine metadata server. Defaults to true, allowing seamless authentication in GCP environments.
                type: boolean
            allow_standard_api:
                description: Allow querying BigQuery using the standard API instead of the Storage Read API. This is less efficient and may lead to higher latency, but can be used as a fallback if the Storage Read API is not available due to insufficient permissions or other issues.
                type: boolean
            driver:
                const: bigquery
                description: Refers to the driver type and must be driver `bigquery`
                type: string
            google_application_credentials:
                description: Raw contents of the Google Cloud service account key (in JSON format) used for authentication.
                type: string
            log_queries:
                description: Controls whether to log raw SQL queries
                type: boolean
            max_bytes_billed:
                description: Maximum number of bytes billed for a query. Queries that exceed this limit will fail with an error. This can help prevent unexpectedly high costs from large queries. It is highly recommended to set this when running on `on-demand pricing` model. The default value is 0 i.e. no limits are enforced in Rill.
                type: integer
            project_id:
                description: Google Cloud project ID
                type: string
          required:
            - driver
          title: BigQuery
          type: object
        - examples:
            - cluster: mycluster
              database: mydatabase
              driver: clickhouse
              host: localhost
              managed: false
              mode: readwrite
              password: '{{ .env.CLICKHOUSE_PASSWORD }}'
              port: 9000
              ssl: true
              type: connector
              username: myusername
          properties:
            can_scale_to_zero:
                description: Indicates if the database can scale to zero
                type: boolean
            cluster:
                description: Cluster name, required for running distributed queries
                type: string
            conn_max_lifetime:
                description: Maximum time a connection may be reused
                type: string
            database:
                description: Name of the ClickHouse database within the cluster
                type: string
            database_whitelist:
                description: Comma-separated list of databases to show
                type: string
            dial_timeout:
                description: Timeout for dialing the ClickHouse server
                type: string
            driver:
                const: clickhouse
                description: Refers to the driver type and must be driver `clickhouse`
                type: string
            dsn:
                description: DSN(Data Source Name) for the ClickHouse connection
                type: string
            embed_port:
                description: Port to run ClickHouse locally (0 for random port)
                type: integer
            host:
                description: Host where the ClickHouse instance is running
                type: string
            log_queries:
                description: Controls whether to log raw SQL queries
                type: boolean
            managed:
                description: '`true` means Rill will provision the connector using the default provisioner. `false` disables automatic provisioning.'
                type: boolean
            max_idle_conns:
                description: Maximum number of idle connections in the pool
                type: integer
            max_open_conns:
                description: Maximum number of open connections to the database
                type: integer
            mode:
                description: '`read` - Controls the operation mode for the ClickHouse connection. Defaults to ''read'' for safe operation with external databases. Set to ''readwrite'' to enable model creation and table mutations. Note: When ''managed: true'', this is automatically set to ''readwrite''.'
                type: string
            password:
                description: Password for authentication
                type: string
            port:
                description: Port where the ClickHouse instance is accessible
                type: integer
            query_settings:
                description: query settings to be set on dashboard queries. `query_settings_override` takes precedence over these settings and if set these are ignored. Each setting must be separated by a comma. Example `max_threads = 8, max_memory_usage = 10000000000`
                type: string
            query_settings_override:
                description: override the default settings used in queries. Changing the default settings can lead to incorrect query results and is generally not recommended. If you need to add settings, use `query_settings`
                type: string
            read_timeout:
                description: Maximum time for a connection to read data
                type: string
            ssl:
                description: Indicates whether a secured SSL connection is required
                type: boolean
            sync_replicas:
                description: Controls whether to run `SYSTEM SYNC REPLICA` before replacing partitions on a replicated table in a cluster, ensuring all inserted parts are visible across replicas before the partition swap. Defaults to true
                type: boolean
            username:
                description: Username for authentication
                type: string
            write_dsn:
                description: Separate connection string for write operations
                type: string
          required:
            - driver
          title: ClickHouse
          type: object
        - examples:
            - catalog: my_catalog
              driver: databricks
              host: my-databricks-instance.cloud.databricks.com
              http_path: /sql/1.0/endpoints/1234567890abcdef
              schema: my_schema
              token: '{{ .env.DATABRICKS_TOKEN }}'
              type: connector
          properties:
            catalog:
                description: Default catalog name. Optional.
                type: string
            driver:
                const: databricks
                description: Refers to the driver type and must be driver `databricks`
                type: string
            dsn:
                description: "DSN (Data Source Name) for the Databricks connection.\n\nThis is intended for **advanced configuration** where you want to specify\nproperties that are not explicitly defined above.  \nIt can only be used when the other connection fields (host, http_path, token, catalog, schema) are **not used**.\nRefer to https://github.com/databricks/databricks-sql-go for the full list of supported DSN parameters and their formats.\n"
                type: string
            host:
                description: Host where the Databricks instance is running
                type: string
            http_path:
                description: HTTP path sets up the endpoint to the warehouse
                type: string
            log_queries:
                description: Controls whether to log raw SQL queries
                type: boolean
            schema:
                description: Default schema name. Optional.
                type: string
            token:
                description: Token sets up the Personal Access Token
                type: string
          required:
            - driver
            - type
          title: databricks
          type: object
        - examples:
            - driver: druid
              host: localhost
              password: '{{ .env.DRUID_PASSWORD }}'
              port: 8082
              ssl: true
              type: connector
              username: myusername
          properties:
            driver:
                const: druid
                description: Refers to the driver type and must be driver `druid`
                type: string
            dsn:
                description: Data Source Name (DSN) for connecting to Druid
                type: string
            host:
                description: Hostname of the Druid coordinator or broker
                type: string
            log_queries:
                description: Log raw SQL queries sent to Druid
                type: boolean
            max_open_conns:
                description: Maximum number of open database connections (0 = default, -1 = unlimited)
                type: integer
            password:
                description: Password for authenticating with Druid
                type: string
            port:
                description: Port number of the Druid service
                type: integer
            skip_version_check:
                description: Skip checking Druid version compatibility
                type: boolean
            ssl:
                description: Enable SSL for secure connection
                type: boolean
            username:
                description: Username for authenticating with Druid
                type: string
          required:
            - driver
          title: Druid
          type: object
        - examples:
            - allow_host_access: true
              cpu: 4
              driver: duckdb
              init_sql: INSTALL httpfs; LOAD httpfs;
              log_queries: true
              memory_limit_gb: 16
              mode: readwrite
              pool_size: 5
              read_write_ratio: 0.7
              type: connector
          properties:
            allow_host_access:
                description: Whether access to local environment and file system is allowed
                type: boolean
            attach:
                description: Full ATTACH statement to attach a DuckDB database
                type: string
            boot_queries:
                description: Deprecated - Use init_sql instead
                type: string
            conn_init_sql:
                description: SQL executed when a new connection is initialized
                type: string
            cpu:
                description: Number of CPU cores available to the database
                minimum: 1
                type: integer
            create_secrets_from_connectors:
                description: List of connector names for which temporary secrets should be created before executing the SQL.
                type:
                    - string
                    - array
            database_name:
                description: Name of the attached DuckDB database (auto-detected if not set)
                type: string
            driver:
                const: duckdb
                description: Must be "duckdb"
                type: string
            init_sql:
                description: SQL executed during database initialization
                type: string
            log_queries:
                description: Whether to log raw SQL queries executed through OLAP
                type: boolean
            memory_limit_gb:
                description: Amount of memory in GB available to the database
                minimum: 1
                type: integer
            mode:
                default: read
                description: Set the mode for the DuckDB connection.
                enum:
                    - read
                    - readwrite
                type: string
            path:
                description: Path to external DuckDB database
                type: string
            pool_size:
                description: Number of concurrent connections and queries allowed
                minimum: 1
                type: integer
            read_write_ratio:
                default: 0.5
                description: Ratio of resources allocated to read vs write operations
                maximum: 1
                minimum: 0
                type: number
            schema_name:
                description: Default schema used by the DuckDB database
                type: string
          required:
            - driver
          title: DuckDB
          type: object
        - examples:
            - driver: duckdb
              mode: read
              path: /path/to/my-duckdb-database.db
              type: connector
          properties:
            driver:
                const: duckdb
                description: Refers to the driver type and must be driver `duckdb`
                type: string
            mode:
                default: read
                description: Set the mode for the DuckDB connection.
                enum:
                    - read
                    - readwrite
                type: string
            path:
                description: Path to the DuckDB database
                type: string
          required:
            - driver
          title: External DuckDB
          type: object
        - examples:
            - driver: gcs
              google_application_credentials: '{{ .env.GOOGLE_APPLICATION_CREDENTIALS }}'
              type: connector
          properties:
            allow_host_access:
                description: Allow access to host environment configuration
                type: boolean
            driver:
                const: gcs
                description: Refers to the driver type and must be driver `gcs`
                type: string
            google_application_credentials:
                description: Google Cloud credentials JSON string
                type: string
            key_id:
                description: Optional S3-compatible Key ID when used in compatibility mode
                type: string
            path_prefixes:
                description: "A list of bucket path prefixes that this connector is allowed to access. \nUseful when different buckets or bucket prefixes use different credentials, \nallowing the system to select the appropriate connector based on the bucket path.\nExample: `gs://my-bucket/`, ` gs://my-bucket/path/` ,`gs://my-bucket/path/prefix`\n"
                type:
                    - string
                    - array
            secret:
                description: Optional S3-compatible Secret when used in compatibility mode
                type: string
          required:
            - driver
          title: GCS
          type: object
        - examples:
            - driver: https
              headers:
                Authorization: Bearer {{ .env.HTTPS_TOKEN }}
              type: connector
          properties:
            driver:
                const: https
                description: Refers to the driver type and must be driver `https`
                type: string
            headers:
                additionalProperties:
                    type: string
                description: HTTP headers to include in the request
                type: object
            path_prefixes:
                description: |
                    A list of HTTP/HTTPS URL prefixes that this connector is allowed to access.
                    Useful when different URL namespaces use different credentials, enabling the
                    system to choose the appropriate connector based on the URL path.
                    Example: `https://example.com/`, ` https://example.com/path/` ,`https://example.com/path/prefix`
                type:
                    - string
                    - array
          required:
            - driver
          title: HTTPS
          type: object
        - examples:
            - driver: duckdb
              path: md:my_database
              schema_name: my_schema
              token: '{{ .env.MOTHERDUCK_TOKEN }}'
              type: connector
          properties:
            create_secrets_from_connectors:
                description: List of connector names for which temporary secrets should be created before executing the SQL.
                type:
                    - string
                    - array
            driver:
                const: duckdb
                description: Refers to the driver type and must be driver `duckdb`.
                type: string
            init_sql:
                description: SQL executed during database initialization.
                type: string
            mode:
                default: read
                description: Set the mode for the MotherDuck connection. By default, it is set to 'read' which allows only read operations. Set to 'readwrite' to enable model creation and table mutations.
                enum:
                    - read
                    - readwrite
                type: string
            path:
                description: Path to your MD database
                type: string
            schema_name:
                description: Define your schema if not main, uses main by default
                type: string
            token:
                description: MotherDuck token
                type: string
          required:
            - driver
            - path
            - token
          title: MotherDuck
          type: object
        - examples:
            - database: mydb
              driver: mysql
              host: localhost
              password: '{{ .env.MYSQL_PASSWORD }}'
              port: 3306
              ssl-mode: preferred
              type: connector
              user: user
            - driver: mysql
              dsn: '{{ .env.MYSQL_DSN }}'
              type: connector
          properties:
            database:
                description: Name of the MySQL database
                type: string
            driver:
                description: Refers to the driver type and must be driver `mysql`
                type: string
            dsn:
                description: |
                    **Data Source Name (DSN)** for the MySQL connection, provided in [MySQL URI format](https://dev.mysql.com/doc/refman/8.4/en/connecting-using-uri-or-key-value-pairs.html#connecting-using-uri).
                    The DSN must follow the standard MySQL URI scheme:
                    ```text
                    mysql://user:password@host:3306/my-db
                    ```
                    Rules for special characters in password:
                    - The following characters are allowed [unescaped in the URI](https://datatracker.ietf.org/doc/html/rfc3986#section-2.3): `~` `.` `_` `-`
                    - All other special characters must be percent-encoded (`%XX` format).
                    ```text
                    mysql://user:pa%40ss@localhost:3306/my-db   # password contains '@'
                    mysql://user:pa%3Ass@localhost:3306/my-db   # password contains ':'
                    ```
                type: string
            host:
                description: Hostname of the MySQL server
                type: string
            log_queries:
                description: Controls whether to log raw SQL queries
                type: boolean
            password:
                description: Password for authentication
                type: string
            port:
                description: Port number for the MySQL server
                type: integer
            ssl-mode:
                description: 'ssl mode options: `disabled`, `preferred`, or `required`.'
                type: string
            user:
                description: Username for authentication
                type: string
          required:
            - driver
          title: MySQL
          type: object
        - examples:
            - api_key: '{{ .env.OPENAI_API_KEY }}'
              api_type: openai
              api_version: "2023-05-15"
              base_url: https://api.openai.com/v1
              driver: openai
              max_output_tokens: 8192
              model: gpt-4o
              reasoning_effort: medium
              type: connector
          properties:
            api_key:
                description: API key for connecting to OpenAI
                type: string
            api_type:
                description: The type of OpenAI API to use
                type: string
            api_version:
                description: The version of the OpenAI API to use (e.g., '2023-05-15'). Required when API Type is AZURE or AZURE_AD
                type: string
            base_url:
                description: The base URL for the OpenAI API (e.g., 'https://api.openai.com/v1')
                type: string
            driver:
                description: The driver type, must be set to "openai"
                type: string
            max_output_tokens:
                description: 'Maximum number of tokens to generate in the completion (default: 8192)'
                type: number
            model:
                description: The OpenAI model to use (e.g., 'gpt-4o')
                type: string
            reasoning_effort:
                description: Constrains effort on reasoning for reasoning models (e.g., 'low', 'medium', 'high')
                type: string
          required:
            - api_key
          title: OpenAI
          type: object
        - examples:
            - api_key: '{{ .env.claude_api_key }}'
              driver: claude
     

…(truncated)