Databricks Isv Java Jdbc

Databricks JDBC Driver (ISV)

Use this skill when implementing or testing Databricks JDBC Driver (databricks-jdbc v3+) connectivity for PWAF-compliant SQL warehouse integrations.

PWAF Documentation Links

Requirements

• Driver: com.databricks:databricks-jdbc:3.1.1 (or later 3.x) from Maven Central. OSS driver only.
• SDK: com.databricks:databricks-sdk-java:0.54.0 – Required for U2M browser-based token acquisition. Not needed for PAT/M2M/token-env.
• Java: 11+ required. Java 17+ needs --add-opens=java.base/java.nio=ALL-UNNAMED.
• User-Agent (required): Set UserAgentEntry on every connection URL (format <isv>_<product>/<version>).

Authentication Decision Guide

Which authentication method to use?

Production / automated workloads?
  → OAuth M2M (Auth_Flow=1)  ✅ RECOMMENDED

User-interactive (ISV custom OAuth app)?
  → U2M Custom OAuth App (SDK → Auth_Flow=0)  ✅ SUPPORTED

Already have an OAuth access token?
  → U2M Token-Env (Auth_Flow=0)  ✅ SUPPORTED

Local development/testing only?
  → PAT (AuthMech=3)  ⚠️ LIMITED

Note: These examples do NOT use the SDK's built-in databricks-cli OAuth app. ISV/partner applications should register custom OAuth apps in App connections for proper branding, audit trails, and scoped permissions.

Authentication Comparison

Auth Mapping (JDBC Parameters)

Both U2M flows use the same JDBC parameters (Auth_Flow=0 token pass-through). The difference is how the token is obtained.

CLIENT_ID Distinction (CRITICAL)

Using the wrong client_id causes: "OAuth application with client_id not available in Databricks account"

Why SDK + Auth_Flow=0 Instead of JDBC Auth_Flow=2?

The JDBC driver's native Auth_Flow=2 uses a built-in OAuth app with a hardcoded redirect port (8020). It does not support custom OAuth apps or custom redirect URIs.

For ISV integrations that need custom OAuth apps, the proven pattern is:

1. Use the Databricks Java SDK (DatabricksConfig with authType="external-browser") to obtain the token
2. Pass the token to the JDBC driver via Auth_Flow=0 (Auth_AccessToken)

This approach supports custom OAuth apps with configurable redirect URIs, which is required for proper ISV branding and audit trails.

Compute / Target

• SQL warehouse only: Connection uses httpPath (e.g., /sql/1.0/warehouses/<id>). The JDBC driver does not support jobs compute.
• Config: Require either DATABRICKS_HTTP_PATH or DATABRICKS_WAREHOUSE_ID (path derived as /sql/1.0/warehouses/<id>).

User-Agent (Required per PWAF)

Set on every JDBC connection URL:

String url = "jdbc:databricks://" + host + ":443"
    + ";httpPath=" + httpPath
    + ";AuthMech=3;UID=token;PWD=" + token
    + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

For SDK operations, also register static telemetry:

import com.databricks.sdk.core.UserAgent;

UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");

Environment Variables Reference

Important: Do not mix M2M and U2M environment variables. Use env -i for clean test environments.

Complete Examples

PAT Authentication (Testing Only)

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;

public class JdbcPatExample {
    public static void main(String[] args) throws Exception {
        String host = System.getenv("DATABRICKS_HOST");
        String token = System.getenv("DATABRICKS_TOKEN");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");

        // Normalize host
        host = host.replaceFirst("^https://", "").replaceFirst("^http://", "");

        String url = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=3"
            + ";UID=token"
            + ";PWD=" + token
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        try (Connection conn = DriverManager.getConnection(url);
             Statement stmt = conn.createStatement();
             ResultSet rs = stmt.executeQuery("SELECT 1 as test")) {
            if (rs.next()) {
                System.out.println("PAT OK: " + rs.getInt("test"));
            }
        }
    }
}

Env vars: DATABRICKS_HOST, DATABRICKS_TOKEN, DATABRICKS_HTTP_PATH

OAuth M2M Authentication (Production Recommended)

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;

public class JdbcOAuthM2MExample {
    public static void main(String[] args) throws Exception {
        String host = System.getenv("DATABRICKS_HOST");
        String clientId = System.getenv("DATABRICKS_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");

        host = host.replaceFirst("^https://", "").replaceFirst("^http://", "");

        String url = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11"
            + ";Auth_Flow=1"
            + ";OAuth2ClientId=" + clientId
            + ";OAuth2Secret=" + clientSecret
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        try (Connection conn = DriverManager.getConnection(url);
             Statement stmt = conn.createStatement();
             ResultSet rs = stmt.executeQuery("SELECT 1 as test")) {
            if (rs.next()) {
                System.out.println("OAuth M2M OK: " + rs.getInt("test"));
            }
        }
    }
}

Env vars: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET, DATABRICKS_HTTP_PATH

Setup: Create service principal in Account Console → Settings → Service principals. Generate OAuth secret.

U2M Custom OAuth App (SDK → JDBC)

Uses a custom OAuth app registered in App connections:

import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.UserAgent;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Arrays;
import java.util.Map;

public class JdbcU2MCustomOAuthAppExample {
    public static void main(String[] args) throws Exception {
        String host = System.getenv("DATABRICKS_HOST");
        String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET"); // required by Java SDK
        String redirectUri = System.getenv("DATABRICKS_REDIRECT_URI");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");

        UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
        UserAgent.withPartner("YourCompany");

        // Step 1: Get token via SDK with custom OAuth app
        String normalizedHost = host.startsWith("https://") ? host : "https://" + host;
        DatabricksConfig config = new DatabricksConfig()
                .setHost(normalizedHost)
                .setAuthType("external-browser")
                .setClientId(clientId)
                .setScopes(Arrays.asList("all-apis"));

        if (clientSecret != null && !clientSecret.isBlank()) {
            config.setClientSecret(clientSecret);
        }
        if (redirectUri != null && !redirectUri.isBlank()) {
            config.setOAuthRedirectUrl(redirectUri);
        }
        config.resolve();
        Map<String, String> headers = config.authenticate();
        String token = headers.get("Authorization").substring("Bearer ".length());

        // Step 2: Connect via JDBC
        String hostNorm = host.replaceFirst("^https://", "").replaceFirst("^http://", "");
        String url = "jdbc:databricks://" + hostNorm + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + token
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        try (Connection conn = DriverManager.getConnection(url)) {
            System.out.println("U2M Custom OAuth App OK");
        }
    }
}

Env vars: DATABRICKS_HOST, DATABRICKS_U2M_CLIENT_ID, DATABRICKS_HTTP_PATH

Recommended: DATABRICKS_U2M_CLIENT_SECRET (required by Java SDK for proper auth configuration)

Optional: DATABRICKS_REDIRECT_URI (must match App connections registration)

CRITICAL: Use DATABRICKS_U2M_CLIENT_ID for custom OAuth apps — NOT DATABRICKS_CLIENT_ID (that's for M2M).

Note: Testing showed the Java SDK requires DATABRICKS_U2M_CLIENT_SECRET for custom OAuth apps to properly configure external-browser auth. Without it, you may get "cannot configure default credentials" errors.

U2M Token-Env (Pre-obtained Token)

For headless/CI environments with a pre-obtained token:

import java.sql.Connection;
import java.sql.DriverManager;

public class JdbcU2MTokenEnvExample {
    public static void main(String[] args) throws Exception {
        String host = System.getenv("DATABRICKS_HOST");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");
        String accessToken = System.getenv("DATABRICKS_ACCESS_TOKEN");
        if (accessToken == null || accessToken.isBlank()) {
            accessToken = System.getenv("DATABRICKS_TOKEN");
        }

        String hostNorm = host.replaceFirst("^https://", "").replaceFirst("^http://", "");
        String url = "jdbc:databricks://" + hostNorm + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11"
            + ";Auth_Flow=0"
            + ";Auth_AccessToken=" + accessToken
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        try (Connection conn = DriverManager.getConnection(url)) {
            System.out.println("U2M Token-Env OK");
        }
    }
}

Env vars: DATABRICKS_HOST, DATABRICKS_HTTP_PATH, DATABRICKS_ACCESS_TOKEN (or DATABRICKS_TOKEN)

Token Refresh for U2M Flows

U2M access tokens expire after approximately 1 hour. For long-running JDBC applications, you need to handle token refresh and reconnection.

Token Lifetime Summary

JDBC Token Refresh Pattern

Since JDBC connections use a token at connection time, you need to obtain a fresh token and create a new connection when tokens expire:

public class JdbcTokenRefreshExample {
    private DatabricksConfig sdkConfig;
    private Connection jdbcConnection;
    private long tokenExpiresAt;
    private static final long REFRESH_BUFFER_MS = 5 * 60 * 1000; // 5 minutes

    public JdbcTokenRefreshExample() throws Exception {
        this.sdkConfig = createSdkConfig();
        this.jdbcConnection = createConnection();
        this.tokenExpiresAt = System.currentTimeMillis() + (55 * 60 * 1000);
    }

    private DatabricksConfig createSdkConfig() {
        String host = System.getenv("DATABRICKS_HOST");
        String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET");

        DatabricksConfig cfg = new DatabricksConfig()
                .setHost(host.startsWith("https://") ? host : "https://" + host)
                .setAuthType("external-browser")
                .setClientId(clientId)
                .setClientSecret(clientSecret)
                .setScopes(Arrays.asList("all-apis", "offline_access"));

        cfg.resolve();
        return cfg;
    }

    private String getToken() {
        Map<String, String> headers = sdkConfig.authenticate();
        return headers.get("Authorization").substring("Bearer ".length());
    }

    private Connection createConnection() throws Exception {
        String host = System.getenv("DATABRICKS_HOST")
                .replaceFirst("^https://", "").replaceFirst("^http://", "");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");
        String token = getToken();

        String url = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + token
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        return DriverManager.getConnection(url);
    }

    public Connection getConnection() throws Exception {
        if (System.currentTimeMillis() > (tokenExpiresAt - REFRESH_BUFFER_MS)) {
            // Token nearing expiration, refresh and reconnect
            if (jdbcConnection != null && !jdbcConnection.isClosed()) {
                jdbcConnection.close();
            }
            this.sdkConfig = createSdkConfig();
            this.jdbcConnection = createConnection();
            this.tokenExpiresAt = System.currentTimeMillis() + (55 * 60 * 1000);
        }
        return jdbcConnection;
    }

    public void close() throws Exception {
        if (jdbcConnection != null && !jdbcConnection.isClosed()) {
            jdbcConnection.close();
        }
    }
}

Refresh Token Scope

For applications that need to refresh without user interaction, request offline_access scope:

config.setScopes(Arrays.asList("all-apis", "offline_access"));

Important: The offline_access scope must be enabled for your OAuth app in App connections.

Best Practices

1. Use connection pooling with refresh: Combine with HikariCP for production (see Connection Pooling section)
2. Check token before queries: Avoid mid-query failures
3. Close old connections: Prevent connection leaks during refresh
4. Handle SQLException gracefully: Token expiration may manifest as SQL errors

Environment Variables Reference

JDBC URL Parameters Reference

Timeout Configuration

SSL/TLS Configuration

Warehouse Startup Configuration

Example with Timeout/SSL Parameters

String url = "jdbc:databricks://" + host + ":443"
    + ";httpPath=" + httpPath
    + ";AuthMech=11;Auth_Flow=1"
    + ";OAuth2ClientId=" + clientId
    + ";OAuth2Secret=" + clientSecret
    + ";SSL=1"
    + ";ConnectTimeout=30"
    + ";SocketTimeout=300"
    + ";QueryTimeout=600"
    + ";EnableAutoResume=1"
    + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

Statement-Level Timeout

For per-query timeout control, use Statement.setQueryTimeout():

try (Statement stmt = conn.createStatement()) {
    stmt.setQueryTimeout(300); // 5 minutes
    try (ResultSet rs = stmt.executeQuery("SELECT * FROM large_table")) {
        // Process results
    }
}

Required Imports / Maven Dependencies

<!-- Databricks JDBC Driver -->
<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>databricks-jdbc</artifactId>
    <version>3.1.1</version>
</dependency>

<!-- Databricks SDK (for U2M browser flows) -->
<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>databricks-sdk-java</artifactId>
    <version>0.54.0</version>
</dependency>

<!-- HikariCP for connection pooling (recommended for production) -->
<dependency>
    <groupId>com.zaxxer</groupId>
    <artifactId>HikariCP</artifactId>
    <version>5.1.0</version>
</dependency>

<!-- Suppress SDK logging (optional) -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-nop</artifactId>
    <version>2.0.16</version>
</dependency>

Connection Pooling with HikariCP

Production JDBC applications should use connection pooling for better performance and resource management. HikariCP is the recommended pool for Databricks JDBC.

Basic HikariCP Setup (OAuth M2M)

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;

public class JdbcConnectionPoolExample {
    private static HikariDataSource dataSource;

    public static void main(String[] args) throws Exception {
        initializePool();
        try {
            executeQuery();
        } finally {
            closePool();
        }
    }

    private static void initializePool() {
        String host = System.getenv("DATABRICKS_HOST")
                .replaceFirst("^https://", "").replaceFirst("^http://", "");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");
        String clientId = System.getenv("DATABRICKS_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");

        String jdbcUrl = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=1"
            + ";OAuth2ClientId=" + clientId
            + ";OAuth2Secret=" + clientSecret
            + ";SSL=1"
            + ";ConnectTimeout=30"
            + ";SocketTimeout=300"
            + ";EnableAutoResume=1"
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        HikariConfig config = new HikariConfig();
        config.setJdbcUrl(jdbcUrl);
        config.setPoolName("DatabricksPool");

        // Pool sizing
        config.setMinimumIdle(2);
        config.setMaximumPoolSize(10);

        // Connection lifecycle
        config.setIdleTimeout(300000);        // 5 minutes idle before eviction
        config.setMaxLifetime(1800000);       // 30 minutes max connection lifetime
        config.setConnectionTimeout(30000);   // 30 seconds to get connection from pool
        config.setKeepaliveTime(60000);       // 1 minute keepalive interval

        // Validation
        config.setConnectionTestQuery("SELECT 1");

        dataSource = new HikariDataSource(config);
    }

    private static void executeQuery() throws Exception {
        try (Connection conn = dataSource.getConnection();
             Statement stmt = conn.createStatement();
             ResultSet rs = stmt.executeQuery("SELECT count(*) as cnt FROM samples.nyctaxi.trips")) {
            if (rs.next()) {
                System.out.println("Row count: " + rs.getLong("cnt"));
            }
        }
    }

    private static void closePool() {
        if (dataSource != null && !dataSource.isClosed()) {
            dataSource.close();
        }
    }
}

HikariCP with U2M Token Refresh

For U2M flows where tokens expire, implement a custom connection provider that refreshes tokens:

import com.databricks.sdk.core.DatabricksConfig;
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;

public class JdbcU2MPoolExample {
    private final AtomicReference<HikariDataSource> dataSourceRef = new AtomicReference<>();
    private final AtomicReference<Long> tokenExpiresAt = new AtomicReference<>(0L);
    private static final long REFRESH_BUFFER_MS = 5 * 60 * 1000; // 5 minutes

    public synchronized HikariDataSource getDataSource() {
        long now = System.currentTimeMillis();
        if (now > (tokenExpiresAt.get() - REFRESH_BUFFER_MS)) {
            // Token expiring soon, create new pool with fresh token
            HikariDataSource oldDs = dataSourceRef.get();

            String token = obtainFreshToken();
            HikariDataSource newDs = createPoolWithToken(token);

            dataSourceRef.set(newDs);
            tokenExpiresAt.set(now + (55 * 60 * 1000)); // ~55 minutes

            // Close old pool gracefully
            if (oldDs != null && !oldDs.isClosed()) {
                oldDs.close();
            }
        }
        return dataSourceRef.get();
    }

    private String obtainFreshToken() {
        String host = System.getenv("DATABRICKS_HOST");
        String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET");

        DatabricksConfig config = new DatabricksConfig()
                .setHost(host.startsWith("https://") ? host : "https://" + host)
                .setAuthType("external-browser")
                .setClientId(clientId)
                .setClientSecret(clientSecret)
                .setScopes(Arrays.asList("all-apis", "offline_access"));

        config.resolve();
        Map<String, String> headers = config.authenticate();
        return headers.get("Authorization").substring("Bearer ".length());
    }

    private HikariDataSource createPoolWithToken(String token) {
        String host = System.getenv("DATABRICKS_HOST")
                .replaceFirst("^https://", "").replaceFirst("^http://", "");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");

        String jdbcUrl = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + token
            + ";SSL=1;ConnectTimeout=30;SocketTimeout=300"
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        HikariConfig config = new HikariConfig();
        config.setJdbcUrl(jdbcUrl);
        config.setPoolName("DatabricksU2MPool");
        config.setMinimumIdle(2);
        config.setMaximumPoolSize(10);
        config.setMaxLifetime(1800000);  // 30 minutes (less than token TTL)
        config.setConnectionTestQuery("SELECT 1");

        return new HikariDataSource(config);
    }

    public void shutdown() {
        HikariDataSource ds = dataSourceRef.get();
        if (ds != null && !ds.isClosed()) {
            ds.close();
        }
    }
}

Pool Configuration Best Practices

Important for U2M: Set maxLifetime shorter than the token TTL (~1 hour) to ensure connections don't outlive their tokens.

Pool Monitoring

Monitor pool health at runtime using HikariCP's MXBean:

// Get pool statistics
HikariPoolMXBean poolMXBean = dataSource.getHikariPoolMXBean();
System.out.println("Active connections: " + poolMXBean.getActiveConnections());
System.out.println("Idle connections: " + poolMXBean.getIdleConnections());
System.out.println("Total connections: " + poolMXBean.getTotalConnections());
System.out.println("Threads awaiting connection: " + poolMXBean.getThreadsAwaitingConnection());

Health check pattern:

public boolean isPoolHealthy() {
    HikariPoolMXBean mxBean = dataSource.getHikariPoolMXBean();
    return !dataSource.isClosed() 
        && mxBean.getTotalConnections() > 0
        && mxBean.getThreadsAwaitingConnection() < mxBean.getTotalConnections();
}

Multi-Auth Pattern

Support multiple auth types in a single binary using DATABRICKS_AUTH_TYPE:

String authType = System.getenv("DATABRICKS_AUTH_TYPE");
if (authType == null || authType.isBlank()) authType = "oauth_m2m";

String host = System.getenv("DATABRICKS_HOST").replaceFirst("^https://", "").replaceFirst("^http://", "");
String httpPath = System.getenv("DATABRICKS_HTTP_PATH");
String url;

switch (authType.toLowerCase()) {
    case "pat":
        String token = System.getenv("DATABRICKS_TOKEN");
        url = "jdbc:databricks://" + host + ":443;httpPath=" + httpPath
            + ";AuthMech=3;UID=token;PWD=" + token
            + ";UserAgentEntry=YourCompany/1.0.0";
        break;
    case "oauth_m2m":
        String clientId = System.getenv("DATABRICKS_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");
        url = "jdbc:databricks://" + host + ":443;httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=1;OAuth2ClientId=" + clientId
            + ";OAuth2Secret=" + clientSecret
            + ";UserAgentEntry=YourCompany/1.0.0";
        break;
    case "u2m_custom_oauth_app":
        String customToken = getTokenViaCustomOAuthApp(); // see helper below
        url = "jdbc:databricks://" + host + ":443;httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + customToken
            + ";UserAgentEntry=YourCompany/1.0.0";
        break;
    case "u2m_token_env":
        String envToken = System.getenv("DATABRICKS_ACCESS_TOKEN");
        if (envToken == null || envToken.isBlank()) envToken = System.getenv("DATABRICKS_TOKEN");
        url = "jdbc:databricks://" + host + ":443;httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + envToken
            + ";UserAgentEntry=YourCompany/1.0.0";
        break;
    default:
        throw new IllegalArgumentException("Unknown DATABRICKS_AUTH_TYPE: " + authType);
}

Helper for U2M Custom OAuth App token acquisition:

private static String getTokenViaCustomOAuthApp() {
    String host = System.getenv("DATABRICKS_HOST");
    String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
    String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET");
    String redirectUri = System.getenv("DATABRICKS_REDIRECT_URI");
    
    DatabricksConfig config = new DatabricksConfig()
            .setHost(host.startsWith("https://") ? host : "https://" + host)
            .setAuthType("external-browser")
            .setClientId(clientId)
            .setClientSecret(clientSecret)  // Required by Java SDK
            .setScopes(Arrays.asList("all-apis"));
    
    if (redirectUri != null && !redirectUri.isBlank()) {
        config.setOAuthRedirectUrl(redirectUri);
    }
    config.resolve();
    
    Map<String, String> headers = config.authenticate();
    return headers.get("Authorization").substring("Bearer ".length());
}

Implementation Learnings

Key findings from testing that differ from documentation:

1. Java SDK Requires Client Secret for U2M

When using the Java SDK to obtain tokens for JDBC (U2M custom OAuth app), you must provide DATABRICKS_U2M_CLIENT_SECRET:

DatabricksConfig config = new DatabricksConfig()
        .setHost(host)
        .setAuthType("external-browser")
        .setClientId(System.getenv("DATABRICKS_U2M_CLIENT_ID"))
        .setClientSecret(System.getenv("DATABRICKS_U2M_CLIENT_SECRET"))  // Required!
        .setScopes(Arrays.asList("all-apis"));

Without client secret: "cannot configure default credentials"

2. Token Extraction Pattern

Extract the token from SDK authentication headers:

config.resolve();
Map<String, String> headers = config.authenticate();  // Opens browser
String token = headers.get("Authorization").substring("Bearer ".length());

3. Host Normalization Differs

• SDK expects https:// prefix: config.setHost("https://workspace.cloud.databricks.com")
• JDBC expects bare hostname: jdbc:databricks://workspace.cloud.databricks.com:443

Always normalize the host appropriately for each:

// For SDK
String sdkHost = host.startsWith("https://") ? host : "https://" + host;

// For JDBC URL
String jdbcHost = host.replaceFirst("^https://", "").replaceFirst("^http://", "");

SDK Workarounds Summary (Java SDK v0.54.0)

Redirect URI Reference

Validation Query

Verify User-Agent telemetry is being recorded correctly:

SELECT event_time, user_agent, action_name, request_params
FROM system.access.audit
WHERE event_time > current_timestamp() - INTERVAL 1 HOUR
  AND lower(user_agent) LIKE '%yourcompany%'
ORDER BY event_time DESC
LIMIT 10;

Auth Isolation

Run tests with a clean environment using env -i. Important: Include MAVEN_OPTS for Java 17+.

# Set MAVEN_OPTS for Java 17+
export MAVEN_OPTS="--add-opens=java.base/java.nio=ALL-UNNAMED"

# PAT test
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" MAVEN_OPTS="$MAVEN_OPTS" \
  DATABRICKS_HOST="$DATABRICKS_HOST" \
  DATABRICKS_HTTP_PATH="$DATABRICKS_HTTP_PATH" \
  DATABRICKS_TOKEN="$DATABRICKS_TOKEN" \
  DATABRICKS_AUTH_TYPE=pat \
  mvn -q exec:exec -Dexec.mainClass="com.example.JdbcPatExample"

# OAuth M2M test
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" MAVEN_OPTS="$MAVEN_OPTS" \
  DATABRICKS_HOST="$DATABRICKS_HOST" \
  DATABRICKS_HTTP_PATH="$DATABRICKS_HTTP_PATH" \
  DATABRICKS_CLIENT_ID="$DATABRICKS_CLIENT_ID" \
  DATABRICKS_CLIENT_SECRET="$DATABRICKS_CLIENT_SECRET" \
  DATABRICKS_AUTH_TYPE=oauth_m2m \
  mvn -q exec:exec -Dexec.mainClass="com.example.JdbcOAuthM2MExample"

# U2M Custom OAuth App test (needs client secret for Java SDK)
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" MAVEN_OPTS="$MAVEN_OPTS" \
  DATABRICKS_HOST="$DATABRICKS_HOST" \
  DATABRICKS_HTTP_PATH="$DATABRICKS_HTTP_PATH" \
  DATABRICKS_U2M_CLIENT_ID="$DATABRICKS_U2M_CLIENT_ID" \
  DATABRICKS_U2M_CLIENT_SECRET="$DATABRICKS_U2M_CLIENT_SECRET" \
  DATABRICKS_REDIRECT_URI="$DATABRICKS_REDIRECT_URI" \
  DATABRICKS_AUTH_TYPE=u2m_custom_oauth_app \
  mvn -q exec:exec -Dexec.mainClass="com.example.JdbcU2MCustomOAuthAppExample"

# U2M Token-Env test
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" MAVEN_OPTS="$MAVEN_OPTS" \
  DATABRICKS_HOST="$DATABRICKS_HOST" \
  DATABRICKS_HTTP_PATH="$DATABRICKS_HTTP_PATH" \
  DATABRICKS_ACCESS_TOKEN="$DATABRICKS_ACCESS_TOKEN" \
  DATABRICKS_AUTH_TYPE=u2m_token_env \
  mvn -q exec:exec -Dexec.mainClass="com.example.JdbcU2MTokenEnvExample"

Java 17+

The OSS driver 3.x uses Arrow. On Java 17+, the JVM must allow Arrow's reflection via --add-opens=java.base/java.nio=ALL-UNNAMED.

⚠️ CRITICAL: Use exec:exec, NOT exec:java in exec-maven-plugin

exec:java runs code inside the Maven JVM process — jvmArguments are completely ignored. --add-opens flags never take effect and Apache Arrow fails at startup with:

java.lang.ExceptionInInitializerError: Failed to initialize MemoryUtil
InaccessibleObjectException: Unable to make field accessible...

Only exec:exec forks a real child JVM where startup flags work. The <classpath/> element (no attributes) is a Maven magic marker that expands to the full classpath.

Correct pom.xml configuration (exec:exec with --add-opens):

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>3.1.0</version>
    <configuration>
        <executable>java</executable>
        <arguments>
            <argument>--add-opens=java.base/java.nio=ALL-UNNAMED</argument>
            <argument>-classpath</argument>
            <classpath/>  <!-- Maven expands this to the full classpath string -->
            <argument>com.example.YourMainClass</argument>
        </arguments>
    </configuration>
</plugin>

Run with: mvn exec:exec (not mvn exec:java)

Via MAVEN_OPTS (alternative, works with both exec:java and exec:exec):

export MAVEN_OPTS="--add-opens=java.base/java.nio=ALL-UNNAMED"

Error Handling Patterns

Production JDBC applications should handle connection and query errors gracefully.

Basic Error Handling

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

public class JdbcErrorHandlingExample {
    public static void main(String[] args) {
        try {
            executeQuery();
        } catch (SQLException e) {
            handleSQLException(e);
        } catch (Exception e) {
            handleGeneralError(e);
        }
    }

    private static void executeQuery() throws Exception {
        String host = System.getenv("DATABRICKS_HOST");
        String httpPath = System.getenv("DATABRICKS_HTTP_PATH");
        String clientId = System.getenv("DATABRICKS_CLIENT_ID");
        String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");

        // Validate required environment variables
        if (host == null || host.isBlank()) {
            throw new IllegalStateException("DATABRICKS_HOST is required");
        }
        if (httpPath == null || httpPath.isBlank()) {
            throw new IllegalStateException("DATABRICKS_HTTP_PATH is required");
        }

        host = host.replaceFirst("^https://", "").replaceFirst("^http://", "");
        String url = "jdbc:databricks://" + host + ":443"
            + ";httpPath=" + httpPath
            + ";AuthMech=11;Auth_Flow=1"
            + ";OAuth2ClientId=" + clientId
            + ";OAuth2Secret=" + clientSecret
            + ";UserAgentEntry=YourCompany_YourProduct/1.0.0";

        try (Connection conn = DriverManager.getConnection(url);
             Statement stmt = conn.createStatement();
             ResultSet rs = stmt.executeQuery("SELECT 1 as test")) {
            if (rs.next()) {
                System.out.println("Success: " + rs.getInt("test"));
            }
        }
    }

    private static void handleSQLException(SQLException e) {
        String sqlState = e.getSQLState();
        int errorCode = e.getErrorCode();
        String message = e.getMessage();

        System.err.println("SQL Error [" + sqlState + "] (" + errorCode + "): " + message);

        if (message.contains("invalid_client") || message.contains("Invalid credentials")) {
            System.err.println("Authentication failed:");
            System.err.println("  - Verify DATABRICKS_CLIENT_ID is correct");
            System.err.println("  - Verify DATABRICKS_CLIENT_SECRET is correct");
        } else if (message.contains("INVALID_ACCESS_TOKEN") || message.con

…(truncated)