Gapless Crypto Clickhouse Onboarding

Company Employee Onboarding: gapless-crypto-clickhouse + ClickHouse Cloud

Overview

This skill provides step-by-step workflow for onboarding 3-10 company employees (Claude Code CLI users) to use gapless-crypto-clickhouse package with ClickHouse Cloud credentials.

Target: <15 minute onboarding from zero to first successful query

Access Model: Binary access (admins have full ClickHouse Cloud access, non-admins use file-based API only)

Credential Methods: Doppler (recommended) OR local .env file (fallback)

When to Use This Skill

Trigger this skill when:

• Company employee needs first-time ClickHouse Cloud setup
• User mentions "onboarding", "company credentials", "ClickHouse Cloud access"
• User asks "how do I connect to ClickHouse Cloud?"
• User encounters ClickHouse connection errors (guide through troubleshooting)

Onboarding Workflow

Step 1: Verify Prerequisites (2 minutes)

Check installed:

• Python 3.12+ (python --version)
• gapless-crypto-clickhouse package (pip show gapless-crypto-clickhouse)

Install if missing:

pip install gapless-crypto-clickhouse

Verify installation:

python -c "import gapless_crypto_clickhouse; print(gapless_crypto_clickhouse.__version__)"
# Expected: 6.0.0 or higher

Step 2: Choose Credential Access Method (1 minute)

Option A: Doppler (Recommended)

• Centralized credential management
• No local credential storage
• Requires Doppler CLI access to aws-credentials/prd project

Option B: Local .env File (Fallback)

• Local credential file
• Simpler for one-off scripts
• Requires manual credential entry

Guide user to choose:

• If user has Doppler access → Use Option A
• If user prefers local development OR doesn't have Doppler → Use Option B

Step 3A: Configure Doppler Access (If Option A)

Verify Doppler access:

doppler secrets --project aws-credentials --config prd --only-names | grep CLICKHOUSE

Expected output (8 secrets):

CLICKHOUSE_CLOUD_KEY_ID
CLICKHOUSE_CLOUD_KEY_SECRET
CLICKHOUSE_CLOUD_ORG_ID
CLICKHOUSE_CLOUD_SERVICE_ID
CLICKHOUSE_HOST
CLICKHOUSE_PORT
CLICKHOUSE_USER
CLICKHOUSE_PASSWORD

If missing → User needs Doppler access from DevOps team

Reference: references/doppler-setup.md

Step 3B: Configure Local .env File (If Option B)

Copy Cloud template:

cp .env.cloud.example .env

Edit .env with credentials:

• Open .env in editor
• Fill in CLICKHOUSE_PASSWORD (from Doppler or ClickHouse Cloud console)
• Ensure CLICKHOUSE_SECURE=true
• Verify CLICKHOUSE_HOST ends with .aws.clickhouse.cloud

Security check:

grep "^\.env$" .gitignore

Expected: .env is in .gitignore (NEVER commit credentials)

Reference: references/env-setup.md

Step 4: Test Connection (2 minutes)

Option A (Doppler):

doppler run --project aws-credentials --config prd -- python skills/gapless-crypto-clickhouse-onboarding/scripts/test_connection_cloud.py

Option B (.env file):

python skills/gapless-crypto-clickhouse-onboarding/scripts/test_connection_cloud.py

Expected output:

✅ Connection successful!
   ClickHouse version: 25.8.1.8702
   User: default
   Tables visible: 150+
✅ All connection tests passed!

If errors → Guide user through troubleshooting (see Step 6)

Test script: scripts/test_connection_cloud.py

Step 5: Run First Query (5 minutes)

Simple query example (Option A - Doppler):

doppler run --project aws-credentials --config prd -- python -c "
import gapless_crypto_clickhouse as gcch

# Query OHLCV data (auto-ingests if not present)
df = gcch.query_ohlcv('BTCUSDT', '1h', '2024-01-01', '2024-01-31')

print(f'✅ Query successful: {len(df)} rows')
print(df.head())
"

Simple query example (Option B - .env):

python -c "
from dotenv import load_dotenv
load_dotenv('.env')  # Load credentials from .env

import gapless_crypto_clickhouse as gcch

# Query OHLCV data
df = gcch.query_ohlcv('BTCUSDT', '1h', '2024-01-01', '2024-01-31')

print(f'✅ Query successful: {len(df)} rows')
print(df.head())
"

Expected: DataFrame with OHLCV data (11 columns: open_time, open, high, low, close, volume, etc.)

Step 6: Troubleshooting Common Errors

Error: "Connection refused"

• Cause: Wrong hostname or port
• Fix: Verify CLICKHOUSE_HOST ends with .aws.clickhouse.cloud and CLICKHOUSE_HTTP_PORT=8443

Error: "SSL/TLS error" or "certificate verify failed"

• Cause: Missing secure=True parameter
• Fix: Ensure CLICKHOUSE_SECURE=true in environment variables
• Note: Package v6.0.0+ includes secure parameter support (ADR-0026)

Error: "Authentication failed" or "Invalid password"

• Cause: Wrong password or expired credentials
• Fix: Reset password in ClickHouse Cloud console → Update Doppler/.env

Error: "Timeout" (>10 seconds)

• Cause: Service resuming from idle state (15-minute idle scaling)
• Fix: Retry in 30 seconds (service should be awake)

Error: "Table 'gapless_crypto.klines' doesn't exist"

• Cause: Database schema not yet created on Cloud service
• Fix: Expected for new service, query_ohlcv() will auto-create schema

Complete troubleshooting guide: references/troubleshooting.md

Binary Access Model

Admins (3-10 users with ClickHouse Cloud credentials):

• Full access to query_ohlcv() and database features
• Use this onboarding skill for setup

Non-admins (no ClickHouse Cloud credentials):

• Use file-based API only (fetch_data(), download())
• No ClickHouse connection needed
• Package still useful for Binance data collection

Resources

scripts/

• test_connection_cloud.py: Connection validator with diagnostics (Doppler + .env support)

references/

• troubleshooting.md: Common errors + actionable fixes
• doppler-setup.md: Doppler CLI configuration workflow
• env-setup.md: Local .env file setup (fallback method)

Next Steps After Onboarding

1. Explore examples: Review /examples/
2. Read API docs: /docs/guides/python-api.md
3. Review architecture: /docs/architecture/OVERVIEW.md
4. Join support channel: #data-engineering Slack

Service Details

ClickHouse Cloud Service:

• Service ID: <CLICKHOUSE_SERVICE_ID>
• Region: us-west-2 (AWS)
• Console: https://clickhouse.cloud/services/
• Idle scaling: 15 minutes (first query may take 10s to resume)
• Credentials: Stored in Doppler (aws-credentials/prd) + 1Password (Engineering vault)

Related Skills

• clickhouse-cloud-service-setup: Infrastructure-focused (API-driven service discovery)
• clickhouse-cloud-credentials: Credential storage (Doppler + 1Password)
• clickhouse-cloud-connection: Connection validation + troubleshooting