OpenFlow Google Drive Connector Demo
Google Drive to Snowflake (Cortex Search) via OpenFlow. No domain-wide delegation required.
Prerequisites (Gate Checks)
Run these checks before routing to any sub-skill. If any gate fails, STOP and guide the user to fix it.
Gate 0: Environment Bootstrap
MANDATORY: Execute every step in ../bootstrap/SKILL.md before proceeding. Do NOT skip this gate. Do NOT proceed to Gate 1 until bootstrap is fully complete.
The bootstrap:
- Copies scaffold files (
.env.example, .mise.toml, sql/) to $PWD
- Resolves and verifies Snowflake connection
- Creates
.env (ask for OPENFLOW_GDRIVE_ID and OPENFLOW_GDRIVE_FOLDER_NAME)
- Creates
.sfutils/manifest.toml
- Runs
mise install and mise run setup
- Runs
mise run sf-setup (creates roles, database, warehouse)
- Creates PAT via
$sfutils:programmatic-access-token
- Creates EAI via
$sfutils:network-rule
- Discovers runtime and creates nipyapi profile via
$openflow (uses PAT from Step 7)
If any step fails, STOP and help the user fix it. Do NOT skip steps.
Gate 1: Google Shared Drive
Important: The connector works with Google Shared Drives only -- regular Drive folders will silently fail (no files discovered).
Ask the user via ask_user_question:
- "Shared Drive is ready with documents" -- proceed
- "I need to set this up" -- guide them:
- Create a Google Shared Drive (not a regular folder)
- Upload sample documents (run
mise run sample-data for test files)
- Grant the GCP service account access to the Shared Drive
- Come back and confirm
Gate 2: GCP Service Account Credential
test -f "$(eval echo ${GCP_CREDENTIAL_PATH:-~/.snowflake/cortex/secrets/gcp-service-account.json})" && echo "OK" || echo "MISSING"
If MISSING: STOP. Ask user to place the GCP service account JSON key at the expected path.
Config Resolution
Load references/config-resolution.md for the resolution snippet, manifest schema, and naming conventions.
Resolution order: manifest.toml > environment variables > prompt user.
Intent Detection
| Intent |
Triggers |
Action |
| SETUP |
"setup", "provision", "configure", "first time", "initialize" |
Load setup/SKILL.md |
| DEPLOY |
"deploy", "run", "start", "add connector" |
Load deploy/SKILL.md |
| CLEANUP |
"cleanup", "clean", "reset", "tear down", "remove" |
Load cleanup/SKILL.md |
| UPDATE_IP |
"moved networks", "new network", "conference wifi", "hotel wifi", "IP changed", "update IP", "blocked", "can't connect" |
Load update-ip/SKILL.md |
If ambiguous, ask the user which mode.
Forbidden Actions
- Do not delete
.sfutils/ -- the manifest tracks all provisioned resources across runs; losing it means manual cleanup
- Do not write PAT or network rule SQL directly -- always invoke
$sfutils:programmatic-access-token or $sfutils:network-rule which handle keyring, manifest, and IDD compliance
- Do not hardcode config values -- read from manifest so replay and multi-user flows work
- Do not proceed with empty required fields -- empty values cause silent failures in NiFi parameter contexts
- Do not silently use env-sourced values -- env vars may be stale from a previous session; always confirm
- Do not guess warehouse or resource names -- wrong names cause permission errors that are hard to diagnose
Stopping Points
- After Gate checks: if any gate fails
- Each sub-skill has its own stopping points (see individual SKILL.md files)
Troubleshooting
| Issue |
Cause |
Fix |
| nipyapi 403 "Authorization denied" |
PAT role restriction doesn't match OpenFlow Admin role |
Recreate PAT with ROLE_RESTRICTION = OPENFLOW_ADMIN (the role from setup.sql with NiFi canvas access) |
| nipyapi 401 "Integration does not exist" |
SA_ROLE lacks runtime's execute_as_role |
GRANT ROLE {execute_as_role} TO ROLE {SA_ROLE} |
| nipyapi 401 |
PAT expired |
Invoke $sfutils:programmatic-access-token to rotate |
| nipyapi 401 after rotation |
IP changed |
Load update-ip/SKILL.md |
| nipyapi hits localhost:9443 |
Profile not created or wrong name |
Re-run nipyapi profile creation |
| Token extraction fails |
show-pat can't find keyring entry |
pat verify --user {SA_USER} |
| Schema creation fails |
Missing permissions |
Check role grants, use admin_role |
| No data after 2 min |
EAI not attached |
Verify EAI on runtime via DESCRIBE |
| No data after 2 min |
Wrong Drive ID |
Must be a Shared Drive ID, not folder ID |
| No data after 2 min |
Schema ownership |
Verify role owns schema |
pat check-setup --run-setup fails |
Template rendering error with newer snow CLI |
Create DB/schemas manually: CREATE DATABASE IF NOT EXISTS {DB}; CREATE SCHEMA IF NOT EXISTS {DB}.NETWORKS; CREATE SCHEMA IF NOT EXISTS {DB}.POLICIES; |
upload_asset fails for GCP credential |
Sensitive parameter in inherited context cannot be overridden |
Use configure_inherited_params instead of upload_asset for sensitive params |
mise .env parse error |
Unquoted value with spaces |
Always quote values in .env: OPENFLOW_GDRIVE_FOLDER_NAME="My Folder" |
Output
Depending on intent:
- DEPLOY: Running connector with data flowing into Cortex Search
- SETUP: Infrastructure provisioned, manifest populated
- CLEANUP: Schema dropped, flow deleted, manifest reset
- UPDATE_IP: Network rule updated, PAT rotated, connectivity restored
1---2name: openflow-gdrive-demo3description: Deploy, clean up, or manage the Google Drive CDC connector (no DWD) on OpenFlow. Use when: openflow demo, gdrive connector, deploy google drive connector, cleanup, reset demo, setup openflow, deploy connector, start connector, configure openflow.4---56# OpenFlow Google Drive Connector Demo78Google Drive to Snowflake (Cortex Search) via OpenFlow. No domain-wide delegation required.910## Prerequisites (Gate Checks)1112Run these checks before routing to any sub-skill. If any gate fails, STOP and guide the user to fix it.1314### Gate 0: Environment Bootstrap1516**MANDATORY: Execute every step in `../bootstrap/SKILL.md` before proceeding.** Do NOT skip this gate. Do NOT proceed to Gate 1 until bootstrap is fully complete.1718The bootstrap:191. Copies scaffold files (`.env.example`, `.mise.toml`, `sql/`) to `$PWD`202. Resolves and verifies Snowflake connection213. Creates `.env` (ask for `OPENFLOW_GDRIVE_ID` and `OPENFLOW_GDRIVE_FOLDER_NAME`)224. Creates `.sfutils/manifest.toml`235. Runs `mise install` and `mise run setup`246. Runs `mise run sf-setup` (creates roles, database, warehouse)257. Creates PAT via `$sfutils:programmatic-access-token`268. Creates EAI via `$sfutils:network-rule`279. Discovers runtime and creates nipyapi profile via `$openflow` (uses PAT from Step 7)2829If any step fails, STOP and help the user fix it. Do NOT skip steps.3031### Gate 1: Google Shared Drive3233**Important:** The connector works with **Google Shared Drives only** -- regular Drive folders will silently fail (no files discovered).3435Ask the user via `ask_user_question`:36- "Shared Drive is ready with documents" -- proceed37- "I need to set this up" -- guide them:38 1. Create a Google Shared Drive (not a regular folder)39 2. Upload sample documents (run `mise run sample-data` for test files)40 3. Grant the GCP service account access to the Shared Drive41 4. Come back and confirm4243### Gate 2: GCP Service Account Credential4445```bash46test -f "$(eval echo ${GCP_CREDENTIAL_PATH:-~/.snowflake/cortex/secrets/gcp-service-account.json})" && echo "OK" || echo "MISSING"47```4849**If MISSING:** STOP. Ask user to place the GCP service account JSON key at the expected path.5051## Config Resolution5253Load `references/config-resolution.md` for the resolution snippet, manifest schema, and naming conventions.5455Resolution order: manifest.toml > environment variables > prompt user.5657## Intent Detection5859| Intent | Triggers | Action |60|--------|----------|--------|61| SETUP | "setup", "provision", "configure", "first time", "initialize" | **Load** `setup/SKILL.md` |62| DEPLOY | "deploy", "run", "start", "add connector" | **Load** `deploy/SKILL.md` |63| CLEANUP | "cleanup", "clean", "reset", "tear down", "remove" | **Load** `cleanup/SKILL.md` |64| UPDATE_IP | "moved networks", "new network", "conference wifi", "hotel wifi", "IP changed", "update IP", "blocked", "can't connect" | **Load** `update-ip/SKILL.md` |6566If ambiguous, ask the user which mode.6768## Forbidden Actions6970- **Do not delete `.sfutils/`** -- the manifest tracks all provisioned resources across runs; losing it means manual cleanup71- **Do not write PAT or network rule SQL directly** -- always invoke `$sfutils:programmatic-access-token` or `$sfutils:network-rule` which handle keyring, manifest, and IDD compliance72- **Do not hardcode config values** -- read from manifest so replay and multi-user flows work73- **Do not proceed with empty required fields** -- empty values cause silent failures in NiFi parameter contexts74- **Do not silently use env-sourced values** -- env vars may be stale from a previous session; always confirm75- **Do not guess warehouse or resource names** -- wrong names cause permission errors that are hard to diagnose7677## Stopping Points7879- After Gate checks: if any gate fails80- Each sub-skill has its own stopping points (see individual SKILL.md files)8182## Troubleshooting8384| Issue | Cause | Fix |85|-------|-------|-----|86| nipyapi 403 "Authorization denied" | PAT role restriction doesn't match OpenFlow Admin role | Recreate PAT with `ROLE_RESTRICTION = OPENFLOW_ADMIN` (the role from setup.sql with NiFi canvas access) |87| nipyapi 401 "Integration does not exist" | SA_ROLE lacks runtime's `execute_as_role` | `GRANT ROLE {execute_as_role} TO ROLE {SA_ROLE}` |88| nipyapi 401 | PAT expired | Invoke `$sfutils:programmatic-access-token` to rotate |89| nipyapi 401 after rotation | IP changed | **Load** `update-ip/SKILL.md` |90| nipyapi hits localhost:9443 | Profile not created or wrong name | Re-run nipyapi profile creation |91| Token extraction fails | `show-pat` can't find keyring entry | `pat verify --user {SA_USER}` |92| Schema creation fails | Missing permissions | Check role grants, use admin_role |93| No data after 2 min | EAI not attached | Verify EAI on runtime via DESCRIBE |94| No data after 2 min | Wrong Drive ID | Must be a Shared Drive ID, not folder ID |95| No data after 2 min | Schema ownership | Verify role owns schema |96| `pat check-setup --run-setup` fails | Template rendering error with newer snow CLI | Create DB/schemas manually: `CREATE DATABASE IF NOT EXISTS {DB}; CREATE SCHEMA IF NOT EXISTS {DB}.NETWORKS; CREATE SCHEMA IF NOT EXISTS {DB}.POLICIES;` |97| `upload_asset` fails for GCP credential | Sensitive parameter in inherited context cannot be overridden | Use `configure_inherited_params` instead of `upload_asset` for sensitive params |98| mise `.env` parse error | Unquoted value with spaces | Always quote values in `.env`: `OPENFLOW_GDRIVE_FOLDER_NAME="My Folder"` |99100## Output101102Depending on intent:103- **DEPLOY**: Running connector with data flowing into Cortex Search104- **SETUP**: Infrastructure provisioned, manifest populated105- **CLEANUP**: Schema dropped, flow deleted, manifest reset106- **UPDATE_IP**: Network rule updated, PAT rotated, connectivity restored
snowflake-labs