How do I install the Zero Copy Ingestion skill?

Run npx skillmds@latest add googlecloudplatform/zero-copy-ingestion in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Zero Copy Ingestion skill do?

Skill: Zero-Copy Data Exploration & Ingestion It is listed under Coding & Dev Tools on SkillMD.

Is Zero Copy Ingestion safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Zero Copy Ingestion?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Zero Copy Ingestion free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Zero Copy Ingestion?

GoogleCloudPlatform (@googlecloudplatform) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Zero Copy Ingestion by googlecloudplatform

Name: Zero Copy Ingestion
Author: GoogleCloudPlatform

File contents

Skill: Zero-Copy Data Exploration & Ingestion

1. Description & Rationale

This skill enforces the Zero-Copy Architecture of the modern Google Cloud Lakehouse. It eliminates the overhead of copying data to external compute engines or creating intermediate files (CSVs/JSONs) on GCS.

It standardizes:

Reading from BigQuery directly into Spark DataFrames using the Spark-BigQuery connector.
Reading from BigQuery using BigQuery DataFrames (bigframes.pandas) for in-warehouse processing.

2. Environment Prerequisites

Spark: Managed Service for Apache Spark Serverless (Runtime 3.0+ recommended) with the Spark BigQuery Connector pre-bundled.
BigQuery: bigframes Python package installed.

3. Agent Execution Guidelines (System Prompts)

When generating data loading or exploration code:

Never export to GCS first: Do not write code that exports BigQuery tables to GCS as CSV/Avro/Parquet before loading them into Spark or Pandas.
Use BigQuery Storage API: Ensure Spark reads use the "bigquery" format, which utilizes the high-speed BigQuery Storage Read API.
Use BigFrames for BQ: Ensure BigQuery-centric pipelines use bigframes.pandas.read_gbq to create references to the tables.

4. Opinionated Code Patterns

A. Spark-Centric Zero-Copy Load

Use the Spark-BigQuery connector to load tables directly.

# Load BigQuery table directly into Spark DataFrame
users_df = (
    spark.read.format("bigquery")
    .option("table", "bigquery-public-data.thelook_ecommerce.users")
    .load()
)

# Create temporary view for Spark SQL queries
users_df.createOrReplaceTempView("users")

B. BigQuery-Centric Zero-Copy Load

Use BigQuery DataFrames to create a virtual DataFrame.

import bigframes.pandas as bpd

# Configure BigQuery DataFrames global options
bpd.options.bigquery.project = PROJECT_ID
bpd.options.bigquery.location = "US"  # Or your dataset location

# Load table reference (zero data movement)
users_df = bpd.read_gbq("bigquery-public-data.thelook_ecommerce.users")

5. Verification Checklist

No GCS Export: Verify the code does not contain bq extract or GCS export steps before loading.
Spark Format: Ensure the Spark read format is explicitly "bigquery".
BigFrames Config: Ensure bpd.options.bigquery.project is set before calling read_gbq.

---name: zero-copy-ingestiondescription: Skill: Zero-Copy Data Exploration & Ingestion---# Skill: Zero-Copy Data Exploration & Ingestion## 1. Description & RationaleThis skill enforces the **Zero-Copy Architecture** of the modern Google CloudLakehouse. It eliminates the overhead of copying data to external computeengines or creating intermediate files (CSVs/JSONs) on GCS.It standardizes:1. Reading from BigQuery directly into Spark DataFrames using the Spark-BigQuery connector.2. Reading from BigQuery using BigQuery DataFrames (`bigframes.pandas`) for in-warehouse processing.## 2. Environment Prerequisites- **Spark**: Managed Service for Apache Spark Serverless (Runtime 3.0+ recommended) with the Spark BigQuery Connector pre-bundled.- **BigQuery**: `bigframes` Python package installed.## 3. Agent Execution Guidelines (System Prompts)When generating data loading or exploration code:1. **Never export to GCS first**: Do not write code that exports BigQuery tables to GCS as CSV/Avro/Parquet before loading them into Spark or Pandas.2. **Use BigQuery Storage API**: Ensure Spark reads use the `"bigquery"` format, which utilizes the high-speed BigQuery Storage Read API.3. **Use BigFrames for BQ**: Ensure BigQuery-centric pipelines use `bigframes.pandas.read_gbq` to create references to the tables.## 4. Opinionated Code Patterns### A. Spark-Centric Zero-Copy LoadUse the Spark-BigQuery connector to load tables directly.```python# Load BigQuery table directly into Spark DataFrameusers_df = ( spark.read.format("bigquery") .option("table", "bigquery-public-data.thelook_ecommerce.users") .load())# Create temporary view for Spark SQL queriesusers_df.createOrReplaceTempView("users")```### B. BigQuery-Centric Zero-Copy LoadUse BigQuery DataFrames to create a virtual DataFrame.```pythonimport bigframes.pandas as bpd# Configure BigQuery DataFrames global optionsbpd.options.bigquery.project = PROJECT_IDbpd.options.bigquery.location = "US" # Or your dataset location# Load table reference (zero data movement)users_df = bpd.read_gbq("bigquery-public-data.thelook_ecommerce.users")```## 5. Verification Checklist- [ ] **No GCS Export**: Verify the code does not contain `bq extract` or GCS export steps before loading.- [ ] **Spark Format**: Ensure the Spark read format is explicitly `"bigquery"`.- [ ] **BigFrames Config**: Ensure `bpd.options.bigquery.project` is set before calling `read_gbq`.

Frequently asked questions

Zero Copy Ingestion