Google Cloud Solution Agentic AI Borderless Data Lakehouse
Discovers requirements and designs a borderless open data lakehouse using Lakehouse for Apache Iceberg and BigQuery data agents. Use when architecting multi-cloud storage infrastructure (Cloud Storage, AWS S3, Azure Blob), establishing ingestion and AI serving subsystems, configuring Cross-Cloud Interconnect, or deploying Gemini Enterprise Agent Platform and BigQuery data agents. Don't use for single-cloud data warehouses, or when the focus is on Knowledge Catalog metadata governance and Spark-driven IDE analytics workflows (use google-cloud-solution-agentic-analytics-spark-knowledge-catalog instead).
Follow this workflow to help users design and implement a custom multi-product
solution in the cloud for a given workload, use case, or requirement.
Attribution
MUST READ: Before executing any gcloud, REST API, or MCP command, you
MUST read and follow the instructions in
Usage Attribution.
Product Renaming & Terminology
When generating solution designs, architecture diagrams, and documentation, use
the updated Google Cloud product names. For details on legacy vs. updated
product names and terminology, see
references/product_renaming.md.
Workflow
The solution design and implementation workflow consists of the following
phases:
Phase 1: Requirements discovery and analysis: Analyze the workload's
requirements, constraints, dependencies, and current state.
Phase 2: Solution design: Build a technology stack, architecture, and
deployment configuration for the workload based on Google Cloud design best
practices and recommendations.
Phase 3: Implementation plan: Generate automation
and instructions to deploy the solution.
Phase 4: Solution validation: Validate that the deployment meets the
requirements of the workload.
Phase 1: Requirements discovery and analysis
Step 1: Discover requirements: Understand the functional and
non-functional requirements, business goals, and current state (if any) of the
workload, including its architecture, dependencies, and constraints. Use the
following questions to guide the requirements discovery process:
What are your primary data sources?
How do you manage and federate metadata across your data sources?
What are your security and credential management requirements?
What are the analytical and computational requirements to join and
transform this borderless data?
What types of natural language prompts or user queries do you expect AI
agents or end-users to execute against this data?
Step 2: Identify components: Based on the requirements analysis,
identify the components of the workload and their relationships. Also identify
any borderless components, hybrid components, or on-prem components that the
solution needs to integrate with.
Step 3: Generate component decomposition: Generate a technical
decomposition of the components of the workload.
Step 4: Ask for confirmation: Ask the user to confirm whether the
generated technical decomposition matches their workload requirements.
Step 5: Iterate: If the user requests changes, then generate an
updated technical decomposition, and ask the user to confirm the changes.
Continue iterating until the user confirms the technical decomposition.
Phase 2: Solution design
Step 1: Retrieve relevant Google Cloud documentation: Use available
search or fetch tools to read the content of the following Google Cloud
documentation to ground the guidance that you generate in the remaining
steps of this phase before proceeding.
Important: Use the content that you retrieve from Google Cloud
documentation to ground the guidance that you generate in the remaining
steps of this phase.
Step 2: Map components to Google Cloud products: For each component in
the confirmed technical decomposition, identify the appropriate Google Cloud
products and features, based on the guidelines in
references/product_mapping.md.
Step 3: Create architecture diagram: Create an architecture diagram
that shows the components, their relationships, and data/control flows.
The diagram must show a clear distinction between the products in the
data ingestion subsystem and the serving subsystem.
The diagram must show Managed Service for Apache Spark as a shared
component for ETL/ingestion processing, bridging the data ingestion and
serving subsystems (distinct from interactive IDE analytics workflows).
Step 4: Generate design recommendations: Generate design guidance
based on the guidelines in
references/design_recommendations.md.
Step 5: Draft solution architecture: Compile the requirements,
technical decomposition, product mapping, architecture diagram, and design
recommendations into a single Markdown file named
solution-architecture-guide.md, based on the template in
assets/output-template.md.
Step 6: Request review: Present the generated solution architecture to
the user and request their feedback or approval.
Step 7: Iterate: If the user requests changes, generate an updated
solution architecture and repeat steps 2-6 until the user approves the
solution architecture.
Important: Use these resources as the technical foundation for the IaC and
deployment instructions you generate in the remaining steps of this phase.
Step 2: Identify deployment prerequisites: Document prerequisites for
the deployment, including the following:
Projects and billing associations
Required Google Cloud APIs
Required IAM permissions
Any other prerequisites
Step 3: Generate Infrastructure as Code (IaC): Generate code (e.g.,
Terraform) and deployment scripts to automate the provisioning of the proposed
Google Cloud resources.
Step 4: Write deployment instructions: Draft sequential, step-by-step
deployment instructions to execute the IaC and initialize the workload
components.
Step 5: Request review: Present the generated deployment instructions
to the user for feedback and confirmation.
Step 6: Iterate: If the user requests changes, generate an updated
implementation plan and repeat steps 2-5 until the user approves the
implementation plan.
Phase 4: Solution validation
Step 1: Retrieve relevant verification resources (optional): If the
resources from Phase 3 are not already in your context, retrieve the same
implementation resources as the starting point for the validation checks
and verification scripts that you generate in this phase.
Step 2: Define validation checks: Outline validation steps to verify
that the deployed infrastructure meets the workload requirements:
Deployment dry-run: Commands like terraform plan to preview
changes.
Connectivity and routing: Verification of network paths, load
balancer routing, and service endpoints.
Security policies: Verification of restricted access, firewall
rules, and IAM enforcement.
Step 3: Generate verification scripts: Draft lightweight scripts or
command-line instructions (e.g. using curl or gcloud) that the user can
run to perform these validation checks.
Step 4: Compile validation report: Document the validation steps,
verification scripts, and expected outcomes in a single Markdown file.
Step 5: Conduct validation and finalize: Assist the user in executing
the validation checks and troubleshooting any deployment issues. After the
solution is validated successfully, request final approval from the user.
Step 6: Iterate: If the user requests changes, then generate an
updated validation plan and repeat steps 2-5 until the user approves the
validation plan.
1---2description: Discovers requirements and designs a borderless open data lakehouse using Lakehouse for Apache Iceberg and BigQuery data agents. Use when architecting multi-cloud storage infrastructure (Cloud Storage, AWS S3, Azure Blob), establishing ingestion and AI serving subsystems, configuring Cross-Cloud Interconnect, or deploying Gemini Enterprise Agent Platform and BigQuery data agents. Don't use for single-cloud data warehouses, or when the focus is on Knowledge Catalog metadata governance and Spark-driven IDE analytics workflows (use google-cloud-solution-agentic-analytics-spark-knowledge-catalog instead).3metadata:4 source: https://skillmd.com/skills/h3y6e/google-cloud-solution-agentic-ai-borderless-data-lakehouse5 category: MultiProductSolutions6 github-path: skills/cloud/google-cloud-solution-agentic-ai-borderless-data-lakehouse7 github-ref: refs/heads/main8 github-repo: https://github.com/google/skills9 github-tree-sha: 4b3bf7f1bee954e95cf86a2677dbc5f6606794ba10 version: 1.0.111name: google-cloud-solution-agentic-ai-borderless-data-lakehouse12---1314# Borderless open data lakehouse agentic AI system1516Follow this workflow to help users design and implement a custom multi-product17solution in the cloud for a given workload, use case, or requirement.1819## Attribution2021**MUST READ**: Before executing any `gcloud`, REST API, or MCP command, you22MUST read and follow the instructions in23[Usage Attribution](references/attribution.md).2425## Product Renaming & Terminology2627When generating solution designs, architecture diagrams, and documentation, use28the updated Google Cloud product names. For details on legacy vs. updated29product names and terminology, see30[references/product_renaming.md](references/product_renaming.md).3132## Workflow3334The solution design and implementation workflow consists of the following35phases:3637- **Phase 1: Requirements discovery and analysis**: Analyze the workload's38 requirements, constraints, dependencies, and current state.39- **Phase 2: Solution design**: Build a technology stack, architecture, and40 deployment configuration for the workload based on Google Cloud design best41 practices and recommendations.42- **Phase 3: Implementation plan**: Generate automation43 and instructions to deploy the solution.44- **Phase 4: Solution validation**: Validate that the deployment meets the45 requirements of the workload.4647### Phase 1: Requirements discovery and analysis4849- [ ] **Step 1: Discover requirements**: Understand the functional and50 non-functional requirements, business goals, and current state (if any) of the51 workload, including its architecture, dependencies, and constraints. Use the52 following questions to guide the requirements discovery process:53 - What are your primary data sources?54 - How do you manage and federate metadata across your data sources?55 - What are your security and credential management requirements?56 - What are the analytical and computational requirements to join and57 transform this borderless data?58 - What types of natural language prompts or user queries do you expect AI59 agents or end-users to execute against this data?6061- [ ] **Step 2: Identify components**: Based on the requirements analysis,62 identify the components of the workload and their relationships. Also identify63 any borderless components, hybrid components, or on-prem components that the64 solution needs to integrate with.6566- [ ] **Step 3: Generate component decomposition**: Generate a technical67 decomposition of the components of the workload.6869- [ ] **Step 4: Ask for confirmation**: Ask the user to confirm whether the70 generated technical decomposition matches their workload requirements.7172- [ ] **Step 5: Iterate**: If the user requests changes, then generate an73 updated technical decomposition, and ask the user to confirm the changes.74 Continue iterating until the user confirms the technical decomposition.7576### Phase 2: Solution design7778- [ ] **Step 1: Retrieve relevant Google Cloud documentation**: Use available79 search or fetch tools to read the content of the following Google Cloud80 documentation to ground the guidance that you generate in the remaining81 steps of this phase before proceeding.82 - [Build hybrid and borderless architectures using Google Cloud](https://docs.cloud.google.com/architecture/hybrid-multicloud-patterns/one-page-view.md.txt)83 - [Build a borderless open data lakehouse](https://docs.cloud.google.com/architecture/agentic-ai-build-multicloud-open-data-lakehouse.md.txt)84 - [Implement agentic analytics workflows for distributed data](https://docs.cloud.google.com/architecture/agentic-ai-cross-cloud-analytics.md.txt)85 - [Analytics Hybrid and Multicloud Pattern](https://docs.cloud.google.com/architecture/hybrid-multicloud-patterns-and-practices/analytics-hybrid-multicloud-pattern.md.txt)86 - [Google Cloud multi-regional deployment archetype](https://docs.cloud.google.com/architecture/deployment-archetypes/multiregional.md.txt)87 - [Network segmentation and connectivity for distributed applications in Cross-Cloud Network](https://docs.cloud.google.com/architecture/ccn-distributed-apps-design/connectivity.md.txt)88 - [Patterns for Connecting Other Cloud Service Providers with Google Cloud](https://docs.cloud.google.com/architecture/patterns-for-connecting-other-csps-with-gcp.md.txt)8990 *Important*: Use the content that you retrieve from Google Cloud91 documentation to ground the guidance that you generate in the remaining92 steps of this phase.9394- [ ] **Step 2: Map components to Google Cloud products**: For each component in95 the confirmed technical decomposition, identify the appropriate Google Cloud96 products and features, based on the guidelines in97 [references/product_mapping.md](references/product_mapping.md).9899- [ ] **Step 3: Create architecture diagram**: Create an architecture diagram100 that shows the components, their relationships, and data/control flows.101 - The diagram must be in the Mermaid format:102 https://github.com/mermaid-js/mermaid.103 - The diagram must show a clear distinction between the products in the104 data ingestion subsystem and the serving subsystem.105 - The diagram must show Managed Service for Apache Spark as a shared106 component for ETL/ingestion processing, bridging the data ingestion and107 serving subsystems (distinct from interactive IDE analytics workflows).108109- [ ] **Step 4: Generate design recommendations**: Generate design guidance110 based on the guidelines in111 [references/design_recommendations.md](references/design_recommendations.md).112113- [ ] **Step 5: Draft solution architecture**: Compile the requirements,114 technical decomposition, product mapping, architecture diagram, and design115 recommendations into a single Markdown file named116 `solution-architecture-guide.md`, based on the template in117 [assets/output-template.md](assets/output-template.md).118119- [ ] **Step 6: Request review**: Present the generated solution architecture to120 the user and request their feedback or approval.121122- [ ] **Step 7: Iterate**: If the user requests changes, generate an updated123 solution architecture and repeat steps 2-6 until the user approves the124 solution architecture.125126### Phase 3: Implementation plan127128- [ ] **Step 1: Retrieve relevant implementation resources**:129 - [Build a Multicloud Open Data Lakehouse with Agentic AI](https://codelabs.developers.google.com/next26/multicloud-lakehouse)130 - [Terraform Registry documentation for biglake_iceberg_catalog](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/biglake_iceberg_catalog)131 - [Create an Apache Iceberg table with metadata in Lakehouse runtime catalog](https://docs.cloud.google.com/managed-spark/docs/guides/spark-workloads-with-bigquery-metastore.md.txt)132 - [Accelerate Spark batch workloads and sessions with Lightning Engine](https://docs.cloud.google.com/managed-spark/docs/guides/lightning-engine-serverless.md.txt)133 - [Create data agents](https://docs.cloud.google.com/bigquery/docs/create-data-agents.md.txt)134135 *Important*: Use these resources as the technical foundation for the IaC and136 deployment instructions you generate in the remaining steps of this phase.137138- [ ] **Step 2: Identify deployment prerequisites**: Document prerequisites for139 the deployment, including the following:140 - Projects and billing associations141 - Required Google Cloud APIs142 - Required IAM permissions143 - Any other prerequisites144145- [ ] **Step 3: Generate Infrastructure as Code (IaC)**: Generate code (e.g.,146 Terraform) and deployment scripts to automate the provisioning of the proposed147 Google Cloud resources.148149- [ ] **Step 4: Write deployment instructions**: Draft sequential, step-by-step150 deployment instructions to execute the IaC and initialize the workload151 components.152153- [ ] **Step 5: Request review**: Present the generated deployment instructions154 to the user for feedback and confirmation.155156- [ ] **Step 6: Iterate**: If the user requests changes, generate an updated157 implementation plan and repeat steps 2-5 until the user approves the158 implementation plan.159160### Phase 4: Solution validation161162- [ ] **Step 1: Retrieve relevant verification resources (optional)**: If the163 resources from Phase 3 are not already in your context, retrieve the same164 implementation resources as the starting point for the validation checks165 and verification scripts that you generate in this phase.166167- [ ] **Step 2: Define validation checks**: Outline validation steps to verify168 that the deployed infrastructure meets the workload requirements:169 - **Deployment dry-run**: Commands like `terraform plan` to preview170 changes.171 - **Connectivity and routing**: Verification of network paths, load172 balancer routing, and service endpoints.173 - **Security policies**: Verification of restricted access, firewall174 rules, and IAM enforcement.175176- [ ] **Step 3: Generate verification scripts**: Draft lightweight scripts or177 command-line instructions (e.g. using `curl` or `gcloud`) that the user can178 run to perform these validation checks.179180- [ ] **Step 4: Compile validation report**: Document the validation steps,181 verification scripts, and expected outcomes in a single Markdown file.182183- [ ] **Step 5: Conduct validation and finalize**: Assist the user in executing184 the validation checks and troubleshooting any deployment issues. After the185 solution is validated successfully, request final approval from the user.186187- [ ] **Step 6: Iterate**: If the user requests changes, then generate an188 updated validation plan and repeat steps 2-5 until the user approves the189 validation plan.
Run npx get-skill add h3y6e/google-cloud-solution-agentic-ai-borderless-data-lakehouse in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
Discovers requirements and designs a borderless open data lakehouse using Lakehouse for Apache Iceberg and BigQuery data agents. Use when architecting multi-cloud storage infrastructure (Cloud Storage, AWS S3, Azure Blob), establishing ingestion and AI serving subsystems, configuring Cross-Cloud Interconnect, or deploying Gemini Enterprise Agent Platform and BigQuery data agents. Don't use for single-cloud data warehouses, or when the focus is on Knowledge Catalog metadata governance and Spark-driven IDE analytics workflows (use google-cloud-solution-agentic-analytics-spark-knowledge-catalog instead). It is listed under DevOps & Infra on SkillMD.
This skill has not completed SkillMD's automated safety review yet. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
h3y6e (@h3y6e) published this skill. Their other Agent Skills are listed on their SkillMD profile.