How do I install the Iceberg Advanced skill?

Run npx skillmds@latest add 2dmurali/iceberg-advanced in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Iceberg Advanced safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Iceberg Advanced?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Iceberg Advanced free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Iceberg Advanced?

2dmurali (@2dmurali) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Iceberg Advanced by 2dmurali — Agent Skill

Name: Iceberg Advanced
Author: 2dmurali

File contents

Skill: Advanced Iceberg & WAP (Write-Audit-Publish)

Description

Enables the agent to model, implement, and orchestrate the Write-Audit-Publish (WAP) pattern natively using Apache Iceberg branches.

Context

In an Enterprise data lakehouse, writing directly to production tables (main branch) during ETL processes poses a massive risk of data corruption. Apache Iceberg solves this natively with Git-like branching. The agent must implement WAP patterns over traditional staging tables.

Instructions

1. WAP Pattern Flow

Instead of creating _stg tables, instruct the user to use Iceberg branches.

Step 1: Create a Branch Create a branch pointing to the current state of main.

ALTER TABLE prod.sales.transactions CREATE BRANCH etl_job_123;

Step 2: Configure Spark to Write to Branch Spark must be told to write to the specific branch.

# PySpark Example
spark.conf.set("spark.wap.branch", "etl_job_123")

# Now any INSERT or MERGE INTO statement goes to the branch
df.writeTo("prod.sales.transactions").append()

Step 3: Audit / Data Quality Check Read from the branch to verify row counts, nulls, and constraints.

SELECT count(*) FROM prod.sales.transactions VERSION AS OF 'etl_job_123';
-- Trigger Soda or Great Expectations on this branch

Step 4: Publish (Fast-Forward) Once the audit passes, publish the branch back to main.

CALL catalog.system.fast_forward('prod.sales.transactions', 'main', 'etl_job_123');

Step 5: Cleanup

ALTER TABLE prod.sales.transactions DROP BRANCH etl_job_123;

2. Hidden Partitioning

Iceberg uses Hidden Partitioning. If a user tries to partition by year or month by extracting it manually:

❌ Bad: df.withColumn("year", year("ts")).write.partitionBy("year")
✅ Good: CREATE TABLE ... PARTITIONED BY (years(ts))

Explain to the user that Iceberg handles the partition extraction automatically without needing explicit columns.

Output Format: Iceberg WAP Implementation Guide

### 🛡️ Write-Audit-Publish (WAP) Plan: `[Table Name]`

#### 1. Branch Creation
Run: `ALTER TABLE ... CREATE BRANCH ...`

#### 2. Spark Write Configuration
Ensure your SparkSession has: `spark.wap.branch=[branch_name]`

#### 3. Audit Queries
- Run: `SELECT ... VERSION AS OF '[branch_name]'`
- Expected: 0 Nulls.

#### 4. Publish Command
Run: `CALL catalog.system.fast_forward(...)`

---name: iceberg-advanceddescription: Model, implement, and orchestrate the Write-Audit-Publish (WAP) pattern natively using Apache Iceberg branches.---# Skill: Advanced Iceberg & WAP (Write-Audit-Publish)## DescriptionEnables the agent to model, implement, and orchestrate the Write-Audit-Publish (WAP) pattern natively using Apache Iceberg branches. ## ContextIn an Enterprise data lakehouse, writing directly to production tables (`main` branch) during ETL processes poses a massive risk of data corruption. Apache Iceberg solves this natively with Git-like branching. The agent must implement WAP patterns over traditional staging tables.## Instructions### 1. WAP Pattern FlowInstead of creating `_stg` tables, instruct the user to use Iceberg branches.**Step 1: Create a Branch**Create a branch pointing to the current state of `main`.```sqlALTER TABLE prod.sales.transactions CREATE BRANCH etl_job_123;```**Step 2: Configure Spark to Write to Branch**Spark must be told to write to the specific branch.```python# PySpark Examplespark.conf.set("spark.wap.branch", "etl_job_123")# Now any INSERT or MERGE INTO statement goes to the branchdf.writeTo("prod.sales.transactions").append()```**Step 3: Audit / Data Quality Check**Read from the branch to verify row counts, nulls, and constraints.```sqlSELECT count(*) FROM prod.sales.transactions VERSION AS OF 'etl_job_123';-- Trigger Soda or Great Expectations on this branch```**Step 4: Publish (Fast-Forward)**Once the audit passes, publish the branch back to main.```sqlCALL catalog.system.fast_forward('prod.sales.transactions', 'main', 'etl_job_123');```**Step 5: Cleanup**```sqlALTER TABLE prod.sales.transactions DROP BRANCH etl_job_123;```### 2. Hidden PartitioningIceberg uses Hidden Partitioning. If a user tries to partition by `year` or `month` by extracting it manually:- ❌ **Bad:** `df.withColumn("year", year("ts")).write.partitionBy("year")`- ✅ **Good:** `CREATE TABLE ... PARTITIONED BY (years(ts))`Explain to the user that Iceberg handles the partition extraction automatically without needing explicit columns.## Output Format: Iceberg WAP Implementation Guide```markdown### 🛡️ Write-Audit-Publish (WAP) Plan: `[Table Name]`#### 1. Branch CreationRun: `ALTER TABLE ... CREATE BRANCH ...`#### 2. Spark Write ConfigurationEnsure your SparkSession has: `spark.wap.branch=[branch_name]`#### 3. Audit Queries- Run: `SELECT ... VERSION AS OF '[branch_name]'`- Expected: 0 Nulls.#### 4. Publish CommandRun: `CALL catalog.system.fast_forward(...)````

Frequently asked questions

Iceberg Advanced