# Cloud Storage

> Cloud Storage

- Skill: `kienbui1995/cloud-storage` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kienbui1995/cloud-storage`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kienbui1995/cloud-storage/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: kienbui1995 (https://skillmd.com/u/kienbui1995)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kienbui1995/cloud-storage

---


# Cloud Storage

## When to Use
- Designing data lake or file storage on GCP
- Setting up lifecycle management to control costs
- Configuring access control for sensitive data
- Preparing for GCP Professional Data Engineer exam

## Core Jobs

### 1. Storage Class Selection
| Class | Use case | Min storage | Retrieval cost |
|-------|---------|-------------|----------------|
| **Standard** | Frequently accessed (hot data) | None | None |
| **Nearline** | Accessed < once/month | 30 days | Yes |
| **Coldline** | Accessed < once/quarter | 90 days | Yes (higher) |
| **Archive** | Accessed < once/year | 365 days | Yes (highest) |

### 2. Lifecycle Rules
- `SetStorageClass` — move to cheaper class after N days
- `Delete` — remove objects after N days or after N versions
- Typical pattern: Standard → Nearline (30d) → Coldline (90d) → Archive (365d) → Delete (730d)
- Rules apply at bucket level; objects evaluated daily

### 3. Access Control
- **Uniform bucket-level access** — all access via IAM only (recommended, disables ACLs)
- **Fine-grained** — legacy; allows per-object ACLs (avoid for new buckets)
- Always use **uniform bucket-level access** for new buckets
- **Signed URLs** — temporary access (15 min to 7 days) without GCP credentials
- **Signed Policy Documents** — control what can be uploaded via HTML forms

### 4. Object Versioning
- Enable to protect against accidental deletes/overwrites
- Each overwrite creates a new version; previous becomes noncurrent
- Use lifecycle rule to delete noncurrent versions after N days (control cost)

### 5. Retention Policies
- **Retention policy** — objects cannot be deleted or replaced until retention period expires
- **Object locks** — WORM (write once, read many) compliance
- Use for regulatory compliance, audit logs

### 6. Data Transfer
| Tool | Best for |
|------|---------|
| `gsutil` / gcloud CLI | Ad-hoc transfers, scripting |
| Storage Transfer Service | Large-scale from S3/Azure/HTTP/on-prem |
| Transfer Appliance | Petabyte-scale offline transfer |
| BigQuery Data Transfer Service | SaaS source → BigQuery (not GCS) |

## Key Concepts
- **Requester Pays** — requesters pay for egress and operations (useful for public datasets)
- **CMEK** — Customer-Managed Encryption Keys via Cloud KMS (regulatory requirement)
- **VPC Service Controls** — restrict GCS access to within a VPC perimeter
- **Object change notifications** — Pub/Sub notifications on object create/delete/update

## Checklist
- [ ] Uniform bucket-level access enabled?
- [ ] Lifecycle rules set to transition cold data to cheaper classes?
- [ ] Object versioning enabled for critical buckets?
- [ ] Pub/Sub notifications configured for event-driven pipelines?
- [ ] Signed URLs used for temporary access (not service account keys shared)?
- [ ] CMEK applied if regulatory requirement exists?

## Output Format
- 🔴 **Critical** — fine-grained ACLs on new bucket (security risk), no versioning on critical data
- 🟡 **Warning** — no lifecycle rules (cost accumulates), public bucket without intention
- 🟢 **Suggestion** — nearline/coldline transition for infrequently accessed data

## Exam Tips
- **Signed URLs** = temporary access without GCP account (not service account keys!)
- Nearline/Coldline/Archive have minimum storage durations — deleting early still charges
- `SetStorageClass` lifecycle → use for cost optimization; `Delete` → use for cleanup
- Storage Transfer Service = move data FROM S3/Azure/on-prem TO GCS
- Requester Pays = data owner doesn't pay egress; useful for public scientific datasets
- CMEK ≠ Google-managed encryption; CMEK = customer controls the key lifecycle in Cloud KMS