# Expanso Kafka To S3

> Stream Kafka topics to S3 with partitioning and batching

- Skill: `expanso-io/expanso-kafka-to-s3` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add expanso-io/expanso-kafka-to-s3`
- Raw SKILL.md: https://api.skillmd.com/api/skills/expanso-io/expanso-kafka-to-s3/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: expanso-io (https://skillmd.com/u/expanso-io)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/expanso-io/expanso-kafka-to-s3

---


# Kafka to S3

Stream data from Kafka topics to S3 buckets with intelligent partitioning, batching, and compression.

## Category

`data-routing`

## Quick Start

```bash
# Configure environment
export KAFKA_BROKERS=localhost:9092
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
export S3_BUCKET=your-bucket

# Run the pipeline
./run.sh
```

## Pipeline

The `pipeline.yaml` streams from Kafka to S3 with:
- Consumer group management
- Time-based partitioning (hourly/daily)
- Gzip compression
- Batching for efficient S3 writes

## Requirements

- Expanso Edge installed (`clawhub install expanso`)
- Kafka broker access
- AWS credentials with S3 write permissions

## Related

- [Expanso Documentation](https://docs.expanso.io)
- [More Examples](https://examples.expanso.io)