danube-messaging Scenario: Subscription Patterns
Objective
Test the two fundamental messaging patterns in Danube — pub-sub fan-out (broadcast) and queue work distribution (load balancing) — and verify correct message routing behavior.
When to Use
- User wants to understand the difference between Exclusive and Shared subscriptions
- User wants to test "fan-out", "broadcast", "pub-sub", "round-robin", "load balance", "queue"
- User wants to verify message distribution across multiple consumers
- User wants to test consumer churn (join/leave during traffic)
Compatible Infrastructure
| Setup Method | Standalone | Cluster | Notes |
|---|---|---|---|
| Local Binary | ✅ | ✅ | |
| Local Source | ✅ | ✅ | |
| Docker Compose | ✅ | ✅ | |
| Kubernetes | — | ✅ | Use port-forwarded addresses |
All tests work on standalone. Consumer churn tests benefit from but don't require a cluster.
AI Decision Flow
1. Which pattern to test?
Present these options to the user exactly as listed:
Fan-out (Exclusive): Multiple consumers each create their own unique Exclusive subscription on the same topic. Every consumer receives ALL messages. Verify that each consumer gets the complete message set.
Queue (Shared): Multiple consumers share the same Shared subscription on a topic. Messages are distributed round-robin — each message goes to exactly one consumer. Verify even distribution and no duplicates.
Consumer Churn: Consumers join and leave mid-traffic on a Shared subscription. Verify no messages are lost during consumer transitions and remaining consumers pick up the load. Tests subscription resilience under dynamic membership.
Each aspect maps to the corresponding Step 2x in Execution Steps below.
2. Tool or Client Language?
| User says | Choice |
|---|---|
| "cli", "command line" | danube-cli — works for fan-out and basic queue |
| "python", "rust", "go", "java" | Client library — read the appropriate clients/<lang>/SKILL.md |
| (unclear) | Default: danube-cli for fan-out/queue, Python for churn tests |
Note: Consumer churn tests require a client library (CLI can't dynamically join/leave).
3. How many consumers?
| User says | Count |
|---|---|
| "2", "pair" | 2 consumers |
| "3", "standard" | 3 consumers (default) |
| "more", "5", "scale" | 5 consumers |
| (unclear) | Default: 3 |
4. How many messages?
| User says | Count |
|---|---|
| "few", "quick" | 12 messages |
| "medium", "standard" | 36 messages (default — divisible by 3 for even split) |
| "many", "stress", "100" | 100 messages |
| (unclear) | Default: 36 |
Execution Steps
Step 1: Create the Topic
danube-admin topics create /default/pattern-test
Step 2a: Fan-out (Exclusive) (if selected)
Each consumer uses a unique subscription name with SubType::Exclusive. Each consumer receives ALL messages.
Using danube-cli
# Terminal 1: Consumer A (unique subscription)
danube-cli consume -s http://localhost:6650 -t /default/pattern-test \
-m fanout-sub-1 --sub-type exclusive
# Terminal 2: Consumer B (unique subscription)
danube-cli consume -s http://localhost:6650 -t /default/pattern-test \
-m fanout-sub-2 --sub-type exclusive
# Terminal 3: Consumer C (unique subscription)
danube-cli consume -s http://localhost:6650 -t /default/pattern-test \
-m fanout-sub-3 --sub-type exclusive
# Terminal 4: Producer
danube-cli produce -s http://localhost:6650 -t /default/pattern-test \
-m "broadcast message" --count 36
Using Client Libraries
Generate a script that:
- Creates 3 consumers with unique subscription names +
Exclusivetype - Creates 1 producer
- Sends N messages
- Collects per-consumer message counts
- Asserts: each consumer received ALL N messages
Step 2b: Queue (Shared) (if selected)
All consumers use the same subscription name with SubType::Shared. Messages are distributed round-robin.
Using danube-cli
# Terminal 1-3: All use same subscription name
danube-cli consume -s http://localhost:6650 -t /default/pattern-test \
-m queue-sub --sub-type shared
# Terminal 4: Producer
danube-cli produce -s http://localhost:6650 -t /default/pattern-test \
-m "work item" --count 36
Using Client Libraries
Generate a script that:
- Creates 3 consumers with the SAME subscription name +
Sharedtype - Creates 1 producer
- Sends N messages
- Collects per-consumer message counts
- Asserts: total received = N, each consumer received approximately N/3
Step 2c: Consumer Churn (if selected)
Requires client library. Tests consumers joining mid-traffic:
- Start with 2 consumers on a Shared subscription
- Send first batch of messages
- A 3rd consumer joins
- Send second batch of messages
- Verify: all messages delivered exactly once, no duplicates, all 3 consumers got messages after the join
Verification
| Test | Pass Criteria |
|---|---|
| Fan-out | Each consumer received ALL N messages (N × consumers total deliveries) |
| Queue | Total received = N, each consumer received approximately N/3 (± 1) |
| Consumer Churn | Total unique messages = N, no duplicates, all consumers got some messages |
# Check topic state
danube-admin topics describe /default/pattern-test
danube-admin topics subscriptions /default/pattern-test
Cleanup
This scenario only cleans up topics it created. See setups/SKILL.md → Cleanup for cluster teardown.
danube-admin topics delete /default/pattern-test