Conversation Flow Validator

Conversation Flow Validator — Dialog Flow Validation Methodology

A skill that enhances dialog quality verification for the dialog-tester and conversation-designer.

Target Agents

• dialog-tester — Systematically validates conversation scenarios
• conversation-designer — Proactively discovers flow defects during the design phase

Conversation Flow Graph Analysis

Node Types

[START] → Entry point
[BOT] → Bot utterance node
[USER] → User input expectation node
[ACTION] → Backend API call
[CONDITION] → Branching condition
[END] → Exit point
[FALLBACK] → Unrecognized input handling
[HANDOFF] → Human agent handoff

Defect Type Detection

Test Scenario Generation Framework

Happy Path (Normal Flow)

Purpose: Validate the most common usage flow

Structure:
1. Greeting > Express intent > Provide slots > Confirm > Complete

Example (Cafe order):
USER: "Hello"
BOT: "Hello! How can I help you?"
USER: "I'd like to order two Americanos"
BOT: "Two Americanos, correct? Please choose a size."
USER: "Tall"
BOT: "Two tall Americanos, total $9.00. Would you like to place the order?"
USER: "Yes"
BOT: "Your order has been placed! Thank you."

Sad Path (Failure Flow)

Purpose: Validate recovery capabilities in error situations

Scenario types:
1. Ordering unavailable item > Suggest similar items
2. Out of stock > Suggest alternatives or direct to another location
3. API failure > Retry or connect to agent
4. Payment failure > Suggest alternative payment methods

Edge Case (Boundary Cases)

1. Intent switch: Suddenly asking "What are your hours?" mid-order
   > Preserve current context, respond, then guide back

2. Multiple intents: "Place an order and send me the receipt too"
   > Sequential processing or compound handling

3. Ambiguous expression: "One more of that"
   > Reference previous context or ask clarifying question

4. Negative confirmation: Repeated questions after "No"
   > Offer alternatives after maximum 2 attempts

5. Empty input / special characters only / emojis only
   > Safe fallback response

Multi-Turn Verification Checklist

[ ] Is context maintained in conversations of 3+ turns?
[ ] Can the user modify previously provided information?
[ ] Is there an alternative path after a "No" response?
[ ] Can the original flow be resumed after an intent switch?
[ ] Is there appropriate guidance after session timeout?
[ ] Is escalation triggered after 3 repetitions of the same question?
[ ] When modifying a slot, are other already-collected slots preserved?

Fallback Strategy Hierarchy

Level 1: Re-input request
  "I'm sorry, I didn't understand that. Could you please say it again?"
  > Allowed once

Level 2: Suggest options
  "Which of the following would you like? 1. Order 2. View menu 3. Other"
  > On 2nd unrecognized input

Level 3: Human agent handoff
  "Let me connect you with an agent for more accurate assistance."
  > On 3rd unrecognized input or user request

Quality Metrics

Validation Report Template

## Conversation Flow Validation Report

### Test Summary
- Happy Path: N/M passed
- Sad Path: N/M passed
- Edge Case: N/M passed

### Defects Found
| # | Type | Scenario | Severity | Remediation |

### Fallback Analysis
- Fallback rate: N%
- Key unrecognized utterances: [list]

### Metrics
| Metric | Result | Threshold | Verdict |