Human-in-the-Loop
Human-in-the-loop design defines when, where, and how humans intervene in automated workflows. Too little human involvement and the system makes dangerous mistakes. Too much and you've just built an expensive notification system.
Intervention Types
- Approval gates: The system pauses and waits for human approval before proceeding
- Review checkpoints: The system presents results for human review but can continue if no objection
- Correction opportunities: The system shows its work and the human can edit before it's finalised
- Override controls: The human can stop, redirect, or undo the system's actions at any time
- Monitoring dashboards: The human passively observes the system and intervenes only when needed
When to Require Human Intervention
- High stakes: Actions that are expensive, irreversible, or affect many people
- Low confidence: The system is uncertain about the right action
- Novel situations: The input or context is outside the system's training distribution
- Ethical judgments: Decisions that require moral reasoning or value trade-offs
- Legal requirements: Regulatory or compliance requirements mandate human review
- User request: The user explicitly asks for human involvement
Designing Intervention Points
For each intervention point:
- Trigger: What causes the intervention? (confidence threshold, stakes level, user request, policy requirement)
- Presentation: What does the human see? (summary, full context, recommendations, options)
- Time constraint: How quickly must the human respond? What happens if they don't?
- Decision options: What can the human do? (approve, reject, edit, escalate, defer)
- Feedback integration: How does the human's decision feed back into the system?
Avoiding Human Bottlenecks
Human intervention is expensive and slow. Design to minimise unnecessary intervention:
- Graduated autonomy: Start with more human oversight, reduce as the system proves reliable
- Batch review: Group similar decisions for efficient human processing
- Smart routing: Send interventions to the right human based on expertise and availability
- Default actions: If the human doesn't respond within a time window, take a safe default action
- Learning from interventions: Use human decisions to improve the system so it needs less intervention over time
Design Artefacts
- Intervention point map (where in the workflow, what triggers each)
- Intervention interface designs (what the human sees and can do)
- Time constraint specifications
- Graduated autonomy plans
- Intervention analytics specifications (tracking how often, why, and what humans decide)
1---
2name: human-in-the-loop
3description: Designing intervention points where humans review, approve, or redirect agent work.
4---
5
6# Human-in-the-Loop
7Human-in-the-loop design defines when, where, and how humans intervene in automated workflows. Too little human involvement and the system makes dangerous mistakes. Too much and you've just built an expensive notification system.
8## Intervention Types
9- **Approval gates**: The system pauses and waits for human approval before proceeding
10- **Review checkpoints**: The system presents results for human review but can continue if no objection
11- **Correction opportunities**: The system shows its work and the human can edit before it's finalised
12- **Override controls**: The human can stop, redirect, or undo the system's actions at any time
13- **Monitoring dashboards**: The human passively observes the system and intervenes only when needed
14## When to Require Human Intervention
15- **High stakes**: Actions that are expensive, irreversible, or affect many people
16- **Low confidence**: The system is uncertain about the right action
17- **Novel situations**: The input or context is outside the system's training distribution
18- **Ethical judgments**: Decisions that require moral reasoning or value trade-offs
19- **Legal requirements**: Regulatory or compliance requirements mandate human review
20- **User request**: The user explicitly asks for human involvement
21## Designing Intervention Points
22For each intervention point:
23- **Trigger**: What causes the intervention? (confidence threshold, stakes level, user request, policy requirement)
24- **Presentation**: What does the human see? (summary, full context, recommendations, options)
25- **Time constraint**: How quickly must the human respond? What happens if they don't?
26- **Decision options**: What can the human do? (approve, reject, edit, escalate, defer)
27- **Feedback integration**: How does the human's decision feed back into the system?
28## Avoiding Human Bottlenecks
29Human intervention is expensive and slow. Design to minimise unnecessary intervention:
30- **Graduated autonomy**: Start with more human oversight, reduce as the system proves reliable
31- **Batch review**: Group similar decisions for efficient human processing
32- **Smart routing**: Send interventions to the right human based on expertise and availability
33- **Default actions**: If the human doesn't respond within a time window, take a safe default action
34- **Learning from interventions**: Use human decisions to improve the system so it needs less intervention over time
35## Design Artefacts
36- Intervention point map (where in the workflow, what triggers each)
37- Intervention interface designs (what the human sees and can do)
38- Time constraint specifications
39- Graduated autonomy plans
40- Intervention analytics specifications (tracking how often, why, and what humans decide)
gabrielmoreira