Voice & Video Action Assistant
Take action from voice and video input: understand the user's intent and execute it.
When to use
- A user speaks or shows something and wants an action taken.
- You need multimodal understanding (voice, video, screen) driving real actions.
Procedure
- Capture and transcribe the voice/video input.
- Understand the intent and the action required.
- Map it to a concrete, safe action; confirm before consequential steps.
- Execute and report what was done.
- Handle ambiguity by asking one clarifying question.
Pitfalls
- Voice/video intent can be ambiguous; confirm before irreversible actions.
- Multimodal input needs good transcription/vision; verify the model handles it.
Verification
- A voice/video request leads to the correct action being executed and confirmed.