modbender Scrask Bot
Overview
This skill activates when the user sends a screenshot via Telegram. It uses vision AI to extract actionable information from the image, then:
- High confidence (≥ 0.75): Saves immediately and replies with a brief confirmation.
- Low confidence (< 0.75): Shows a structured preview in Telegram and asks for confirmation before saving.
Provider behaviour (auto mode, default):
| Step | What happens |
|---|---|
| 1 | Gemini 2.0 Flash parses the screenshot (fast, cheap) |
| 2 | If any item confidence < 0.60, Claude Opus reruns the parse |
| 3 | Whichever provider scores higher average confidence wins |
| 4 | Output includes provider, fallback_triggered, and confidence delta |
Set vision_provider to "claude" or "gemini" to lock a specific provider.
Output destinations (AI-decided by content type):
| Detected type | Destination |
|---|---|
| Event (has date+time, venue, or invite link) | Google Calendar |
| Reminder (deadline, due date, personal action) | Google Tasks (with due date) |
| Task (no date, pure action item) | Google Tasks (no due date) |
Trigger Conditions
Activate when:
- The user sends a message in Telegram that contains an image attachment
- The image appears to be a screenshot — not a photo of a person, place, or physical object
- No other skill has already claimed the image
Do not activate for:
- Photos of people, places, food, scenery
- Screenshots of code, errors, or UI bugs (leave for other skills)
- Images the user explicitly asks to edit, describe, or analyze for another purpose
Step-by-Step Instructions
Step 1: Acknowledge Immediately
Reply in Telegram right away so the user knows the skill is working:
"📸 Got it — analyzing your screenshot..."
Do not make the user wait silently.
Step 2: Run the Parser
python3 ~/.openclaw/skills/scrask-bot/scripts/scrask_bot.py \
--image-path "<path-to-temp-image>" \
--provider "$CONFIG_VISION_PROVIDER" \
--timezone "$CONFIG_TIMEZONE" \
--google-credentials "$GOOGLE_CREDENTIALS"
The script auto-resolves the API key from ANTHROPIC_API_KEY or GEMINI_API_KEY
depending on the provider — no need to pass it explicitly.
The script returns a JSON object with:
success— whether parsing workedno_actionable_content— true if nothing foundresults[]— one entry per detected item, each withconfidence,type,destination,needs_confirmation,action_takentelegram_reply— the pre-formatted message to send back to the user
Step 3: Handle the Output
If no_actionable_content is true:
Reply: "🤷 I couldn't find any event, reminder, or task info in that screenshot. Could you describe what you'd like to add?"
If success is true:
Send the telegram_reply value directly back to the user in Telegram. The script has already:
- Saved high-confidence items silently
- Formatted confirmation prompts for low-confidence items
Do not rephrase or reformat the telegram_reply — send it as-is.
Step 4: Handle Confirmation Responses
If the script returned items with needs_confirmation: true, wait for the user's reply.
"yes" or "save" or "add":
Re-run the script for that specific item with confirmed=true, or use the calendar_create / tasks_create tools directly with the extracted fields.
"edit": Ask what to change, update the relevant field, then save.
"skip" or "no": Reply: "Got it, skipped ✓"
Step 5: Confirm Saves
For items saved silently (high confidence), the telegram_reply from the script already contains the confirmation message. Examples of what the user will see:
📅 Added to Calendar: **Team Standup** — 2026-03-01 at 09:00🔔 Added to Tasks: **Pay electricity bill** (due 2026-02-28)✅ Added to Tasks: **Review PR for Arjun**
Edge Cases
| Scenario | Behavior |
|---|---|
| Screenshot is in Hindi, Tamil, or another language | Extract and translate silently; save title in English |
| Recurring event ("every Monday") | Set RRULE on the calendar event; mention it in the reply |
| Date has already passed | Flag in the reply: "⚠️ This date has already passed (Feb 10). Save anyway?" |
| Multiple items in one screenshot | Process each independently; confirm per item if needed |
| Screenshot of someone's calendar | Detect already_in_calendar_hint; reply: "Looks like this event is already in your calendar 🗓️" |
| Google API auth failure | Reply with the specific error and suggest re-checking GOOGLE_CREDENTIALS |
| Zoom/Meet link found | Add to Calendar as both location and description |
Configuration
{
"skills": {
"entries": {
"scrask-bot": {
"enabled": true,
"env": {
"GEMINI_API_KEY": "AIza-your-gemini-key",
"ANTHROPIC_API_KEY": "sk-ant-your-key-here",
"GOOGLE_CREDENTIALS": "/home/user/.openclaw/google-creds.json"
},
"config": {
"vision_provider": "auto",
"fallback_threshold": 0.60,
"timezone": "Asia/Kolkata",
"confidence_threshold": 0.75,
"reminder_minutes_before": 30
}
}
}
}
}
Permissions Required
image:read— to access the screenshot from Telegramnetwork:outbound— to call Anthropic API and Google APIstelegram:reply— to send confirmation messages back to the user