Ocas Look

Interactive Menu

When invoked interactively, present a two-level menu. See references/interactive-menu.md for the full menu structure.

When to Use

• Image analysis and visual content extraction
• Reverse image search and source identification
• Visual verification of web content
• Screenshot analysis and OCR
• When any skill needs image understanding
• A flyer or poster → calendar event or ticket purchase draft
• A meal photo → macro estimation
• A storefront or sign → save to try-list
• A product photo → pricing comparison or order draft
• A civic issue photo → 311 report draft
• A receipt → expense entry
• A document → searchable PDF and filing draft

When NOT to Use

• Image generation (use Imagine)
• Video processing or analysis
• Bulk image processing at scale
• Real-time camera feed analysis
• Generic OCR utility
• Background surveillance or tracking
• Generic automation framework
• Images where the user has no clear intent in mind

Workflow

The image-to-draft workflow routes user images through intent inference, domain routing, and draft generation. This workflow exists because raw images alone are not actionable — the agent must extract structured meaning before proposing next steps.

1. Ingest — Accept user-provided image (file, URL, or paste)
2. Classify intent — Determine domain (event, meal, place, product, receipt, document, civic)
3. Extract — OCR, vision analysis, reverse image search as needed
4. Generate draft — Produce actionable output (calendar entry, macro estimate, expense log, etc.)
5. Confirm — Present draft to user for approval before acting

Example: a user provides a flyer image → agent classifies as "event" → extracts date/venue → generates calendar draft → confirms with user.

Responsibility boundary

Look owns image-to-action conversion: ingest, context inference, domain routing, draft generation, and execution with confirmation.

Look does not own: web research (Sift), preference persistence (Taste), communications (Dispatch).

Ontology types

Look works with these types from spec-ocas-ontology.md:

• Entity/Person — people identified in images (public figures, named contacts).
• Place — venues, locations, and scenes extracted from image context.
• Concept/Event — events visible in images (gatherings, occasions, dated scenes).
• Thing/DigitalArtifact — the source image itself.

Look includes entity signals in journal payloads using the Signal schema from spec-ocas-shared-schemas.md. The payload.type field must be set to the ontology type of the primary extracted entity (Person, Place, Event, or DigitalArtifact). source_journal_type is "Observation".

Supported domains

Events, food macros, places, products, civic issues, receipts, documents.

Read references/domain_playbooks.md for detailed per-domain behavior.

Core workflow

See references/workflow.md for the full step-by-step procedure, error handling table, and clarification rules.

Commands

• look.ingest.image — ingest image(s) with optional EXIF and device pre-parse
• look.propose.actions — generate ActionDrafts with DecisionRecords
• look.execute.action — execute a confirmed draft (requires explicit approval)
• look.rollback.action — attempt rollback for reversible actions
• look.status — last ingest, pending drafts, items awaiting confirmation
• look.config.set — update configuration
• look.journal — write journal for the current run; called at end of every run
• look.update — pull latest from GitHub source; preserves journals and data
• lookup.reverse_search — perform Google reverse image search on an image URL or local file. Returns matching pages, titles, and similar image URLs.

Confirmation and rollback rules

• Draft-first always. No execution without explicit confirmation.
• High-risk actions (purchases, 311 submission, health writes): require per-draft confirmation token.
• Reversible actions (calendar, maps): expose rollback information.

Permission discipline

Default deny. Request minimally. Drafting continues even without execution permissions. Blocked execution reported, not silently skipped.

Boundaries

• Never invent OCR text, barcodes, prices, or license plates
• EXIF capture location is not the event venue
• iOS relay pre-parse is optional evidence, not truth
• The skill must work without relay upload

Error handling

See references/workflow.md for the full error handling table.

Reverse Image Search

Look uses reverse image search during step 5 (Research and validate externally). Primary method: Yandex Images via browser (works from cloud IPs). Fallback: google-image-source-search PyPI package (residential IPs only).

Quick reference:

• Local file → upload to Imgur first → public URL → Yandex search
• Already a public URL → use directly with Yandex
• Google fallback only on residential IP — never retry more than once from cloud

Detailed instructions: references/google-image-source-search.md (Google fallback), references/credential-files.md (Imgur client ID).

Gotchas (also in Gotchas section):

• Google blocks cloud IPs completely — use Yandex
• TinEye returns JS-rendered results — use browser
• Yandex results may contain Cyrillic tags — URLs still usable

Storage layout

{agent_root}/commons/data/ocas-look/
  config.json
  state.json
  events.jsonl
  decisions.jsonl
  reports/
  artifacts/

{agent_root}/commons/journals/ocas-look/
  YYYY-MM-DD/
    {run_id}.json

Default config: see references/default_config.md.

OKRs

See references/okrs.md for the full OKR definitions. Universal OKRs from spec-ocas-journal.md apply to all runs.

Optional skill cooperation

• Sift — web research for validation during draft generation (via SearchX)
• Chronicle — entity signals emitted via journal payloads after draft generation

Journal outputs

Observation Journal — all image ingestion and draft generation runs.

Initialization

On first invocation of any Look command, run look.init:

1. Create {agent_root}/commons/data/ocas-look/ and subdirectories (reports/, artifacts/)
2. Write default config.json and state.json if absent
3. Create empty JSONL files: events.jsonl, decisions.jsonl
4. Create {agent_root}/commons/journals/ocas-look/
5. Ensure journal payload fields (see interfaces specification) exists (create if missing)
6. Register cron job look:update if not already present (check the platform scheduling registry first)
7. Log initialization as a DecisionRecord in decisions.jsonl
8. Reverse image search setup (run once):
   • Read references/pip-venv.md for the full setup pattern (install google-image-source-search, pin requests, verify import)

Background tasks

# Task declared in SKILL.md frontmatter metadata.{platform}.cron

Self-update

look.update pulls the latest package from the source: URL in this file's frontmatter. Runs silently — no output unless the version changed or an error occurred.

See references/self_update.md for the full self-update procedure.

Visibility

public

Gotchas

• Google reverse image search is blocked from cloud IPs — Use Yandex Images via browser instead. Do not retry Google more than once.
• Vision model config: vision_analyze uses the vision.model and vision.provider fields in config.yaml, NOT the auxiliary model. Set to a vision-capable model like nvidia/nemotron-nano-12b-v2-vl:free (provider: openrouter). The agent cannot patch config.yaml directly — use hermes config set vision.model <model> or edit manually.
• Reasoning model incompatibility: Some free vision models (nex-agi/nex-n2-pro:free, nvidia/nemotron-3-nano-omni) are reasoning-only models that return content: null with the answer in message.reasoning. The vision_analyze tool doesn't handle this. Use nvidia/nemotron-nano-12b-v2-vl:free for reliable vision analysis.
• Wikimedia image URLs blocked: OpenRouter proxy returns 400 when fetching wikimedia.org image URLs. Use base64-encoded images or alternative image hosts.
• LLM7.io has no free vision model: GLM-4.6V-Flash is paywalled. Use OpenRouter free vision models instead.
• EXIF location ≠ event venue — The capture location embedded in a photo's EXIF data is where the photo was taken, not necessarily the event venue.
• Never invent OCR text, barcodes, prices, or license plates — If the image content is ambiguous, state uncertainty rather than guessing.
• TinEye returns JS-rendered results — Use the browser if TinEye is needed; it can't be scraped with curl.
• Reverse image search requires a public URL — Upload local files to Imgur first, then use the returned URL for Yandex reverse search.

Support File Map

Update command

This skill self-updates every 24 hours via:

look.update

This pulls the latest version from GitHub and restarts the skill's background tasks if applicable.