Agentic Harness Evidence Loop

Agentic Harness Evidence Loop

Use this skill when a user, Agent Chat prompt, system event, or CLI request asks why an execution/candidate failed, requests diagnostic evidence, or asks for a repair direction backed by local evidence.

Phase Boundary

Phase 6.8A local collection is always allowed without Control Plane.

• Do not call Control Plane write APIs.
• Do not read Control Plane tokens.
• Do not upload, copy, or migrate raw artifacts.
• Do not modify code.
• Do not run Git mutation, CI trigger, deploy, rollback, or repair commands.
• Do not put raw transcripts, raw logs, raw traces, cookies, kubeconfig, Authorization headers, bearer tokens, host absolute paths, or credential-bearing URLs into summary-safe evidence.

Phase 6.8C may perform Control Plane writes only when both conditions are true:

• The request runs in a trusted runtime or service-token context.
• agentic-evidence index register is allowed only when allowed actions explicitly include index_register.
• agentic-evidence storage apply is allowed only when allowed actions explicitly include storage_apply.

Do not route write requests through agent-runtime. Use the Control Plane runtime API directly from the trusted runtime/service-token context.

Do not introduce CONTROL_PLANE_BASE_URL or CONTROL_PLANE_TOKEN as product-level configuration. Use these inputs according to the spec:

• APAAS_CONTROL_PLANE_RUNTIME_API_BASE_URL for the Control Plane runtime API base URL.
• APAAS_RUNTIME_CONTEXT_PATH for the runtime context file when present.
• APAAS_CONTROL_PLANE_TOKEN_PATH for the service-token file path.
• /run/secrets/control-plane-token/token as the default service-token file in runtime deployments.
• APAAS_CONTROL_PLANE_TOKEN only for local/test-only execution, never as product-level configuration.

Missing base URL, missing token, forbidden responses, unavailable services, and payload errors must preserve the local bundle. Do not delete, rewrite, or hide local diagnostic artifacts when write-side registration or storage application fails.

Never print token values, Authorization headers, cookies, kubeconfig, signed storage URLs, or credentialed URLs.

Workflow

1. Build or load a HarnessEvidenceRequest.
2. Validate the request:

agentic-evidence request validate --request <request.json>

3. Classify and collect local evidence:

agentic-evidence collect --request <request.json> --out <staging-root>

4. Validate the bundle:

agentic-evidence bundle validate --bundle <diagnostic-run-dir>

5. Build or refresh the local repair route:

agentic-evidence repair-route --bundle <diagnostic-run-dir>

6. Explain harness-feedback.json and repair-route.json to the user using only summary-safe evidence refs.

Conversation Diagnostic Intake

Do not ask the user to write request JSON. When a user or Agent Chat message describes a failure in natural language, build the HarnessEvidenceRequest yourself from the available conversation, Builder, execution, diagnostic drawer, and workspace context.

Use existing context first:

• Prefer explicit workspace_id, task_id, candidate_id, or execution_id from structured request payloads, selected Builder state, current automation evidence, selected diagnostic run, or Agent Chat message actions.
• If no subject ref is available, Ask at most one concise follow-up question before running collection. Ask for the smallest identifier that can locate the failed subject, such as the task, candidate, execution, phase, or diagnostic run.
• Optional background such as time, page or route, button/action, observed error, recent change, browser/preview URL, and retry count should be summarized into subject_summary or user_message_summary when available.
• Do not block collection waiting for optional background fields when a subject ref is already available.
• Do not infer task, candidate, execution, Control Plane URL, token, storage target, or host path from prose when the context does not provide it.

For conversation_implicit, keep the request local-only unless a trusted structured request explicitly authorizes more:

{
  "trigger_kind": "conversation_implicit",
  "requested_profiles": [],
  "allowed_actions": ["collect_summary", "repair_route", "repair_prompt"]
}

Write-Side Workflow

Only run each write-side command when its specific Phase 6.8C allowed action is present.

Register the diagnostic run index:

agentic-evidence index register --bundle <diagnostic-run-dir>

Apply storage decisions:

agentic-evidence storage apply --bundle <diagnostic-run-dir>

Explain write results from manifest.json, storage-result.json, and summary-safe diagnostics only. Do not expose raw payloads, secrets, credential-bearing URLs, or raw artifact contents.

Trigger Defaults

For conversation_implicit, default allowed actions are:

collect_summary
repair_route
repair_prompt

Only include collect_debug_only when the user or trusted system event explicitly allows debug-only local artifacts. index_register and storage_apply are not active in Phase 6.8A.

RepairRoute Rules

RepairRoute is a next-step recommendation only. It can point to spec_revision_required, bugfix_request_required, confirmed_implementation_followup, agent_chat_inline_candidate, human_triage_required, or no_action, but it never authorizes code changes by itself.

Repair-route output does not authorize code changes, Git mutation, CI trigger, deploy, rollback, or repair commands.