Results for “confidence-scoring”

Capture confidence ratings before and after a learning attempt to identify overconfidence and underconfidence patterns. Use when a student wants to understand how well they actually know something versus how well they think they know it.

Scores generated image captions against reference captions and source images using the Polos metric, which is trained to align with human judgments and probes hallucination robustness and open-vocabulary evaluation.

Before any explanation or answer, require the learner to produce a free-recall attempt and confidence rating. Use when a student wants help understanding or reviewing a topic — this skill ensures the AI works from what the learner already knows.

Assigns behavioral trust grades (A–F) to MCP servers by running probes for prompt injection, permission overreach, data leaks, and adversarial-input handling, and publishes reproducible onchain attestations.

Sets up and audits the Claude Code harness for a project: CLAUDE.md, .claude/rules, skills, subagents, settings.json permissions, hooks, verification loop. Commands: kickoff (scaffold new setup), check-playbook (score an existing one), scouting-report (last scorecard), adjust (fix bloat/misplaced instructions), drill (turn advisory prose into real hooks/permissions/CI), decline (record an accepted risk), gameplan (work order with acceptance criteria before building), watch-film (check a diff against that order for scope creep/weakened tests/false claims). Use for setting up Claude Code, or on: "Claude ignores my CLAUDE.md", "it's huge and still misses things", "it said done but ran nothing", "it changed files I didn't ask about", "it weakened a test to pass", "rule, skill, or hook?", "is my setup any good". Not for code quality (code-audit), test coverage (test-assessment), one-off prompt wording (genie-proof-prompts), new skill authoring (skill-creator), or compacting a conversation (handoff).

Validates the Emotional Attitude Score (EAS) metric by measuring its consistency with human judgment on word-level sentiment polarity, using the AmbGIMT dataset and pairwise score comparisons.

Validates agent outputs against schemas and quality criteria, scores confidence, detects hallucinations, monitors convergence, decides when to iterate, and synthesizes actionable feedback. Use when checking if a node's output is acceptable, scoring confidence, detecting fabricated content, deciding whether to re-execute, or generating improvement feedback. Activate on "validate output", "check quality", "confidence score", "hallucination check", "should we iterate", "improvement feedback". NOT for executing DAGs (use dag-runtime), planning DAGs (use dag-planner), or matching skills (use dag-skills-matcher).

Correlate findings across OSINT sources—username enumeration, email lookups, social media profiles, domain records, breach databases, and dark-web mentions—into unified intelligence profiles with confidence scoring and link analysis.