Results for “faithfulness”

Evaluates the faithfulness of abstractive summaries by verifying if factual claims (masked as cloze questions) in the reference summary can be correctly answered using only the generated summary, compared against a gold-standard answer derived from the source context.

Guardian Angel gives AI agents a moral conscience rooted in Thomistic virtue ethics. Rather than relying solely on rule lists, it cultivates stable virtuous dispositions— prudence, justice, fortitude, temperance—that guide every interaction. The foundation is caritas: willing the good of the person you serve. From this flow the cardinal virtues as practical habits of right action and sound judgment. v3.0 introduced virtue-based disposition as the primary evaluation layer, providing deeper coherence than checklists alone. The agent's character becomes the safeguard. v3.1 adds: Plugin enforcement layer with before_tool_call hooks, approval workflows for ambiguous cases, and protections for sensitive infrastructure actions.

Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always

Implement continuous identity verification for zero trust using phishing-resistant MFA (FIDO2/WebAuthn), risk-based conditional access, and identity governance aligned with the CISA Zero Trust Maturity Model.

How to read, resolve, and reason about commitments — durable records of future obligations (your own promises and the operator's requests). Use whenever the `## Upcoming commitments` prompt block surfaces something you might act on, or when you want to inspect what's pending beyond what the block shows.

Slow down interpretation from observation to action. Use when students or adults need to examine assumptions in conflict, dialogue, or inquiry.

Review dependency manifests, lockfiles, and risky install surfaces.

Sets up and audits the Claude Code harness for a project: CLAUDE.md, .claude/rules, skills, subagents, settings.json permissions, hooks, verification loop. Commands: kickoff (scaffold new setup), check-playbook (score an existing one), scouting-report (last scorecard), adjust (fix bloat/misplaced instructions), drill (turn advisory prose into real hooks/permissions/CI), decline (record an accepted risk), gameplan (work order with acceptance criteria before building), watch-film (check a diff against that order for scope creep/weakened tests/false claims). Use for setting up Claude Code, or on: "Claude ignores my CLAUDE.md", "it's huge and still misses things", "it said done but ran nothing", "it changed files I didn't ask about", "it weakened a test to pass", "rule, skill, or hook?", "is my setup any good". Not for code quality (code-audit), test coverage (test-assessment), one-off prompt wording (genie-proof-prompts), new skill authoring (skill-creator), or compacting a conversation (handoff).

Evaluate RAG pipelines with Ragas — measure faithfulness, answer relevancy, context precision/recall, and noise sensitivity using LLM-as-judge metrics; run automated test suite generation with TestsetGenerator; integrate with LangChain, LlamaIndex, and CI pipelines.