Results for “footgun-analysis”

Analyzes game AI systems in a codebase, covering behavior trees, finite state machines, GOAP, utility AI, pathfinding, steering, perception, difficulty adaptation, NPC dialogue, and AI debugging tools for Unity, Unreal, and Godot projects.

Analyzes production code to find gaps in existing tests by reasoning about hypothetical mutations and checking whether tests would catch them.

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

Debug LangChain and LangGraph agents by fetching execution traces from LangSmith. Analyze agent behavior, investigate errors, and review tool calls and performance metrics.

Evaluates automated metrics and vision-language models on identifying granular errors in detailed image descriptions and ranking paired descriptions against human judgments, using macro F1, pairwise accuracy, Spearman rank ρ, and Kendall's τ.

Assess AI/ML systems for prompt injection, jailbreak vulnerabilities, model inversion risk, data poisoning exposure, and agent tool abuse, with MITRE ATLAS mapping and guardrail recommendations.

Fetch and analyze LangSmith execution traces to debug LangChain and LangGraph agents, investigate errors, and review tool calls and performance.

Benchmarks zero-shot in-context learning on BIG-Bench Hard multiple-choice tasks, comparing self-generated demonstrations against direct prompting and chain-of-thought baselines, and reports accuracy.

`analysis-agent`/`task-agent`/`review-agent`: use for changed assets, trust boundaries, reachable abuse paths, impact, or control placement; skip without a security delta.

Use `analysis-agent` to analyze permissions, secrets, sensitive data, trust boundaries, and injection; `task-agent` to implement controls; and `review-agent` to assess evidence. Skip self-review and no-trust-impact work.