Mandate 2.4.2 Code Evaluation Skill
Mandate
- ID: 2.4.2
- Title: Secure Output Handling
Mitigates
- ML09 Output Integrity Attack
- LLM05 Improper Output Handling
- ASI05 Unexpected Code Execution (RCE)
- MCP01 Tool Poisoning and Rug Pull Attack
Inputs
- Agent/model output handling code
- Downstream adapters (DB, shell, HTTP, template, file)
- Validation/sanitization libraries
- Security tests
Workflow
- Identify untrusted output sources.
- Model completions, tool-generated text, RAG responses, chain-of-thought-derived instructions.
- Identify sensitive sinks.
- SQL queries, shell/command execution, HTML rendering, template rendering, deserialization, file paths, dynamic code execution.
- Build source-to-sink dataflow.
- Trace where model output reaches each sink directly or indirectly.
- Validate context-specific protections.
- SQL: parameterized queries.
- HTML/UI: context-aware escaping.
- Shell: allowlisted commands and argument escaping.
- File paths: canonicalization and path traversal checks.
- Validate strict schema checks.
- Confirm output must match expected structured schema before action.
- Detect unsafe shortcuts.
- Flag direct string interpolation into sinks and bypass flags that skip validation.
- Verify negative tests.
- Confirm tests include malicious payloads and assert blocked/sanitized behavior.
- Emit findings with source, sink, missing control, and exploit path.
Decision Rules
- Pass if all untrusted outputs are validated/sanitized/encoded before every sensitive sink.
- Fail if any sink receives untrusted output without required controls.
Severity Rules
- Critical: untrusted output reaches shell/code/SQL execution unsafely.
- High: unsafe UI rendering or template injection path.
- Medium: sanitization exists but inconsistent coverage.
Output Template
{
"mandate_id": "2.4.2",
"status": "fail",
"severity": "critical",
"vulnerability_tags": ["ML09", "LLM05", "ASI05", "MCP01"],
"evidence": [
{"file": "src/exec/runCommand.ts", "line": 31, "detail": "Model output passed directly to shell command"},
{"file": "src/db/queryBuilder.ts", "line": 54, "detail": "String interpolation used in SQL query"}
],
"remediation": "Add strict schema validation and context-safe sink handling before execution/rendering."
}
Guardrails
- Use code and repo configuration evidence only.
- Always include file-level evidence for each finding.
- If required evidence is missing in code, mark control as not implemented.
1---2name: mandate-2-4-2-code-evaluation3description: Evaluate compliance for Mandate 2.4.2 (Secure Output Handling) using repository code and configuration analysis. Use when producing deterministic pass/fail findings with severity, mapped mitigated vulnerabilities, and file-level evidence.4---56# Mandate 2.4.2 Code Evaluation Skill78## Mandate9- ID: 2.4.210- Title: Secure Output Handling1112## Mitigates13- ML09 Output Integrity Attack14- LLM05 Improper Output Handling15- ASI05 Unexpected Code Execution (RCE)16- MCP01 Tool Poisoning and Rug Pull Attack1718## Inputs19- Agent/model output handling code20- Downstream adapters (DB, shell, HTTP, template, file)21- Validation/sanitization libraries22- Security tests2324## Workflow251. Identify untrusted output sources.26 - Model completions, tool-generated text, RAG responses, chain-of-thought-derived instructions.272. Identify sensitive sinks.28 - SQL queries, shell/command execution, HTML rendering, template rendering, deserialization, file paths, dynamic code execution.293. Build source-to-sink dataflow.30 - Trace where model output reaches each sink directly or indirectly.314. Validate context-specific protections.32 - SQL: parameterized queries.33 - HTML/UI: context-aware escaping.34 - Shell: allowlisted commands and argument escaping.35 - File paths: canonicalization and path traversal checks.365. Validate strict schema checks.37 - Confirm output must match expected structured schema before action.386. Detect unsafe shortcuts.39 - Flag direct string interpolation into sinks and bypass flags that skip validation.407. Verify negative tests.41 - Confirm tests include malicious payloads and assert blocked/sanitized behavior.428. Emit findings with source, sink, missing control, and exploit path.4344## Decision Rules45- Pass if all untrusted outputs are validated/sanitized/encoded before every sensitive sink.46- Fail if any sink receives untrusted output without required controls.4748## Severity Rules49- Critical: untrusted output reaches shell/code/SQL execution unsafely.50- High: unsafe UI rendering or template injection path.51- Medium: sanitization exists but inconsistent coverage.5253## Output Template54```json55{56 "mandate_id": "2.4.2",57 "status": "fail",58 "severity": "critical",59 "vulnerability_tags": ["ML09", "LLM05", "ASI05", "MCP01"],60 "evidence": [61 {"file": "src/exec/runCommand.ts", "line": 31, "detail": "Model output passed directly to shell command"},62 {"file": "src/db/queryBuilder.ts", "line": 54, "detail": "String interpolation used in SQL query"}63 ],64 "remediation": "Add strict schema validation and context-safe sink handling before execution/rendering."65}66```6768## Guardrails69- Use code and repo configuration evidence only.70- Always include file-level evidence for each finding.71- If required evidence is missing in code, mark control as not implemented.
jefflyt