jefflyt Mandate 2.1.4 Code Static Evaluation Skill
Mandate
- ID: 2.1.4
- Title: Sensitive Data Handling in Training Data and System Prompts
Mitigates
- ML03 Model Inversion Attack
- ML04 Membership Inference Attack
- LLM02 Sensitive Information Disclosure
- LLM07 System Prompt Leakage
- ASI03 Identity and Privilege Abuse
- ASI06 Memory and Context Poisoning
Inputs
- Repository root path.
- Tech stack/language map.
- Optional architecture notes if provided.
Workflow
- Scope relevant code surfaces tied to this mandate.
- Run mandate-specific static checks and trace source-to-sink/control paths.
- Extract exact evidence with file/line references.
- Classify each control as implemented, partial, or missing.
- Produce partial-confidence findings and list required runtime/process evidence for closure.
Mandate-Specific Static Method
- Detect training/RAG ingestion pipelines and prompt template locations.
- Verify PII/secret sanitization and redaction functions in ingestion and logging paths.
- Flag prompt files containing sensitive patterns (credentials, secrets, proprietary artifacts).
- Verify logging middleware excludes or masks sensitive fields.
- Detect test fixtures/sample datasets containing unredacted sensitive data.
- Emit findings by data flow stage: ingestion, prompting, storage, logging.
Output Contract
Return a single structured result with:
- mandate_id, mandate_title
- static_status: implemented|partial|missing
- confidence: low|medium|high (static-only confidence)
- evidence: list of file/line findings
- gaps: unresolved items requiring operational proof
- next_evidence_requests: runtime/process artifacts needed
Guardrails
- Avoid final compliance pass/fail using code alone.
- Mark assumptions explicitly.
- Prefer deterministic checks over heuristics when possible.