How do I install the Audit skill?

Run npx skillmds@latest add glebis/audit in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Audit safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: executes scripts. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Audit?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Audit free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

glebis (@glebis) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Audit by glebis — Agent Skill

Name: Audit
Rating: 4.2 (5 reviews)
Author: glebis

Audit

Run a corpus-scale, STATS-ONLY PII audit over a folder of session transcripts LOCALLY and produce an aggregate report — counts by type and by layer, the per-session redaction-rate distribution, document lengths, and a coarse residual proxy. Use when the user says "audit my sessions", "scan folder for PII", "how much PII across these transcripts", "PII stats for my corpus", "is my redaction holding at scale", or points at a directory of transcripts and asks how much personal data it contains. Fully local — raw text never leaves the machine; the report carries ZERO PII values, transcript substrings, or filenames (only anonymized own-NN ids and counts), so the aggregates are safe to surface. Run it on a RED (raw) corpus to size the PII, or on a GREEN (already-redacted) corpus to check residual leakage.

glebis f56cb28 2 files · 20.9 KB Updated Sep 9, 2026

File contents

confide:audit — corpus-scale, stats-only PII audit

Measure how much PII lives across a whole folder of sessions, without ever exposing any of it. The audit runs the layered LOCAL detector stack from shared/confide_core.py (regex → Natasha → local LLM) over each file and emits only aggregates. This mirrors the real_session_eval privacy contract: read text only in-process, emit counts.

Privacy invariants (do not violate)

Local-only. No cloud APIs. Raw transcript text never leaves the machine.
Stats-only output. The report (markdown + json + optional HTML) contains ONLY counts and rates — never a transcript substring, never a detected PII value.
No filenames. Per-file rows are keyed by anonymized ids own-00, own-01, … The original path/name is never written or printed. On an unreadable file, only the index + exception class name is recorded.
Safe to surface. Because it is counts-only, the aggregate report can be shared with a cloud agent or pasted into a chat. The PII stays on the machine.

What it reports

n_files, total / mean / min / max document chars
spans_by_type (PERSON, EMAIL, PHONE, DATE, …) and spans_by_layer (regex / natasha / llm)
overall_redaction_rate plus the per-session redaction-rate distribution (min / median / mean / max)
a coarse residual proxy: spans still detectable after redaction — ~0 on a clean RED corpus, a leakage signal on a GREEN corpus.

Run it

Point it at a folder (recurses, processes every .md/.txt; skips confide's own *.green.md / *.stats.json outputs):

python3 skills/audit/scripts/audit.py FOLDER

Options:

--list paths.txt — also/instead audit absolute paths listed one per line.
--layers regex,natasha,llm — choose detection layers (default from config). Use --layers regex for a fully offline, deterministic pass (no models/network).
--out report.md — report path; a report.json sibling is written alongside.
--html — also write a Tufte-ish dashboard (report.html, counts only).

Writes the markdown + json report (and optional HTML) and prints the aggregate summary — all counts only.

RED vs GREEN

RED (raw) corpus: sizes the PII problem before any redaction.
GREEN (redacted) corpus: the residual proxy and remaining spans_by_type tell you whether redaction is holding at scale.

After running

Report the aggregate summary (file count, span totals by type/layer, redaction-rate distribution, residual proxy) — never paste PII.
If residual is non-trivial on a GREEN corpus, point the user at confide:anon to re-redact and confide:red to probe re-identification risk.

Setup

Layer availability (Natasha, local LLM via Ollama) comes from config — run confide:setup if they aren't installed. --layers regex always works offline.

---name: auditdescription: Run a corpus-scale, STATS-ONLY PII audit over a folder of session transcripts LOCALLY and produce an aggregate report — counts by type and by layer, the per-session redaction-rate distribution, document lengths, and a coarse residual proxy. Use when the user says "audit my sessions", "scan folder for PII", "how much PII across these transcripts", "PII stats for my corpus", "is my redaction holding at scale", or points at a directory of transcripts and asks how much personal data it contains. Fully local — raw text never leaves the machine; the report carries ZERO PII values, transcript substrings, or filenames (only anonymized own-NN ids and counts), so the aggregates are safe to surface. Run it on a RED (raw) corpus to size the PII, or on a GREEN (already-redacted) corpus to check residual leakage.--- # confide:audit — corpus-scale, stats-only PII audit Measure how much PII lives across a whole folder of sessions, without ever exposing any of it. The audit runs the layered LOCAL detector stack from `shared/confide_core.py` (regex → Natasha → local LLM) over each file and emits **only aggregates**. This mirrors the `real_session_eval` privacy contract: read text only in-process, emit counts. ## Privacy invariants (do not violate) - **Local-only.** No cloud APIs. Raw transcript text never leaves the machine. - **Stats-only output.** The report (markdown + json + optional HTML) contains ONLY counts and rates — never a transcript substring, never a detected PII value. - **No filenames.** Per-file rows are keyed by anonymized ids `own-00`, `own-01`, … The original path/name is never written or printed. On an unreadable file, only the index + exception class name is recorded. - **Safe to surface.** Because it is counts-only, the aggregate report can be shared with a cloud agent or pasted into a chat. The PII stays on the machine. ## What it reports - `n_files`, total / mean / min / max document chars - `spans_by_type` (PERSON, EMAIL, PHONE, DATE, …) and `spans_by_layer` (regex / natasha / llm) - `overall_redaction_rate` plus the **per-session** redaction-rate distribution (min / median / mean / max) - a **coarse residual proxy**: spans still detectable after redaction — ~0 on a clean RED corpus, a leakage signal on a GREEN corpus. ## Run it Point it at a folder (recurses, processes every `.md`/`.txt`; skips confide's own `*.green.md` / `*.stats.json` outputs): ```bash python3 skills/audit/scripts/audit.py FOLDER ``` Options: - `--list paths.txt` — also/instead audit absolute paths listed one per line. - `--layers regex,natasha,llm` — choose detection layers (default from config). Use `--layers regex` for a fully offline, deterministic pass (no models/network). - `--out report.md` — report path; a `report.json` sibling is written alongside. - `--html` — also write a Tufte-ish dashboard (`report.html`, counts only). Writes the markdown + json report (and optional HTML) and prints the aggregate summary — all counts only. ## RED vs GREEN - **RED (raw) corpus:** sizes the PII problem before any redaction. - **GREEN (redacted) corpus:** the residual proxy and remaining `spans_by_type` tell you whether redaction is holding at scale. ## After running 1. Report the aggregate summary (file count, span totals by type/layer, redaction-rate distribution, residual proxy) — never paste PII. 2. If residual is non-trivial on a GREEN corpus, point the user at **confide:anon** to re-redact and **confide:red** to probe re-identification risk. ## Setup Layer availability (Natasha, local LLM via Ollama) comes from config — run **confide:setup** if they aren't installed. `--layers regex` always works offline.

Frequently asked questions