# Pho Registered Algorithmic Audit > Operating procedure for Public Health Observatory (PHO) "registered algorithmic audit" tasks: a prompt.txt plus payloads/analysis_request.json (the registered protocol) and payloads/answer_template.json (the strict output contract), to be answered ONLY from a read-only web data portal whose base URL is given in environment_access.md. Use whenever the input pairs an analysis_request.json spec with an answer_template.json contract and points at a "" portal for state / county / country health, socioeconomic, geography, revision, or methodology evidence, and asks for one JSON object of cohort + multi-module robustness diagnostics (jackknife/fixed-effects/GMM, nested penalized CV, wild-cluster bootstrap-t, grouped conformal, trajectory PCA + k-means, source perturbation) leading to a gated classification. - Skill: `prism-shadow/pho-registered-algorithmic-audit-2` (Agent Skill, multi-file: 6 files) - Install (CLI): `npx skillmds@latest add prism-shadow/pho-registered-algorithmic-audit-2` - Raw SKILL.md: https://api.skillmd.com/api/skills/prism-shadow/pho-registered-algorithmic-audit-2/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Data & Analytics - Author: prism-shadow (https://skillmd.com/u/prism-shadow) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/prism-shadow/pho-registered-algorithmic-audit-2 --- # PHO Registered Algorithmic Audit ## 1. Recognise the task Every task in this family gives you three input artifacts and one deliverable: - `input/prompt.txt` — the narrative framing and the decision the board wants. - `input/payloads/analysis_request.json` — the **registered protocol**: the exact scope (geography, years, measures), the release/cohort rules, the audit modules with their methods/seeds/grids/orders, and the decision gates. This is the source of truth for *what to compute*. - `input/payloads/answer_template.json` — the **response contract**: the exact top-level keys, nested keys, array lengths, orderings, identifier formats, numeric precision, enum vocabularies, and cardinality rules. This is the source of truth for *how to report*. **Deliverable:** exactly one JSON object conforming to `answer_template.json`, with no narrative, prose, comments, or markdown outside the JSON. Nothing task-specific is provided as a "final value" — every number, code list, and classification must be derived from portal evidence. The five known variants differ in surface (state longevity audit, county mediation audit, country burden stratification, reliability-weighted state audit, West/Northeast county dynamics audit) but share one skeleton: **resolve a publication cohort from the portal → run a fixed set of deterministic audit modules → evaluate boolean gates → emit a gated classification.** Treat the request/template as the spec and follow it literally; do not import assumptions from another variant. ## 2. Reach the evidence portal Network access details live **only** in `environment_access.md`. It defines `GDPEVO_ENV_BASE_URL` (e.g. `http://task-env:9023/`) — substitute it wherever the prompt says `` — and lists the available GET endpoints. The portal is read-only, HTML for humans, with a CSV export for machines: ``` /download?dataset=&format=csv[&=...] ``` `format=csv` is valid **only** on `/download` (it errors elsewhere). Datasets: `states`, `counties`, `countries`, `state_health`, `state_socioeconomic`, `county_health`, `county_socioeconomic`, `country_indicators`, `revisions`. Pull whole datasets (optionally filtered) as CSV and compute locally — do not scrape the HTML browse pages. Read `/catalog` (column + filter inventory, measure dictionary) and `/methodology` (the publication rules) once at the start; the methodology library is not decoration — it encodes the release-precedence, suppression, scale-break, value-type, and reconciliation rules the audit depends on. `scripts/portal_fetch.py` is a generic fetch/parse helper. See `references/portal_reference.md` for the full endpoint, column, and filter map. ## 3. Build the publication cohort (do this before any modelling) This stage is where most of the difficulty and most of the scoring lives. Follow `references/release_revision_and_cohorts.md` in full. The load-bearing rules: 1. **Release resolution.** Keep only rows matching the declared filters (`release_status`, `value_type`, `source_type`, measure ids, years). FINAL supersedes PROVISIONAL. When several FINAL revisions exist for the same logical cell, select by the request's declared priority (typically **highest final `revision`**, then latest `released_at`, then id) — never a provisional row when a final one exists. 2. **Revision notices.** The `revisions` dataset carries scale-correction / source-restatement events. Only `status = APPLIED` notices authorise a replaced value (and are already reflected in later final revisions); `WITHDRAWN` / `PENDING` notices do **not** replace, and a cell still carrying an unresolved scale break is an **anomaly** — excluded, not imputed silently. Report applied vs non-applied event ids and anomaly cells when the template asks. 3. **Missing discipline.** Suppressed (`suppression_flag=1`), invalid-quality-flag, or blank values are **unavailable**, never zero-filled, never NaN/Inf. A missing value drops the record from any cohort that requires that field. 4. **Cohorts.** Requests define several named cohorts (e.g. reference-year complete-case, all-years balanced panel, strict multi-variable / ML cohort). Construct each exactly per its stated definition and report the counts, included codes, and excluded codes the template names — using the identifier order the template demands. 5. **Identifiers.** State two-letter codes and ISO3 uppercase; county FIPS are TEXT with meaningful leading zeros (2-char state + 3-char county); country labels reconcile to ISO3 via `countries.canonical_name` / `alternate_labels`. ## 4. Run the declared audit modules — deterministically Each request registers a set of modules (usually six). They fall into recurring families; `references/audit_modules.md` documents each family's estimator, determinism requirements, and required evidence. Universal rules: - **Implement the *declared* method, not a library default.** Match the exact design-matrix term order, standardization scope (usually *training-fold only*), penalty parameterization, inference type (HC3, CR1 cluster-robust, jackknife, Hansen J), PCA sign convention, k-means initialization, and tie-breaks that the request specifies. Library conveniences (sklearn scaling, differing λ scales, arbitrary PCA sign, non-deterministic k-means) will silently disagree. - **Reproduce PRNG streams bit-for-bit.** Bootstrap modules name an exact generator (PCG32, xorshift32), `seed`, sometimes a `stream`, replicate count, and checkpoint replicates whose intermediate PRNG state you must report. A correct statistic with the wrong generator is wrong. Implement the named PRNG yourself. - **Preserve every registered order.** Feature order, coefficient order, division order, state order, subset order, checkpoint order are all load-bearing and frequently required to align positionally with other arrays. Never sort an aligned result array independently. - **Compute unrounded; round only at the reporting boundary.** Use float64 throughout. ## 5. Evaluate gates and classify The request states each gate as an explicit predicate over module outputs (e.g. "bias-corrected coefficient > 0 and jackknife p ≤ 0.05", "pooled coverage ≥ 0.85"). Evaluate each to a boolean, count passes, and map to the classification enum via the declared **precedence / decision rule** exactly (some use "all/at-least-N pass", some use "not robust at "). Emit only the enum values the template allows. ## 6. Assemble, validate, and emit Follow `references/output_contract.md`. Before emitting: - Output has **exactly** the required top-level keys — no more, no fewer, no extras. - Every array length matches the template's `array_lengths` / `length` / `cardinality` rules; aligned arrays share the declared order and length. - Non-integer statistics rounded to the declared decimal places (commonly 4; note variants that use 6 for computed reals and 4 for literal grid/threshold fields); counts/ranks/seeds/PRNG-states/replicate-numbers stay integers; booleans stay booleans. - `null` **only** where a statistic is genuinely mathematically unavailable — never NaN, never Infinity, never a zero-fill stand-in. - Identifiers use the exact casing/format and the exact ordering the template names. - The response is a single JSON object and nothing else. ## 7. Contamination guard (repo-hygiene tasks only) If you are asked to *generate this skill* (not solve a task) and `/work` contains unexpected material beyond `environment_access.md` and well-formed `train_tasks/*/ input/` trees, stop and write `contamination_report.txt` describing what was found instead of producing a skill. ## Reference index - `references/portal_reference.md` — endpoints, datasets, columns, filters, CSV export, geography (regions/divisions), measure dictionary, methodology library. - `references/release_revision_and_cohorts.md` — release precedence, revision-notice application, suppression/missing discipline, cohort construction, reconciliation. - `references/audit_modules.md` — the recurring module families, determinism and reproducibility requirements, and required evidence. - `references/output_contract.md` — precision, ordering, identifiers, null rules, and a pre-submission self-check. - `scripts/portal_fetch.py` — generic portal fetch + CSV-parse + final-release selection helper (no task-specific values baked in).