Assembling an evidence pack for an external body
An information request from a regulator or auditor is not a reporting task. The
difference is the standard: every number has to be reproducible by a stranger, every
definition stated, and every gap disclosed by you rather than discovered by them.
The most damaging outcome is not a bad number. It is a number that cannot be
reproduced, or a gap they find that you did not mention — because both convert a
question about the data into a question about your control environment, which is a much
worse conversation.
Read the request as written
Requests are drafted precisely and read loosely. Before extracting anything:
- Define every term the request uses in your own data, and write down where the
request's language and your system's language differ. "Complaint" in the request may
not equal your
complaint tag. "Customer" may mean account, contact, or legal entity.
This mapping is usually the most contentious part of the pack and it belongs in it
explicitly.
- Pin the period, and the date field. A request for a calendar quarter needs to say
whether membership is by event date, receipt date, or resolution date. Pick, state,
and be consistent across every table.
- Note what is not asked for. Do not volunteer additional analysis into an evidence
pack; it invites scope you were not asked about. Answer the question.
- Identify what you cannot answer, early, and raise it with whoever owns the
relationship. A late-discovered gap is worse than an early-declared one.
- Check who is asking and through whom. Responses to regulators are usually
channelled through compliance or legal. Do not send anything directly, and do not
assume the requester in your inbox is the right recipient.
The standard for every figure
For each number in the pack, record — in the pack, not in a side document:
- The definition, in plain language.
- The population and the exclusions, with counts for each exclusion.
- The source systems, and the extract date.
- The query or method, reproducible.
- The figure, with absolute counts alongside any percentage.
- Known limitations.
Then verify: have a second person reproduce the figure from the written method
alone. If they cannot, the method is not written well enough, and that is far better
to discover now. This step catches more errors than any amount of re-checking by the
original analyst.
Reconcile to the systems of record before submitting
Every figure that also exists elsewhere must tie to it, or the difference must be
explained in the pack:
- Complaint counts against the complaints system.
- Customer and account counts against the CRM or ledger.
- Any financial figure against finance's own numbers.
- Any figure previously reported to the same body, in any prior submission.
An inconsistency with something you have previously said is the finding they will pull
on. Check the prior submissions specifically, and if a number has legitimately
changed, disclose the change and its reason rather than hoping nobody compares.
Disclose the gaps yourself
The section that most improves how a pack is received, and the one most often left out.
Be specific:
- Channels not covered — complaints arriving by post, social, review sites, a
regulator's own portal.
- Records not retrievable — deleted, redacted, outside retention, lost in a migration.
- Periods with a different definition, system or taxonomy, and what that does to
comparability.
- Known data quality issues in the fields you relied on.
- Manual steps in the extraction, and who performed them.
Where a gap materially affects a figure, say what the figure would look like at both
plausible bounds rather than presenting the convenient one.
Handle the material properly
- Evidence must be as-at, not as-of-now. If a status has changed since the period,
present the position at the period end and note the subsequent change separately.
Silently reflecting today's state misrepresents the period.
- Freeze the extract. Take a dated snapshot, keep it, and record its hash or a
version reference. A pack whose underlying data has moved on cannot be defended, and
you will be asked about it months later.
- Keep a working file separate from the submission. Drafts, hypotheses and internal
commentary should not travel with the pack.
- Preserve the chain. Record who extracted what, when, and from where. If the pack's
provenance is questioned, this is the answer.
- Expect follow-up questions months later and expect to have to reproduce the pack
exactly. Store the method and the frozen extract together.
Redaction and minimisation
- Provide what was asked for, at the granularity asked for. Volunteering full
transcripts where a summary was requested expands the disclosure and the customer data
you have shared, and it cannot be undone.
- Where customer-level detail is required, follow the instruction from legal on
redaction and on whether identifiers are needed. Do not decide this yourself.
- Do not include agent names unless specifically requested; where they are, flag it,
because it may engage employment and data-protection considerations.
- Special-category data — health, vulnerability disclosures — needs an explicit
decision before it goes anywhere.
Guardrails
- Do not submit anything directly to an external body. Compliance or legal owns the
channel, the covering language and the timing.
- Do not interpret the regulation, and do not characterise your own compliance.
Present evidence. "We complied" is not an analytical output, and asserting it in a pack
is a specific risk.
- Do not adjust a definition to produce a better figure. If two defensible definitions
give different answers, present the one that matches the request and disclose the other.
- Do not omit an unfavourable figure that was asked for. If a number is bad, it goes in
with its explanation and the action taken.
- Flag anything that looks like a reportable matter discovered during preparation,
immediately and separately. It may carry its own notification clock that starts when
someone is told.
- Do not speculate in writing. Anything in the pack may be read literally and quoted
back.
Present results to the user
- The request, restated, with every term mapped to your data and the mismatches named.
- The pack, figure by figure, each with definition, population, exclusions, source,
method, and absolute counts.
- Reconciliation to systems of record and to prior submissions, with differences
explained.
- The gaps section, written by you, with bounds where a gap moves a figure.
- The frozen extract reference and the provenance record.
- Second-person reproduction result — who reproduced which figures from the method
alone.
- What could not be answered, and what would be needed to answer it.
- Anything requiring a compliance or legal decision before submission: redaction,
agent names, special-category data, and any suspected reportable matter.
1---2name: cx-regulatory-reporting-pack3description: Use to assemble a pack of support evidence for a regulator, auditor or supervisory request, at the standard of reproducibility those audiences apply. Trigger for "the regulator has asked for", "prepare an audit pack", "evidence request from compliance", supervisory information request, s166-style review support, or assembling complaint and QA evidence for an external body.4---56# Assembling an evidence pack for an external body78An information request from a regulator or auditor is not a reporting task. The9difference is the standard: every number has to be reproducible by a stranger, every10definition stated, and every gap disclosed by you rather than discovered by them.1112**The most damaging outcome is not a bad number. It is a number that cannot be13reproduced, or a gap they find that you did not mention** — because both convert a14question about the data into a question about your control environment, which is a much15worse conversation.1617## Read the request as written1819Requests are drafted precisely and read loosely. Before extracting anything:2021- **Define every term the request uses in your own data**, and write down where the22 request's language and your system's language differ. "Complaint" in the request may23 not equal your `complaint` tag. "Customer" may mean account, contact, or legal entity.24 This mapping is usually the most contentious part of the pack and it belongs in it25 explicitly.26- **Pin the period, and the date field.** A request for a calendar quarter needs to say27 whether membership is by event date, receipt date, or resolution date. Pick, state,28 and be consistent across every table.29- **Note what is not asked for.** Do not volunteer additional analysis into an evidence30 pack; it invites scope you were not asked about. Answer the question.31- **Identify what you cannot answer**, early, and raise it with whoever owns the32 relationship. A late-discovered gap is worse than an early-declared one.33- **Check who is asking and through whom.** Responses to regulators are usually34 channelled through compliance or legal. **Do not send anything directly**, and do not35 assume the requester in your inbox is the right recipient.3637## The standard for every figure3839For each number in the pack, record — in the pack, not in a side document:40411. **The definition**, in plain language.422. **The population and the exclusions**, with counts for each exclusion.433. **The source systems**, and the extract date.444. **The query or method**, reproducible.455. **The figure**, with absolute counts alongside any percentage.466. **Known limitations.**4748Then verify: **have a second person reproduce the figure from the written method49alone.** If they cannot, the method is not written well enough, and that is far better50to discover now. This step catches more errors than any amount of re-checking by the51original analyst.5253## Reconcile to the systems of record before submitting5455Every figure that also exists elsewhere must tie to it, or the difference must be56explained in the pack:5758- Complaint counts against the complaints system.59- Customer and account counts against the CRM or ledger.60- Any financial figure against finance's own numbers.61- Any figure previously reported to the same body, in any prior submission.6263**An inconsistency with something you have previously said is the finding they will pull64on.** Check the prior submissions specifically, and if a number has legitimately65changed, disclose the change and its reason rather than hoping nobody compares.6667## Disclose the gaps yourself6869The section that most improves how a pack is received, and the one most often left out.70Be specific:7172- **Channels not covered** — complaints arriving by post, social, review sites, a73 regulator's own portal.74- **Records not retrievable** — deleted, redacted, outside retention, lost in a migration.75- **Periods with a different definition, system or taxonomy**, and what that does to76 comparability.77- **Known data quality issues** in the fields you relied on.78- **Manual steps** in the extraction, and who performed them.7980Where a gap materially affects a figure, say what the figure would look like at both81plausible bounds rather than presenting the convenient one.8283## Handle the material properly8485- **Evidence must be as-at, not as-of-now.** If a status has changed since the period,86 present the position at the period end and note the subsequent change separately.87 Silently reflecting today's state misrepresents the period.88- **Freeze the extract.** Take a dated snapshot, keep it, and record its hash or a89 version reference. A pack whose underlying data has moved on cannot be defended, and90 you will be asked about it months later.91- **Keep a working file separate from the submission.** Drafts, hypotheses and internal92 commentary should not travel with the pack.93- **Preserve the chain.** Record who extracted what, when, and from where. If the pack's94 provenance is questioned, this is the answer.95- **Expect follow-up questions** months later and expect to have to reproduce the pack96 exactly. Store the method and the frozen extract together.9798## Redaction and minimisation99100- **Provide what was asked for, at the granularity asked for.** Volunteering full101 transcripts where a summary was requested expands the disclosure and the customer data102 you have shared, and it cannot be undone.103- **Where customer-level detail is required**, follow the instruction from legal on104 redaction and on whether identifiers are needed. Do not decide this yourself.105- **Do not include agent names** unless specifically requested; where they are, flag it,106 because it may engage employment and data-protection considerations.107- **Special-category data** — health, vulnerability disclosures — needs an explicit108 decision before it goes anywhere.109110## Guardrails111112- **Do not submit anything directly to an external body.** Compliance or legal owns the113 channel, the covering language and the timing.114- **Do not interpret the regulation, and do not characterise your own compliance.**115 Present evidence. "We complied" is not an analytical output, and asserting it in a pack116 is a specific risk.117- **Do not adjust a definition to produce a better figure.** If two defensible definitions118 give different answers, present the one that matches the request and disclose the other.119- **Do not omit an unfavourable figure that was asked for.** If a number is bad, it goes in120 with its explanation and the action taken.121- **Flag anything that looks like a reportable matter** discovered during preparation,122 immediately and separately. It may carry its own notification clock that starts when123 someone is told.124- **Do not speculate in writing.** Anything in the pack may be read literally and quoted125 back.126127## Present results to the user1281291. **The request, restated**, with every term mapped to your data and the mismatches named.1302. **The pack**, figure by figure, each with definition, population, exclusions, source,131 method, and absolute counts.1323. **Reconciliation** to systems of record and to prior submissions, with differences133 explained.1344. **The gaps section**, written by you, with bounds where a gap moves a figure.1355. **The frozen extract reference** and the provenance record.1366. **Second-person reproduction result** — who reproduced which figures from the method137 alone.1387. **What could not be answered**, and what would be needed to answer it.1398. **Anything requiring a compliance or legal decision** before submission: redaction,140 agent names, special-category data, and any suspected reportable matter.