# Evidence Based Competitor Map > Build or update a verifiable competitive census, for ANY company or sector: file structure, roles, evidence scale, wave-based verification, five batteries of integrity checks, market scoring, deliverables. A universal, fully documented version, with no references to any specific company — the CONFIG block adapts to your own product. Use it for competitive analyses, market maps, competitor censuses, or competitive intelligence whenever you need a verifiable method, not a list compiled by eye. - Skill: `pas54/evidence-based-competitor-map` (Agent Skill, multi-file: 5 files) - Install (CLI): `npx skillmds@latest add pas54/evidence-based-competitor-map` - Raw SKILL.md: https://api.skillmd.com/api/skills/pas54/evidence-based-competitor-map/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: Pas54 (https://skillmd.com/u/pas54) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/pas54/evidence-based-competitor-map --- # Evidence-Based Competitor Map (universal method) A competitive analysis almost always fails the same way: not because companies are missing, but because the file **asserts more than it actually knows**. A third-party mention becomes a fact, a row already "verified" never gets reopened, a number copied into a document quietly goes stale. This method exists to make those errors visible — and it works the same way for a wearable-device maker, a B2B software company, a retail chain, or an industrial supplier: only the content of the cells changes, never the structure. ## The golden rule > **A strong label only holds where someone actually looked, and the proof is the link > to the page they looked at.** Everything else follows from this. --- ## 0. Before you start: what you need to bring This method knows nothing about your market. Before applying it, decide these five things — they are the only points where the skill depends on your company, and they are all collected in one place (the `CONFIG` block of the script, section 5), so the rest of the method stays identical from one census to the next: 1. **Your pillars and sub-criteria** — the product or market characteristics you want to compare across every subject (see §1, "The criteria"). 2. **Your decisive criteria** — one or two sub-criteria that, on their own, decide whether a competitor genuinely affects you (used by check D3, §4 and §5). 3. **Your strategic platform or dependency** — the technology, supplier, or ecosystem your company is exposed to (a cloud platform, a standard, a distribution channel). Used by check D4, the most important one of all (§4). 4. **Your priority markets** — the countries or segments where you want a higher verification standard than elsewhere (used by §4 battery C and the score in §6). 5. **Your exclusion rules** — the categories of subjects you already know you want to exclude (resellers, adjacent markets, etc.) and the textual signal that identifies them, not a list of names (Decision 3, §2). With these five answers in hand, the rest of the skill applies mechanically. Section 9 shows a complete, fictional example of how these five answers translate into a real file. --- ## 1. File structure ### Roles — exclusive, one per row | Role | Definition | |---|---| | **Competitor** | Sells technology or a service to your same type of end customer | | **Market initiative** | Pilot project, consortium, public program, research body. Has no product for sale | | **Potential customer** | The subject buys, not sells: an operator, a service provider, an end distributor, a cooperative | | **Technical supplier** | Sells a component that can be integrated into your product or a competitor's, or distributes and installs it | | **Out of scope** | Adjacent market, or excluded by an explicit rule (see Decision 3) | | **— us —** | Your own row, kept in the file for direct comparison | Two recurring mistakes, in any sector, both of which inflate the counts: **the contracted service provider** (wins a tender and buys technology from a third party → potential customer, not a competitor) and the **distributor/reseller** (resells other companies' products without making them → technical supplier, not a competitor). ### The evidence scale — six states, not three | State | Meaning | |---|---| | `Verified present` | Documented on a primary source, with URL and date | | `Verified absent` | The product pages were read and the feature/characteristic is not there | | `Stated, not verified` | A third-party source says so; no one has opened the subject's own site | | `Not verifiable` | The site cannot be read, or the page doesn't say so clearly | | `Not confirmed` | A later check **disproved** a value we had previously asserted | | `Not surveyed` | The collection grid never asked the question for this row | The distinction between **proven absence** and **unverified** is the only defense against self-confirmation: without it, every criterion not yet checked silently drifts toward "no". `Not surveyed` matters just as much: it separates "we looked and it wasn't there" from "we never asked" — two situations that require different work. This scale applies **per cell**: every pillar, on every row, receives one of these states. It is a different — finer-grained — level than the `Verification level` column used by the script in §5: that one is a **row-level** flag, saying whether the subject's identity and general description have been read from a primary source at least once. A row can have `Verification level = Verified on primary source` and, at the same time, have individual pillars still at `Not surveyed`, because no one has yet looked for that specific characteristic. The two scales answer different questions: "is this subject real, and have we looked at it?" (row) versus "is this specific characteristic confirmed?" (cell). ### The criteria (pillars) Define them yourself, for your own product — this is part 1 of the checklist in §0. Only one structural rule holds, in any sector: all pillars sit **on the same level, no cross-cutting pillar**. A cross-cutting pillar (one that applies "depending on" the others) makes the count non-additive: you can no longer say "how many competitors cover N pillars out of M" if M changes row by row. Every sub-criterion must have a written **discriminating question**, one a verifier can answer yes/no by reading a page — not a judgment call. Two practical rules, valid in any sector: (a) aim for 6-10 pillars with 2-5 sub-criteria each — fewer, and the census discriminates nothing; many more, and no verifier applies them consistently row after row; (b) a good discriminating question names an observable piece of evidence ("does the site state a public per-unit price?"), not a quality judgment ("is the product affordable?") — if two people reading the same page could answer differently, the question needs rewriting. ### Attribution by market served, not by registered office A country (or market segment) is attributed **only with direct proof**: an address, a local legal entity, a dedicated national site, a reseller named for that country. Where the site only says "we operate in Europe" or "present in over N countries", the country is not added. The method therefore **underestimates**, and that is the right direction to err in: a shorter, defensible census beats a larger one inflated by marketing claims. --- ## 2. The eight decisions that hold in any sector 1. **A third-party source cannot prove an absence.** If an article or an industry list doesn't mention a characteristic, that doesn't mean the subject lacks it: it only means that source doesn't talk about it. 2. **A row without a description cannot be classified.** No automatic rule can decide a subject's role or criteria from an empty text field: what it actually sells, and to whom, must be described first. 3. **Exclusion rules apply by evidence, not by name.** If you decide to exclude a category of subjects (for example: generic management tools with nothing specific to your sector, or agencies that resell without developing anything), the rule must look *in the description* for the distinguishing signs of that category **and** the absence of the characteristics that would exclude it — never a hand-written list of company names, which goes stale the moment a new company appears. 4. **A reclassification must be logged with its reason**, so it can be revisited later instead of looking like a mysterious error. 5. **Closed lists must be normalized**: free text collected from research easily produces dozens of variants for the same category or a single country (e.g., "USA", "United States", "U.S." counted as three different countries). 6. **Multiple markets or segments are verified one at a time**, and the check must be *wired into the pipeline* that rebuilds the file — it's easy to do it once and forget to reapply it when the file is refreshed. 7. **Strong labels exist only on rows verified from a primary source.** On an unverified row, `Verified present` must be demoted to `Stated, not verified`, and `Verified absent` to `Not verifiable`: a strong label cannot survive the loss of the verification that supported it. 8. **A verification label is only valid with its link.** A row marked "verified" without the URL of the page that was read carries a label *inherited* from an earlier version of the file: it has decayed. Because every real verification always writes down the link, the absence of a link is exactly what identifies which inherited labels need to be checked. ### Two note columns, never just one - **Evidence notes** — only what was read on the source. This is the text the automatic classification rules use as proof. - **Working notes** — internal annotations only: demotions, duplicates, domain corrections, editorial decisions. If the two are mixed, **a rule starts reading what you yourself wrote**, and the system confirms itself without any real data having changed. Check E4 in §4/§5 exists specifically to catch this. --- ## 3. The work cycle: verification in waves Each wave is a code file (`verified.py`) with, for every subject: `domain → (name, [one value per criterion], proof URL, note)`. Plus an `ALIASES` map for superseded domains, `ROLES` for reclassifications, and `DUPLICATES` for rows that turn out to be the same subject. Advantages, independent of sector: verification is **rerunnable**, every wave is dated, and the pipeline reapplies everything from scratch on every run — there's no hidden state that only one person remembers. ### How to instruct a verifier (agent or human) The prompt template below has held up across dozens of companies in more than one sector. The anti-hallucination rules are not decorative: without them, a verifier (human or agent) produces values that look plausible but are invented. ``` You are a primary-source verifier. You must OPEN the official websites and read what is actually written there. You must NOT invent anything. ANTI-HALLUCINATION RULES (mandatory): - Use search to find the right pages, then read them. If reading is blocked, search the domain first: URLs in the search results become readable. - If a site is unreachable after 3 attempts on different pages, write "UNREACHABLE" and mark everything N. - Do not use prior knowledge or inference: if the page doesn't say it, it's N. - If the domain redirects elsewhere, flag it explicitly. [list of subjects with domain] FOR EACH SUBJECT return ONE row: NAME | actual_domain | | URL of the strongest evidence | note [numbered list of your criteria, with the discriminating question for the ambiguous ones] LETTERS: V = present and documented. A = absent, you checked the product pages and it isn't there. N = not verifiable. In the note (3-5 sentences): what it actually sells, to whom, whether it manufactures or resells, which markets it serves WITH PROOF, named partners or customers, and whether it's a genuine competitor, a technical supplier, a contracted service provider, or out of scope. Output: rows only, no preamble. ``` **Be obsessive about the criterion that truly sets you apart.** An ambiguous criterion, left without a threshold, produces contradictory answers from one verifier to the next. Generic example: if the criterion is "declared native integration with [your strategic platform]", the discriminating question must be explicit — *it only counts if the site explicitly names that integration*; a generic "integrates with the industry's leading platforms", with no name given, counts as `N`, not `V`. Write your own version of this sentence for your most delicate criterion and put it in the prompt: without it, a consistent share of the verifier's answers will get the same point wrong, every time. ### When automated tools aren't enough Some sites respond to any automated fetch with an antibot wall or a 401. They need to be opened with a **real browser**. Never reclassify a row based on indirect clues: three converging clues can all point the wrong way — the convergence of secondary sources doesn't converge toward the truth, it converges toward what **gets said about a company most often outside its own site**. --- ## 4. The five batteries of checks | | What it looks for | It finds | |---|---|---| | **A** form | The cell is filled in as the template expects | incomplete rows | | **B** sense | The row doesn't contradict itself | inconsistent rows | | **C** method | The file as a whole holds together | coverage gaps | | **D** notes | **The rows we think we already know** | **rows that are complete and wrong** | | **E** documents | The deliverables say what the data says | **stale numbers** | A, B, and C look for incomplete rows, and an incomplete file is easy to spot even by eye. **D and E find things that are complete and wrong, invisible precisely because they look finished** — the kind a quick second human review never catches, in any sector. On a real census of a few hundred rows, over three days battery D typically surfaced competitors never seen before, duplicates, corrected domains, and at least one reversed reclassification — on a file that had already been cleaned up twice. Battery E, the first time it's run against a set of documents already thought "finished", almost always finds anomalies introduced by the editorial work itself, not by the underlying data. ### The check that matters most: D4 Looks for **anyone naming your strategic platform or dependency** (point 3 of the checklist in §0) without having the corresponding criterion confirmed. It's the radar pointed at your own infrastructure: if a competitor depends on the same platform, standard, or channel you also rely on, it's a signal that often goes unsearched because it's "taken for granted" that the competitor list is already complete. In a real case, this check revealed that the most relevant competitor in the sector had already been in the file for months, with its decisive box marked "not confirmed" — and therefore, by construction, outside every work queue, simply because no one had ever reopened that row. ### Structural fix: numbers rewrite themselves Wherever a document contains a figure that can be calculated from the data file, **have the figure rewritten by the calculation every time the document is rebuilt**, leaving the hand-written sentence around it untouched. Example: a regex that replaces `N established companies` in the text with the current value computed from the file. This is the exact point where a copied number goes stale, and there's no reason to leave its upkeep to an attentive human — sooner or later it gets forgotten, always in the same spot. --- ## 5. The checks script Save it as `checks.py` next to the data. **It adapts to any census by changing only the `CONFIG` block**: nothing else in the script contains column names or values specific to your sector. Every entry in the `CONFIG` block below is written as a **placeholder to replace**, not a real value — they are the same five points from the checklist in §0, translated into code. ```python #!/usr/bin/env python3 # -*- coding: utf-8 -*- """Integrity checks for a verified competitive census. Works for any sector: the only part to rewrite is the CONFIG block. python3 checks.py # all batteries python3 checks.py D E # only some python3 checks.py --json anomalies.json # output as JSON """ import json import re import sys from collections import defaultdict from urllib.parse import urlparse # ===================================================================== # CONFIG — the ONLY part to rewrite for your own census. # Every value between <> is a placeholder: replace it with your real data. # The rest of the script should not be touched. # ===================================================================== CONFIG = { "data_file": "data.json", # list of dicts, one row per subject "score_file": "score.json", # {market: metrics} — "" to skip check E5 "text_deliverables": ["notes.js", "method.js", "build_score.py"], # your generated documents # --- your data file's column names: change these to match your own template --- "col_name": "Company", "col_website": "Official website", "col_role": "Role in census", "col_verification": "Verification level", "col_proof_url": "Functional proof URL", "col_markets": "Markets served", "col_presence_url": "Presence proof URL", "col_hq": "Registered office", "col_description": "What it sells", "col_evidence_notes": "Merge notes", # evidence from sources ONLY "col_working_notes": "Working notes", # internal annotations ONLY "col_direct": "Provisional direct", "pillar_regex": r"^P\d\.\d ", # how you recognize pillar columns by name # --- status values used in your template: change these if you use different labels --- "role_competitor": "Competitor", "role_us": "— us —", "verified": "Verified on primary source", "state_disproven": "Not confirmed", "state_present": "Verified present", # --- checklist §0 point 2: YOUR decisive criteria --- # WATCH THE ASYMMETRY: check D3 (§4) rereads ALL the criteria in this list. # Check D4 (the platform radar, right below) reads ONLY the first element # of the list, by design. Put your single most decisive criterion first — # the one that on its own separates a real competitor from an innocuous # mention — even if you add others afterward for D3. "decisive_criteria": ["", ""], # --- checklist §0 point 3: YOUR strategic platform or dependency (D4) --- # THE MOST IMPORTANT CHECK IN THE SCRIPT. # Examples of what a "radar" can be: the name of a cloud provider you # depend on, a technical standard, a distribution channel, a hardware # ecosystem. "radar": r"(||)", # The columns to search the radar in: use the ones in your data file that # contain free text (descriptions, notes, references). If a column listed # here doesn't exist in your template, that's fine: the script simply # skips it (this isn't a list of required columns, it's a list of "where # I'd look if it were there"). "radar_columns": ["What it sells", "Merge notes", "Short description", "Named customers and references", "Declared partners"], # --- checklist §0 point 4: YOUR priority markets or segments (battery C, §6) --- "priority_markets": ["", "", ""], # --- E1: every figure you write in your documents, and how to recompute it from the data file --- # Add one entry here for every "citable" number that appears in your deliverables. # Use \d+ (not a fixed number of digits): this works for a census of 6 rows # as well as one of 600 — with \d{3}, a very small or very large census # stops triggering the check without any error flagging it. "figures": { "established rows": ([r"(\d+) established companies"], lambda D, C: len(D)), "competitive scope": ([r"out of (\d+) in the competitive scope"], lambda D, C: len([r for r in D if r.get(C["col_role"]) == C["role_competitor"]])), "verified on primary source": ([r"of (\d+) companies verified on a primary source"], lambda D, C: len([r for r in D if r.get(C["col_verification"]) == C["verified"]])), }, "procedural_phrases": [r"Strong labels demoted to ", r"Verification label decayed", r"DUPLICATE ROW of ", r"Website corrected \("], # --- third-party domains that don't count as a primary source: add your sector's portals --- "third_party_domains": (r"(fair|messe|\bexpo|exhibitor|linkedin|facebook|//x\.com|youtube|" r"crunchbase|wikipedia|zoominfo|europages|kompass|" r"press\.|prnewswire|businesswire|)"), "two_level_suffixes": {"co.uk", "org.uk", "ac.uk", "gov.uk", "com.au", "co.nz", "com.es", "co.at", "or.at", "com.br"}, } # ===================================================================== C = CONFIG D = json.load(open(C["data_file"], encoding="utf-8")) PILLARS = [c for c in D[0] if re.match(C["pillar_regex"], c)] COMPETITORS = [r for r in D if r.get(C["col_role"]) == C["role_competitor"]] VERIFIED = [r for r in D if r.get(C["col_verification"]) == C["verified"]] anomalies, out = [], [] def val(r, key): return (r.get(C[key]) or "").strip() def row_name(r): return val(r, "col_name") def report(code, title, why, rows): """Every anomaly carries the REASON the check exists: a list of codes that nobody knows how to read never gets looked at twice.""" if not rows: out.append("") out.append("[ 0] %s %s — none. ok" % (code, title)) return for x in rows: anomalies.append({"code": code, "check": title, "why": why, **x}) out.append("") out.append("[%4d] %s %s" % (len(rows), code, title)) out.append(" %s" % why) for x in rows[:8]: out.append(" %-40s %s" % (x.get("row", "")[:40], x.get("detail", "")[:104])) if len(rows) > 8: out.append(" ... and %d more" % (len(rows) - 8)) def print_title(t): out.append("") out.append("=" * 78) out.append(t) out.append("=" * 78) def host(u): try: h = urlparse(u if "://" in u else "https://" + u).netloc.lower() except Exception: return "" return h[4:] if h.startswith("www.") else h def root_domain(h): p = h.split(".") if len(p) >= 3 and ".".join(p[-2:]) in C["two_level_suffixes"]: return ".".join(p[-3:]) return ".".join(p[-2:]) if len(p) >= 2 else h def read_text(path): """Resolves \\uXXXX escapes: the comparison must happen on the text that will end up in the document, not on its source-code encoding.""" t = open(path, encoding="utf-8").read() return re.sub(r"\\u([0-9a-fA-F]{4})", lambda m: chr(int(m.group(1), 16)), t) # ===================================================================== # A — FORM # ===================================================================== def battery_A(): print_title("A FORM — the cell is filled in as the template expects") report("A1", "competitor with no market served", "It doesn't enter any per-market analysis: it's invisible to every count.", [{"row": row_name(r), "detail": "no market"} for r in COMPETITORS if not val(r, "col_markets")]) report("A2", "market attributed with no proof URL", "Attribution by market served requires proof: an address, a local legal " "entity, a national site, or a named reseller.", [{"row": row_name(r), "detail": val(r, "col_markets")[:80]} for r in COMPETITORS if val(r, "col_markets") and not val(r, "col_presence_url")]) report("A3", "direct competitor with no official website", "It can't be verified and can't be tied to any per-domain check: it's a row " "that can never leave the queue.", [{"row": row_name(r), "detail": "no website"} for r in COMPETITORS if r.get(C["col_direct"]) == "Yes" and not val(r, "col_website")]) report("A4", "direct competitor with no description", "Without a description you can't decide if it's in scope, and no " "classification rule can work on it.", [{"row": row_name(r), "detail": "no description"} for r in COMPETITORS if r.get(C["col_direct"]) == "Yes" and not val(r, "col_description")]) dups = defaultdict(list) for r in D: k = re.sub(r"[^a-z0-9]", "", row_name(r).lower())[:14] if k: dups[k].append(row_name(r)) report("A5", "two rows with near-identical names", "Possible duplicate that escaped consolidation: it would count twice in " "every per-market count.", [{"row": " | ".join(v), "detail": "very similar names"} for v in dups.values() if len(v) > 1]) # ===================================================================== # B — SENSE # ===================================================================== def battery_B(): print_title("B SENSE — the row doesn't contradict itself") NEGATION = r"(? 0.6: out.append(" %-52s %3d%% not derivable from public sources" % (c[:52], 100 * nv // len(directs))) out.append("") out.append(" Coverage by priority market") for p in C["priority_markets"]: here = [r for r in COMPETITORS if p in val(r, "col_markets")] local = [r for r in here if val(r, "col_hq") == p] vl = [r for r in local if r.get(C["col_verification"]) == C["verified"]] out.append(" %-16s %3d present | %3d with local HQ | %3d local & verified" % (p, len(here), len(local), len(vl))) if local and not vl: out.append(" WARNING: no competitor headquartered in %s is verified." % p) # ===================================================================== # D — THE ROWS WE THINK WE ALREADY KNOW (the battery that matters) # ===================================================================== def battery_D(): print_title("D THE ROWS WE THINK WE ALREADY KNOW") out.append("A, B, and C look for incomplete rows, and an incomplete file is easy to spot.") out.append("This one looks for rows that are COMPLETE AND WRONG, invisible because they look finished.") third_party = re.compile(C["third_party_domains"], re.I) d1 = [] for r in VERIFIED: proof, site = val(r, "col_proof_url"), val(r, "col_website") if not proof: continue hp, hs = root_domain(host(proof)), root_domain(host(site)) if third_party.search(proof): d1.append({"row": row_name(r), "detail": "proof on a third-party domain: " + proof[:70]}) elif hs and hp and hp != hs: d1.append({"row": row_name(r), "detail": "the proof is on %s but the official site is %s" % (hp, hs)}) report("D1", "row verified on a page that isn't its own", "A primary-source verification only counts if the source is the subject's own " "site. If the proof sits on a trade fair, an article, or a partner's site, the " "row is documented but not verified. It can also mean the file is still " "holding a superseded domain.", d1) report("D2", "verified but with no proof URL", "A verification no one else can recheck isn't a verification. Since a real " "verification always writes down the link, a missing link identifies labels " "INHERITED from an earlier version: these are exactly the ones that escape " "every demotion.", [{"row": row_name(r), "detail": "no link to the page that was read"} for r in VERIFIED if not val(r, "col_proof_url")]) d3 = [] for r in D: if val(r, "col_role") == C["role_us"]: continue disproven = [c for c in C["decisive_criteria"] if r.get(c) == C["state_disproven"]] if disproven: d3.append({"row": row_name(r), "detail": "disproven on %s" % ", ".join(s[:4] for s in disproven)}) report("D3", "disproven on a decisive criterion: to be reviewed, not archived", "A \"disproven\" state is the RESULT of a verification, so no queue ever " "reopens it. This is exactly where the competitor you think you've already " "excluded is hiding.", d3) radar = re.compile(C["radar"], re.I) d4 = [] for r in D: if val(r, "col_role") == C["role_us"]: continue text = " ".join((r.get(c) or "") for c in C["radar_columns"] if c in r) m = radar.search(text) if m and r.get(C["decisive_criteria"][0]) != C["state_present"]: d4.append({"row": row_name(r), "detail": "..." + text[max(0, m.start() - 50):m.end() + 70] .replace("\n", " ")}) report("D4", "names your own platform without having the criterion confirmed", "This is the radar pointed at the technology you're exposed to. Anyone who " "names it without the criterion confirmed is either a competitor you haven't " "read yet, or an innocuous mention: either way, it needs a look. This is how " "your biggest competitor goes unnoticed. IF YOU ONLY RUN ONE CHECK, RUN THIS " "ONE.", d4) per_host = defaultdict(list) for r in D: h = root_domain(host(val(r, "col_website"))) if h: per_host[h].append(row_name(r)) report("D5", "multiple rows sharing the same domain", "Two rows on the same site are almost always the same subject counted twice, " "and every per-market count suffers for it.", [{"row": " | ".join(v), "detail": "share " + k} for k, v in per_host.items() if len(v) > 1]) names = {} for r in D: base = re.split(r"[(—]", row_name(r))[0].strip() if len(base) >= 5 and val(r, "col_role") != C["role_us"]: names[base] = row_name(r) OWNERSHIP = re.compile(r"(brand of|part of the group|a brand|subsidiary of|acquired|" r"absorbed by|division of|owned by)", re.I) d6 = [] for r in D: text = " ".join((r.get(c) or "") for c in C["radar_columns"] if c in r) if not OWNERSHIP.search(text): continue for base, full in names.items(): if full != row_name(r) and re.search(r"\b%s\b" % re.escape(base), text): d6.append({"row": row_name(r), "detail": "names \"%s\", already in the census, with a " "corporate-ownership term" % base}) break report("D6", "possible brand or subsidiary of another row", "Corporate ownership is usually stated in the text. Every duplicate that goes " "unfound is a market counted twice.", d6) d7 = [] for r in COMPETITORS: if r.get(C["col_direct"]) != "Yes" or r.get(C["col_verification"]) == C["verified"]: continue markets = [p for p in C["priority_markets"] if p in val(r, "col_markets")] if markets: d7.append({"row": row_name(r), "detail": "not verified in " + ", ".join(markets)}) report("D7", "direct competitor in a priority market never checked on-site", "This is the real work queue, ordered by how much it matters.", d7) # ===================================================================== # E — THE DELIVERABLES SAY WHAT THE DATA SAYS # ===================================================================== def battery_E(): print_title("E DOCUMENTS AGAINST DATA") out.append("The other batteries check the file. This one checks the DOCUMENTS: the numbers") out.append("written into them by hand, which go stale in silence.") docs = {} for p in C["text_deliverables"]: try: docs[p.split("/")[-1]] = read_text(p) except Exception: pass e1 = [] out.append("") for label, (pats, compute) in C["figures"].items(): expected = compute(D, C) found = False for docname, text in docs.items(): for pat in pats: for m in re.finditer(pat, text): found = True written = int(m.group(1)) ok = written == expected out.append(" %-32s expected %4d written %4d %-14s %s" % (label, expected, written, docname, "ok" if ok else "<<< MISMATCH")) if not ok: e1.append({"row": docname, "detail": "%s: expected %d, written %d" % (label, expected, written)}) if not found: out.append(" %-32s expected %4d no occurrence found in the documents" % (label, expected)) report("E1", "stale figure inside a document", "The number in the document no longer matches the file. It needs correcting, " "and where possible it should be rewritten by the calculation instead of by " "hand.", e1) out.append("") out.append(" E2 UNIQUENESS CLAIMS — \"the only\", \"no one\", \"the sole\"") out.append(" These are claims by absence, the most fragile kind there is.") uniq = re.compile(r"[^\"']{0,120}(the only\b|no one\b|the sole\b|none\b|nobody\b)" r"[^\"']{0,150}", re.I) seen = set() for docname, text in docs.items(): for m in uniq.finditer(text): phrase = " ".join(m.group(0).split()) if len(phrase) > 45 and phrase not in seen: seen.add(phrase) out.append(" [%s] %s" % (docname, phrase[:190])) out.append(" -> each one needs to be re-checked against the count behind it.") proc = [re.compile(p) for p in C["procedural_phrases"]] report("E4", "a classification rule may be reading text we wrote ourselves", "If working annotations end up in the column the rules use as evidence, the " "system starts confirming itself, and the flaw is invisible because every " "single step looks correct.", [{"row": row_name(r), "detail": "procedural note found in the evidence column"} for r in D if any(p.search(val(r, "col_evidence_notes")) for p in proc)]) if not C["score_file"]: return try: S = json.load(open(C["score_file"], encoding="utf-8")) except Exception: return S = {m["market"]: m for m in S} if isinstance(S, list) else S e5 = [] for market, m in S.items(): if not m.get("surveyed"): continue here = [r for r in COMPETITORS if market in {p.strip() for p in val(r, "col_markets").split(",")}] if m["surveyed"] != len(here): e5.append({"row": market, "detail": "the score counts %d surveyed, the file has %d" % (m["surveyed"], len(here))}) pc, pr = m.get("pressure_confirmed"), m.get("pressure") if pc is not None and pr is not None and pc > pr + 0.05: e5.append({"row": market, "detail": "confirmed pressure (%.1f) exceeds detected " "pressure (%.1f): impossible, since the first is " "a subset of the second" % (pc, pr)}) for k in ("reliability", "proven_share"): if m.get(k, 0) > 100: e5.append({"row": market, "detail": "%s is above 100 percent" % k}) report("E5", "the score doesn't match the census", "An index calculated on a different base than the file's is a number that " "means nothing, and nobody notices until it's recalculated.", e5) # ===================================================================== BATTERIES = {"A": battery_A, "B": battery_B, "C": battery_C, "D": battery_D, "E": battery_E} if __name__ == "__main__": # Not a plain filter on "--": the filename after --json is not a battery # letter and must be skipped along with the flag, not left in the list. argv, letters, i = sys.argv[1:], [], 0 while i < len(argv): if argv[i] == "--json": i += 2 continue if not argv[i].startswith("--"): letters.append(argv[i].upper()) i += 1 for s in (letters or list(BATTERIES)): if s in BATTERIES: BATTERIES[s]() print_title("SUMMARY") counts = defaultdict(int) for a in anomalies: counts[a["code"]] += 1 for k in sorted(counts): out.append(" %-4s %4d" % (k, counts[k])) out.append(" %-4s %4d total" % ("", len(anomalies))) print("\n".join(out)) if "--json" in sys.argv: dest = sys.argv[sys.argv.index("--json") + 1] json.dump(anomalies, open(dest, "w", encoding="utf-8"), ensure_ascii=False, indent=1) print("\nwritten %s" % dest) ``` --- ## 6. The market score Measures **detected competitive pressure**, not the size of demand: without verified demand data for your sector, don't invent it — a competitive pressure score is defensible even without third-party market data; an invented market-size score never is. ``` pressure = 25·direct_norm + 20·integrated_norm + 15·(criteria_covered/N) + 25 …(truncated)