Name Framework Migration First Step

Perform Step 1 of the name framework migration in $ARGUMENTS: introduce h.review_names alongside the existing cleaning logic.

Branch setup

Before making any changes:

1. Derive a branch name from the crawler path by taking the dataset name (the directory containing crawler.py) and prefixing it with name-migration/. For example, datasets/us/ga/med_exclusions/crawler.py → name-migration/us-ga-med-exclusions.
2. Create and check out the branch:

   git checkout -b <branch-name>
   

3. Confirm you are on the new branch before proceeding.

Crawler source

!cat $ARGUMENTS

Read first (in order)

• examples/migrations.md — real before/after migrations; read before touching any crawler
• zavod/docs/extract/names.md#migrating-to-the-name-cleaning-helpers — full three-step procedure and rationale
• zavod/zavod/helpers/names.py — exact signatures for review_names, check_names_regularity, Names

Trigger patterns to find

Scan the crawler for any of these before acting:

# Delimiter splits
last_name, name = name_raw.split(",", 1)
name, *aliases = h.multi_split(raw, SPLITS)

# Bracket/parenthesis stripping
name = name.replace("(Acting)", "")
name = name.strip("„")

# Regex substitutions or splits on name content
parts = re.split(r"(?i)\baka\b", name, maxsplit=1)
names = h.multi_split(name_raw, ["(w zapisie także", "(", ")"])

# Conditional checks on name content before apply_name
if len(name_split) > 1:
    entity.add("alias", name_split[1:])

Migration steps

zavod/docs/extract/names.md (the "Step 1" subsections) is the authoritative procedure and holds the exact code for each path — follow it, do not rely on a paraphrase here. This section only orients you and flags what's specific to running it as a skill.

First decide the path by dataset type:

• Sanctions / debarment dataset (e.g. tagged list.sanction or list.debarment): mirror the existing cleaned names into a suggested = h.Names(), then call h.check_names_regularity and h.review_names(..., suggested=..., is_irregular=..., default_accepted=True).
• Non-sanctions dataset: leave the existing logic untouched and add h.review_names(context, entity, original=original, llm_cleaning=True) — no suggested, no is_irregular. Then reconcile any custom alias-marker list (e.g. a NAME_SPLITS constant) against rigour.names.name_split_phrases_list() and add uncovered phrases as reject_strings under names.schema_rules, per the doc.

Invariants for both paths:

• original is the unmodified raw source string, captured before any cleaning.
• Existing entity.add / h.apply_name calls stay in place unchanged and keep driving output; reviews are not applied until Step 3.
• Call h.review_names at most once per entity.

Tell the user, after the edit: for sanctions, check a sample of created reviews and confirm the export has no name changes, and deploy Step 3 ASAP so new entities aren't default-accepted; for non-sanctions, check a sample of the LLM extraction.

After changes

After every edit to the crawler file, run:

uvx ruff check --fix $ARGUMENTS && uvx ruff format $ARGUMENTS

Fix any errors ruff reports before proceeding.

Once all changes are complete and ruff passes, stage the file:

git add $ARGUMENTS

Then output the suggested commit message (do not commit):

[<dataset_slug>] name migration

where <dataset_slug> is derived from the path by stripping datasets/ and /crawler.py and replacing / with _ (e.g. datasets/us/ga/med_exclusions/crawler.py → [us_ga_med_exclusions] name migration).

Do not

• Do not proceed to Step 3 of the three-step migration procedure (switching to apply_reviewed_names) — that requires completed reviews first
• Do not construct Names by guessing field names — read zavod/zavod/helpers/names.py first
• Do not add explanatory comments beyond what the code requires

Source: opensanctions/opensanctions — distributed by TomeVault.