Ortho Multilingual Data Scout

Orthopaedics Multilingual Data Scout

Use this skill when the user asks to find existing research data, cohorts, registries, or previously collected measurements relevant to an orthopaedic research question.

Hard limits for the PoC

• Public web information only. Do not authenticate into subscription databases.
• Search at most 2 rounds.
• Do not use English as a gatekeeper for other languages.
• Do not exclude by publication year or sample size unless the user explicitly asks.
• Grey literature is allowed.
• Detailed inspection of full text / supplements / variable dictionaries: maximum 5 candidates per user query.
• Verifying an availability class is not detailed inspection and is not rationed. Opening a repository record, registry page or data availability statement to establish A/B/C and the access route is required for every candidate you classify, and does not consume the budget of 5.
• Do not download or statistically analyze raw participant-level data in this PoC.
• Human language review happens after automated search, not during query generation.
• Never claim that a dataset is available merely because a paper exists.
• Never claim that a source was searched if only a generic web result mentioned it.

Inputs

The user provides a natural-language research question.

Read:

• config/languages.json
• config/sources.json
• docs/DESIGN.md

Workflow

1. Interpret the question

Follow prompts/interpret_query.md.

Create runs/<run_id>/query.json using schemas/query.schema.json.

Extract:

• condition/procedure
• exposure/phenotype
• outcome/variable
• population
• desirable measurements
• desired data characteristics

Separate concepts into:

• core
• desirable
• screening_only

Ask the user only if there is a major ambiguity that would change the research question. Otherwise search broadly.

2. Generate language queries

Follow prompts/expand_terms.md.

For every enabled language:

• preserve the canonical meaning;
• generate natural academic/medical terminology;
• add common synonyms, abbreviations, spelling variants and instrument names when known;
• do not translate proper names or validated scale acronyms unnecessarily.

Write the generated queries to runs/<run_id>/queries.json using schemas/queries.schema.json.

Set confidence per language. Anything other than established is a flag for the human language reviewers, so mark literal translations honestly rather than presenting them as settled terminology.

3. Round 1 search

Search each enabled language independently.

Use:

1. generic web search;
2. source-restricted web search using relevant source_hints;
3. public native search pages when directly usable without authentication.

Global sources and local-language sources should both be searched. English results must not be required before local-language searches begin.

For every executed search append one JSON object to: runs/<run_id>/search_events.jsonl

Use schemas/search_event.schema.json.

Record the exact query and actual search route. Do not record a database as searched unless its own search or a source-restricted query was actually used.

4. Harvest expansion terms

From Round 1 results extract:

• local medical terms
• measurement names
• questionnaires/scales
• cohort/registry/study names
• study acronyms
• authors/institutions
• dataset DOI/accession
• variables actually measured

Preserve where each term came from.

5. Round 2 search

Share useful harvested terms across language adapters without changing the canonical question.

Run one additional search round only. Log it exactly as Round 1.

Anything that would require a third round becomes an unexplored_lead.

6. Build candidate entities

Create or update runs/<run_id>/candidates.jsonl using schemas/candidate.schema.json.

Candidate classes:

• A: public data files
• B: application/request route is documented
• C: data collection is evidenced, but current availability is unknown
• lead: data collection not yet confirmed

Keep Study, Report, Dataset, Cohort/Registry and Lead conceptually distinct.

Record evidence reports as their own entities in runs/<run_id>/reports.jsonl using schemas/report.schema.json, and link them from candidates with evidence_report_ids. The link must be traversable both ways: a report cited by a candidate must list that candidate in its candidate_ids. A report that evidences two candidates is stored once, not duplicated.

Merge multiple reports from the same underlying dataset/cohort unless they represent materially different subcohorts, waves, releases, observation periods, variable collections or inclusion criteria.

Every candidate must contain discovery provenance linking back to search event IDs.

7. Select up to 5 candidates for detailed inspection

Follow prompts/inspect_candidate.md.

First, for every candidate you intend to classify, verify its availability class from public material and set availability_verified. Class A may only be asserted when the repository record was actually opened. This step is unrationed.

Then choose at most 5 candidates for detailed inspection. Prefer:

1. research-question fit;
2. required-variable match;
3. population/timepoint match;
4. if otherwise similar, diversity of discovery language/source.

Do not prefer open data merely because it is open.

Inspect public full text, supplements and variable dictionaries where available. Mark all others not_reviewed.

8. Assign fit without an overall score

Set research_fit by this rule, applied to the core concepts identified in step 1:

• high — every core concept is verified as actually measured in the same participants, and the outcome's ascertainment method is described, with event counts where applicable.
• medium — some core concepts are verified and the rest are plausible but unverified; or all are present but the design cannot answer the question as asked (no follow-up, zero outcome events, or cross-sectional where the question needs longitudinal data).
• low — only exposure-side or peripheral concepts are evidenced; the outcome is not ascertained.

State the rule and its application in SUMMARY.md so two runs can be compared.

Keep accessibility, N, longitudinal status, availability class and evidence confidence as separate fields. Do not calculate an overall numeric ranking.

9. Output

Create:

• runs/<run_id>/SUMMARY.md
• runs/<run_id>/query.json
• runs/<run_id>/queries.json
• runs/<run_id>/search_events.jsonl
• runs/<run_id>/candidates.jsonl
• runs/<run_id>/reports.jsonl
• runs/<run_id>/unexplored_leads.jsonl when applicable

SUMMARY.md should show:

• the research_fit rule you applied;
• detailed candidates first (max 5);
• additional candidates;
• leads;
• languages/sources searched;
• important coverage gaps;
• explicit distinction between "not found" and "not searched / inaccessible".

Provenance rule

The central invariant is:

research question -> generated query -> search event -> discovered report/entity -> candidate dataset/cohort

and the reverse relation must also be recoverable from candidate metadata.

Validation

Run:

python scripts/validate_run.py runs/<run_id>

Fix validation errors before presenting the result.

For benchmark runs, also run:

python scripts/evaluate_known.py runs/<run_id> evaluation/known_candidates/<benchmark>.json

Report multilingual incremental yield for every run:

python scripts/evaluate_yield.py runs/<run_id>

Quote the headline number together with its breakdown by research_fit. A count of uniquely non-English-discovered entities means little without it.

Do not

• invent URLs, DOIs, sample sizes or variables;
• infer data availability from publication alone;
• count several papers from one cohort as several independent datasets;
• silently discard non-English or grey-literature candidates;
• bypass CAPTCHA, paywalls, robots restrictions or authentication;
• treat zero search hits as proof that no relevant data exist.