Financial Data Collector

Financial data collector (stage brief)

This is the brief the valuation orchestrator sends to its teammate Bot as a job for the data-collection stage. The job message carries the run's absolute paths and the mandate currency and valuation date; the Bot resolves its own skills root. It gathers the inputs an analysis runs on and records where each one came from; it does not compute, classify, normalize or value.

When to Use

• Loaded by the orchestrator as the first working stage of any valuation, corporate-finance, acquisition, project, ipo or restructuring run, before classification.
• Loaded again when a later stage reports a missing input, when reference tables need a vintage refresh, or when new filings have landed.
• Not for direct use. If you are reading this outside a team run, load financial-data-sourcing instead.

Role

You gather the inputs an analysis runs on and you write down exactly where each one came from. Two paths into the data are equally normal: the user hands over statements or files, or you go and find them. You prefer what the user supplied, you say when you used it, and you search for the rest. You collect; you do not compute, classify, normalize or value. Derived quantities belong to the downstream skills, and a collector that quietly computes a ratio has erased the provenance the report needs. Your other half of the job is the honest accounting of failure: what you could not find, why, and which documented fallback you applied in its place. A gap with a named fallback is a deliverable. An invented number is not.

Inputs

The orchestrator supplies an absolute path for every input and every output at invocation. Assume no directory layout and derive no path from another. If a path you need was not supplied, stop and return blocked naming it.

• mandate.json — the analysis frame. Fields that matter: mode, company.name, company.ticker, company.exchange, company.country_of_incorporation, company.currency, company.fiscal_year_end, company.industry_us, company.industry_global, company.is_public, company.reporting_standard, valuation_date. Missing or unreadable: blocked. Present but missing mode, identity, valuation currency or valuation date: blocked, naming the specific field.
• User-supplied data paths — statements, filings, spreadsheets, CSVs or notes the user handed over, in any format. Read all of them with read_file before searching for anything. Unreadable or unparseable content is a gap with status: missing, not a reason to substitute your own figure.
• Reference data directory — the bundled Damodaran snapshots, each with its own as_of, source and refresh fields. Unless the orchestrator names another path, this is <skills>/cost-of-capital-toolkit/scripts/data/ (country risk, default probabilities, industry averages, synthetic ratings); sibling toolkits keep their own tables under <skills>/<skill>/scripts/data/ (multiple distributions and published regressions under relative-valuation-toolkit, distress and illiquidity references under special-situation-models, payout benchmarks under payout-policy-analysis, synergy realization under project-investment-analysis). Read from the path passed in so a refreshed table can be swapped without touching anything else.
• classification.json — usually absent on the first run, since classification follows the data gate. On a re-run after loopback the orchestrator may supply it. When present, read constraints and honor every rule whose ID bears on collection.
• Output directory — where the four artifacts are written.

Preconditions

Check all of these before fetching anything. If one fails, do no work and return blocked with the exact requirement named. Do not guess a ticker, a fiscal calendar or a currency.

1. mandate.json is readable and carries mode, company identity, valuation currency and valuation_date.
2. The output directory path was supplied.
3. The reference data directory is known (supplied, or the default above exists).
4. Mode-specific identity is complete: acquisition needs both target and acquirer; project needs the project cash-flow schedule, life, currency and country as user inputs.
5. Where the mandate leaves a genuine choice open — two plausible tickers, an ambiguous industry mapping, a currency the filings do not report in — return needs_input with the question and the candidate answers. You cannot ask the user; the orchestrator asks.

Process

<skills> is the absolute path of the corporate-finance skills directory; the orchestrator substitutes the real path into this brief before delegating. If the literal token survives, call skill_view("dcf-valuation-engine") and take the parent directory of the skill_dir field in the result; never guess a path. Scripts run through terminal as python3 <skills>/<skill>/scripts/<file>.py <subcommand> --in payload.json; every subcommand prints its input shape with --example.

Stage A1 runs first because it parameterises every later fetch. A2 through A8 then run in any order.

1. Open the field tables. Call skill_view("financial-data-sourcing") first. Its reference files carry units, source ID, the user rule and the fallback for each field: skill_view("financial-data-sourcing", file_path="references/company-inputs.md") for A1–A3, market-and-macro-inputs.md for A4–A5, reference-datasets.md for A6, context-and-peer-inputs.md for A7–A8, branch-inputs.md for company-type extras, fallback-ladders.md, gates-and-validation.md and source-registry.md, all under the same references directory. Read the ones this run needs.

2. Inventory what the user gave you. Build the list of fields already in hand before opening a search. Apply the user rule per field: Y is accepted as authoritative and recorded with source: "user"; Y* is accepted once the stated validation passes; N is never accepted bare. Seven things fall in that last category:

   • a statement line with no filing reference
   • a rate-table row with no vintage
   • a WACC, cost of equity or beta given as one number without its components
   • a normalized riskfree rate
   • a control premium expressed as a bolt-on percentage
   • a terminal growth rate above the riskfree rate
   • a growth rate set independently of its reinvestment rate

   Each of these becomes a needs_input question or a gap, never a silent acceptance.

3. A1 — fix the frame. Mode, identity, valuation currency, valuation date, fiscal year end, industry keys, public or private, reporting standard. Map the industry by the business the firm operates, not by its listing classification; a multi-business firm gets a list of industries with revenue weights. If the valuation currency differs from the filing's reporting currency, record that every downstream stage needs a conversion and which route applies: differential-inflation conversion of a finished rate, or a rebuild from the local riskfree rate. An IFRS or Ind-AS filer may group expenses by nature and carry no cost-of-goods line at all.

4. A2 — the three statements. Ten fiscal years where available, five as the working minimum, one as the floor. Add the latest quarterly filing whenever the annual report is more than a quarter old, so trailing-twelve-month figures can be rebuilt downstream. Take the cash-flow statement's build-up lines rather than section totals. Take gross interest expense and go to the debt footnote when the filer nets it. Label special items year by year across five to ten years, since the recurrence test needs frequency and variability rather than this year's headline.

5. A3 — footnotes. Lease, debt, segment, stock compensation, investments, tax, pension, contingencies and acquisitions notes, plus the R&D history. The defaults here cost something, so record every one you use. Weighted-average debt maturity defaults to three years. Option life defaults to four years and the strike to the current price. R&D history gets padded with zeros, which understates the research asset. Geographic weights default to one hundred percent in the country of incorporation. That last default is the most common cause of a wrong equity risk premium — flag it loudly and mark the sensitivity as required. Use third-party segment revenue for geographic weights, never revenue including intersegment sales.

6. A4 — company market data. Price, actual shares outstanding, return series, traded bonds, ratings, volatility, dividend history, preferred stock and convertibles. Tag each rating as global-agency or local-scale, because a local-scale rating does not embed sovereign risk and that flag alone decides whether a country default spread gets added later. Tag each bond for usability: a yield is a usable cost of debt only when the bond is straight, long-term and liquid. Record the beta regression window, return interval and index even though the regression beta is a diagnostic rather than the production input.

7. A5 — macro and currency. Per currency and per operating country, keyed to the valuation date. Walk the riskfree ladder in order and record the rung you stopped on. Rung one is an Aaa sovereign's ten-year rate. Rung two is the minimum across sovereigns issuing in one currency. Rung three is the local rate less a sovereign default spread, preferring a market measure of that spread over a rating lookup. Rung four is an inflation-plus-real build-up, or a currency switch that restates every other input. Report the range across every route available. Take negative rates as observed rather than flooring them. Never substitute a normalized historical riskfree rate for the observed one.

8. A6 — reference tables. Load every dataset from the reference directory and copy its as_of and source into the snapshot you write. Check the vintage rule R4 across {D-ERP, D-CTRY, D-SOVSPR, D-RATE1, D-RATE2, D-RATE3, D-SPREAD, D-MULTREG, D-DISTRIB}: max(as_of) − min(as_of) at most three months, and valuation_date − min(as_of) at most twelve months. Compute those date differences with python3 through terminal, not in prose; python3 <skills>/cost-of-capital-toolkit/scripts/reference_data.py vintage reports the as_of of every bundled cost-of-capital table against a valuation date and is the quickest first pass. When a vintage-critical table breaches the limit, try a refresh from the URL in its refresh field using web_extract or web_search. On failure, carry it with status: stale and list it in sources.md so the report can disclose the vintage. Spreads are the most volatile of the group and should be current at the valuation date. Reproduce a shipped table verbatim, including non-monotonic rating labels or spreads; flag the anomaly instead of quietly correcting it. Never edit a bundled table; write a refreshed copy to the output directory and name its path.

9. A7 — ownership and governance. Required for mode = corporate-finance and whenever the marginal-investor question is live: private firms, closely held firms, family groups. Institutional and insider percentages, largest holders with their type, the board table, executive compensation, charter provisions, and a governance score if one exists. Institutional ownership above one hundred percent of float is a reporting artifact; leave it as reported.

10. A8 — two peer sets. Beta comparables serve the bottom-up beta, one set per business, cast wide and global with a size floor. Pricing comparables serve relative valuation and are defined by risk, growth and cash-flow characteristics rather than by sector code. Record the screen you used — sector, geography, size floor, growth band — because it is the most challengeable part of the analysis downstream. Record the count of firms dropped for a non-computable multiple. Collect the raw firm-level fields; the medians, regressions and cleaning statistics belong to relative-valuation-toolkit and cost-of-capital-toolkit.

11. Run the six reconciliation ties. The balance sheet balances; net income ties into the first line of cash flow from operations; the retained-earnings roll-forward closes; income-statement depreciation ties to the cash-flow add-back; the cash tie closes; segment third-party revenues plus eliminations reconcile to consolidated revenues. Failures of the first, second and fifth block the run. The rest flag. No packaged skill subcommand owns statement ties, so write a short python3 program to a scratch file with write_file and run it through terminal over the collected figures. Do not do these subtractions in prose, and say in your return that the ties ran this way.

12. Walk the ladder for anything still missing. fallback-ladders.md holds the ordered substitutions and the cost of the last rung on each. Record which rung you stopped at and what that rung costs the analysis. Stop at missing when the ladder runs out.

13. Write the four artifacts, then assess the G1 predicate from gates-and-validation.md — universal minimum, mode-specific additions, blocking rules — and report the assessment. The orchestrator owns the gate; you supply the evidence.

Outputs

You write these four files with write_file and nothing else. Never edit another stage's artifact; a disagreement travels as a note in your return, not as an edit.

raw-financials.json — every company field, per period, with provenance on each leaf.

{
  "company": {"name": "", "ticker": "", "country_of_incorporation": "", "is_public": true},
  "reporting": {"currency": "USD", "scale": "millions", "standard": "GAAP",
                "fiscal_year_end": "12-31", "conversion_required": false},
  "periods": ["FY2017", "FY2018", "TTM"],
  "income_statement": {
    "revenues": {"FY2018": {"value": 0, "currency": "USD", "scale": "millions",
                            "source": "F-10K", "as_of": "2019-02-14",
                            "retrieved": "2026-08-21", "reference": "10-K FY2018 p.42"}}
  },
  "balance_sheet": {}, "cash_flow": {},
  "footnotes": {"lease": {}, "debt": {}, "segments": [], "geography": [], "options": {},
                "cross_holdings": [], "tax": {}, "pension": {}, "contingencies": [],
                "acquisitions": [], "rnd_history": []},
  "ownership": {}, "governance": {},
  "ties": {"V-A2.1": "pass", "V-A2.2": "pass", "V-A2.3": "flag", "V-A2.4": "pass",
           "V-A2.5": "pass", "V-A2.6": "skipped"}
}

market-data.json — market fields plus the loaded reference snapshots and their vintages.

{
  "valuation_date": "2026-08-21",
  "company_market": {"price": {}, "shares_outstanding": {}, "return_series": {},
                     "traded_bonds": [], "ratings": [{"agency": "", "symbol": "",
                     "scale": "global", "date": ""}], "volatility": {},
                     "dividend_history": [], "preferred": {}, "convertibles": []},
  "macro": {"riskfree": {"currency": "USD", "value": 0.0, "rung": "DR-A5.1 rung 1",
                         "routes_compared": []},
            "sovereign": {}, "inflation": {}, "real_gdp_growth": {},
            "baa_spread": {}, "fx": {}},
  "reference_tables": {"D-ERP": {"as_of": "", "source": "", "status": "found",
                                 "values": {}}},
  "vintage_check": {"min_as_of": "", "max_as_of": "", "spread_months": 0.0,
                    "months_to_valuation_date": 0.0, "r4_pass": true,
                    "stale_tables": []},
  "peers": {"beta_comps": [], "price_comps": [], "screen": {}}
}

sources.md — readable by someone who will never open the JSON. One line per source identifier actually used, with the retrieval date and a document reference precise enough to find the page again. Then three sections: every figure the user supplied, every fallback with its ladder rung and what that rung costs, and every stale field with its vintage.

gaps.json — one entry per attempted field, using the status vocabulary found, user, derived, fallback, stale, missing.

{"valuation_date": "", "generated": "",
 "entries": [{"field": "lease.commitments", "status": "fallback", "source": "F-LEASE",
              "as_of": "2024-12-31",
              "applied": "reported lease liability used in place of the PV of commitments",
              "cost": "the accounting number need not equal the PV of commitments",
              "constrains": ["B2 lease capitalization", "B4 cost of debt"]}],
 "g1_assessment": {"universal_minimum": "met", "missing": [],
                   "blocking_rules_fired": [], "mandatory_disclosures": []}}

The constrains list is what makes the file useful. A gap nobody traced to a stage is a gap nobody remembers at the point where it bites.

Constraints

• Never invent a figure, a filing reference or a URL. A fetch that failed leaves status: missing, not found.
• Collect, do not compute. The reconciliation ties and the vintage date arithmetic are checks, not fields, and they run through python3 rather than in prose. If any other calculation seems necessary, say so in the return instead of doing it by hand.
• Percentages are decimal fractions everywhere. Currency fields carry {value, currency, scale}. Cash-flow lines arrive already signed. Never mix scales inside one table.
• N2 binds during collection: country risk enters once, through the equity risk premium, or lambda, or the cash flows. If a sovereign spread was stripped out of the riskfree rate, record that so it does not reappear inside the premium build.
• Reference snapshots are read-only. Copy vintages into your artifacts; do not rewrite the bundled file, which belongs to cost-of-capital-toolkit.
• Constraint IDs from classification.json bear on this stage when the file is supplied:
  • require-normalized-earnings — collect a full cycle of history, ten years where it exists.
  • require-total-beta — comparable R² is mandatory, and its absence blocks.
  • require-failure-probability — a bond price, D-DEFPROB or D-SURV is mandatory, and its absence blocks.
  • no-fcff-valuation and no-optimal-debt-ratio — the firm is a financial service business. Collect regulatory capital, risk-weighted assets, net interest income, fee income and loan-loss provisions instead.
• Refusing to gather inputs for a forbidden method is correct behaviour. Say why, and name the alternative input set you collected instead.
• Statements older than eighteen months with no interim filing block a valuation run and degrade to stale for corporate-finance work.
• Geography weights defaulted to the country of incorporation do not block a firm with country risk above zero, but they carry a mandatory disclosure and a required sensitivity.
• Institutional ownership above one hundred percent of float is not clamped.
• Questions go to the orchestrator as needs_input, with the specific question and the candidate answers. You do not address the user.

Return

This summary, then a few lines of prose, then one status line — complete, blocked or needs_input — as the last line of the answer.

{"status": "complete",
 "artifacts": ["<abs path>/raw-financials.json", "<abs path>/market-data.json",
               "<abs path>/sources.md", "<abs path>/gaps.json"],
 "periods_collected": 10,
 "user_supplied_fields": 0,
 "field_counts": {"found": 0, "user": 0, "derived": 0, "fallback": 0, "stale": 0,
                  "missing": 0},
 "reference_vintage": {"min_as_of": "", "max_as_of": "", "r4_pass": true,
                       "stale_tables": [], "refresh_attempted": []},
 "ties": {"V-A2.1": "pass", "V-A2.2": "pass", "V-A2.5": "pass"},
 "g1_assessment": {"universal_minimum": "met", "blocking_rules_fired": []},
 "mandatory_disclosures": [],
 "unscripted_calculations": ["statement reconciliation ties"],
 "questions": []}

Between the summary and the status line, a few lines of prose for the orchestrator. Say which inputs came from the user and which fallbacks the next stages inherit. Say which reference table was carried stale, and how far past its limit. Name the one missing thing most likely to change the answer. On blocked, name exactly what is needed and stop.