Factor Models

Factor models

A factor portfolio is a subscript problem before it is a statistics problem. The formula for HML is public and takes one line; what decides whether your HML is a factor or a look-ahead is when the characteristic was measured relative to the return it earns. Get that wrong and the t-statistic is not weak evidence, it is manufactured evidence, and every standard error correction downstream is beside the point.

characteristic known at t -> portfolio formed at t -> return measured over t -> t+1

Every number below is printed by scripts/factor_regression.py (numpy / pandas, seed 0, about 3 s, statsmodels optional). The panel is 500 stocks x 240 months in which the truth is alpha = 0 and there is no value premium: r_it = beta_i * m_t + e_it, book equity is a slow random walk, so book-to-market moves almost entirely with the stock's own price. Anything a sort finds in it is an artefact.

1. 🚨 Forming on the contemporaneous characteristic manufactures the alpha

✅ Measured, seed 0, top 30 % minus bottom 30 %, re-sorted monthly, Newey-West lags floor(4(T/100)^(2/9)) = 4. mean and CAPM a are percent per month; CAPM a is the intercept of the long-short return on the value-weighted market:

Read the two bolded pairs. The same panel, the same code, the same true alpha of zero: formed honestly the alpha is +0.002 %/month with t = 0.02; formed on the contemporaneous characteristic it is -1.318 %/month with t = -14.63, or +5.180 %/month with t = 52.36 if the leaked signal is momentum instead of value.

🚨 The sign is a distraction. The leaked book-to-market sort is negative because a stock that fell during month t has a higher B/M at the end of month t: "value" becomes a list of this month's losers, and the long-short is short the winners. Sort the identical panel on a signal whose leak points the other way — a momentum window whose last month is the month being predicted — and the same bug reads as a 5 %/month premium with a t-statistic of 52. A leak does not announce itself with an implausible sign; it announces itself with an implausible t-statistic. A t of 15 on 240 months of a single characteristic is not a discovery.

The guard. sort_portfolio() and fama_french_2x3() refuse lag=0 unless you pass allow_lookahead=True, so a pipeline can call them and fail loudly rather than print a great number:

ValueError: lag=0: the sorting characteristic would be measured at the end of the return
period it is supposed to predict (a one-period look-ahead). ...

Value-weight unless you mean to bet on microcaps. ✅ Measured: the equal-weight leak is 40 % larger than the value-weight leak (-1.846 vs -1.318 %/month alpha), because equal weighting puts the same money in the panel's 12 %-a-month idiosyncratic minnows as in its 5 % giants. The honest rows differ by almost nothing (+0.002 vs -0.003). Equal weighting does not fix a leak; it amplifies one, and it changes what the portfolio is a bet on.

2. The 2x3 sort, and what Ken French actually publishes

✅ Source-verified 2026-09-09 at Ken French's Data Library (mba.tuck.dartmouth.edu/pages/faculty/ken.french/Data_Library/, factor and six-portfolio description pages):

• The factors come from "6 value-weight portfolios formed on size and book-to-market" — they are already value-weighted; do not re-weight them.
• SMB = 1/3 (SV + SN + SG) - 1/3 (BV + BN + BG) and HML = 1/2 (SV + BV) - 1/2 (SG + BG), verbatim from the factor page. fama_french_2x3() implements exactly these two averages.
• Rm-Rf is the value-weighted return of all CRSP firms incorporated in the US and listed on NYSE, AMEX or NASDAQ, minus the one-month Treasury bill rate. It is an excess return; SMB and HML are long-short and already excess.
• Breakpoints: "The size breakpoint for year t is the median NYSE market equity at the end of June of year t" and "The BE/ME breakpoints are the 30th and 70th NYSE percentiles" — NYSE-only breakpoints applied to a NYSE + AMEX + NASDAQ universe. A full-universe breakpoint puts far more names in the small bucket and is a different factor.
• Portfolios are formed at the end of each June and held from July of year t through June of t+1 — an annual re-form, not a monthly re-sort.
• 🔑 BE/ME for June of year t is book equity for the last fiscal year end in t-1 divided by ME for December of t-1. So the accounting number is 6 to 18 months stale by the time it earns a return, and market equity in the ratio is from the previous December, not from June. That gap is deliberate: it is the delay that guarantees the data was public.
• Files exist at daily, weekly, monthly and annual frequency (the site listed July 1926 - July 2026 for the first three, 1927 - 2025 annual, on 2026-09-09).

✅ Measured on the synthetic panel (same seed, same NW lags), where truth is again zero:

The leak reaches HML through the B/M sort (t = -18.97) and SMB through the size sort — a stock that fell is "small" at the end of the month — but SMB's leak is an order of magnitude weaker (-0.089, t = -1.20) because market cap is dominated by its own cross-sectional spread rather than by one month of return. A leak in a two-way sort shows up in one leg and hides in the other.

⚠️ Reproducing the published factors is a different job from implementing the formulas. Monthly re-sorting, full-universe breakpoints and a contemporaneous ME in the ratio each give a series that satisfies the formulas above and correlates imperfectly with the file you downloaded. If your HML "does not match Ken French", check the re-form calendar and the breakpoint universe before you check your code: references/ken_french_data_library.md has the full timing diagram, the two different market equities in the ratio and the sort, and a seven-step reproduction checklist.

3. Newey-West in statsmodels: three call sites, three different answers

✅ Source-verified in the installed statsmodels 0.15.0 (statsmodels/stats/sandwich_covariance.py, statsmodels/base/covtype.py):

• ✅ weights_bartlett(nlags) returns 1 - np.arange(nlags + 1) / (nlags + 1.0), so the weight on lag j is 1 - j/(L+1) and L is the highest lag included, not counting lag 0. An implementation that divides by L instead of L+1, or that treats L as a count of terms, gives different standard errors from statsmodels on the same data.
• 🚨 The floor(4(T/100)^(2/9)) rule is not applied by fit(). It lives in S_hac_simple and only fires when nlags is None. ✅ Measured: fit(cov_type="HAC") with no maxlags raises KeyError: 'maxlags'. There is no default lag length in the fit() path — you must state one, and you should state it in the paper too.
• ✅ Measured: the module's own numpy implementation matches fit(cov_type="HAC", cov_kwds={"maxlags": 4}) to 1.04e-17 on standard errors and 5.18e-19 on coefficients. statsmodels labels the result "Standard Errors are heteroscedasticity and autocorrelation robust (HAC) using 4 lags and without small sample correction".
• ✅ Measured: cov_hac_simple(res, nlags=4) standard errors are 1.00421054x the fit() ones, exactly sqrt(T/(T-k)) = 1.00421054. Same estimator, two defaults; quoting the ratio is how you tell which one produced a t-statistic.
• ✅ Measured on the honest long-short: plain OLS SE 0.001071, HAC(0) = White HC0 SE 0.001049, HAC(4) SE 0.000990. HAC standard errors are not always larger. They are larger when the residuals are positively autocorrelated and smaller when they are not; "I used Newey-West so I am being conservative" is not a valid sentence.
• ⚠️ cov_hac_simple's own docstring says it is "verified only for nlags=0" and that the correction factor is a guess needing a reference. The lag-0 case matches White exactly; the general case matched this module's independent implementation to 1e-17, which is the check worth running rather than trusting either.

4. Fama-MacBeth: the leak survives the second pass

fama_macbeth() runs a cross-sectional regression of ret[t] on the characteristics at t - lag every period, then tests the mean slope. ✅ Measured, 239 monthly cross-sections, slopes x 100:

Two things to take from it. First, the two-pass procedure does not repair a subscript error — the leaked slope is 300x the honest one and significant at any threshold. Second, the leak's plain t of -19.62 falls to -11.61 once the slope series is given Newey-West errors, a 41 % haircut, because the leaked slopes are strongly autocorrelated. On the honest rows the two t-statistics are indistinguishable (0.08 vs 0.07). 🚨 The Fama-MacBeth standard error is the standard deviation of the slope series; it fixes cross-sectional correlation and nothing else. Serial correlation in the slopes needs a HAC correction on top, and it is the leaked or persistent specifications where that matters most.

5. Joining to the Data Library: the month is not a timestamp

✅ Measured: pd.to_datetime("202607", format="%Y%m") returns 2026-07-01, a first-of-month timestamp. Inner-joining six monthly factor rows parsed that way to a month-end strategy series keeps 0 of 6 rows; french_period_index() + align_monthly(), which convert both sides to a PeriodIndex(freq="M"), keep 6 of 6. A silent empty join is the most common way factor regressions "have no data", and an inner join that silently keeps 3 of 6 is worse.

🚨 Frequency must match on both sides. The monthly file's row for 202607 is the return over the whole month of July 2026; regress monthly strategy returns on it. The daily file stamps YYYYMMDD and joins on the day. Regressing daily returns on monthly factors (or the reverse, after a forward-fill) produces betas that are arithmetic, not estimates.

🚨 Check the units. ✅ Measured: the same decimal strategy regressed on the same factor column gives beta -0.004243 if the column is treated as percent and -0.4243 if it is treated as a decimal — a factor of exactly 100. ⚠️ This skill did not verify the units of the archive files themselves; read the first rows of your own download and confirm before you divide by 100.

6. Traps

• 🚨 lag=0 on any characteristic derived from price. Book-to-market, market cap, dividend yield, earnings yield, 1/P, and any momentum window whose last period is the return period. Section 1 is what it costs.
• 🚨 A t-statistic that is too good. On zero-alpha data, honest |t| was 0.02-0.67 and leaked |t| was 11.6-52.4. If your single-characteristic factor prints |t| > 10, suspect the subscript before the economics.
• 🚨 fit(cov_type="HAC") with no maxlags. It raises; there is no default. And the two call paths disagree by sqrt(T/(T-k)) (section 3).
• 🚨 Equal weighting by default. It is a different bet (microcaps), and it amplified the leak by 40 % here.
• 🚨 Re-sorting monthly and calling it Fama-French. French re-forms once a year at the end of June with NYSE breakpoints and a Dec-t-1 market equity in the ratio.
• 🚨 Survivorship in the panel. Nothing in section 1 detects a universe that only contains today's listed names; the sort is honest and the panel is not. That is ../../../fin-core/skills/research-integrity-guards/SKILL.md.
• 🚨 Every breakpoint, lag and weighting scheme is a trial. Deciles vs terciles, value vs equal, lag 1 vs lag 6, monthly vs annual re-form: this skill's own demo contains eight specifications of one idea. Record them in ../../../fin-core/skills/backtest-validation/scripts/trial_ledger.py before reporting the best one.
• ⚠️ linearmodels (which ships a panel FamaMacBeth) is not installed in this environment and nothing about it was verified here.

7. Scripts

• scripts/factor_regression.py - all five sections. simulate_panel, sort_portfolio, momentum, fama_french_2x3, newey_west, ts_alpha, fama_macbeth, french_period_index, align_monthly, and statsmodels_hac_check, which prints the comparison against statsmodels rather than asserting it. Without statsmodels the numpy Newey-West is still the reference and section 3's cross-check says it was skipped. About 3 s.

8. Where this sits

../../../fin-core/skills/factor-and-timeseries-research/SKILL.md owns scoring a single alpha signal (alphalens, IC, decay, turnover) and volatility forecasting; ../../../fin-core/skills/backtest-validation/SKILL.md counts the specifications; ../../../fin-core/skills/research-integrity-guards/SKILL.md owns point-in-time universes and the result gate; ../covariance-and-risk-models/SKILL.md turns these factors into a covariance matrix; ../portfolio-optimizers/SKILL.md turns expected returns into weights; and ../../../fin-libraries/skills/lib-alphalens/SKILL.md is the per-library deep dive for the signal-scoring tool.