Theme Detection Methodology

Theme Detection Methodology

Overview

The Theme Detector uses a 3-Dimensional Scoring Model to identify, rank, and assess market themes. Unlike single-score ranking systems, this approach separates the intensity of a theme (heat), its maturity stage (lifecycle), and the reliability of the signal (confidence) into independent dimensions.

Dimension 1: Theme Heat (0-100)

Theme Heat measures the direction-neutral strength of a theme. A high heat score means the theme is generating significant market activity, regardless of whether it is bullish or bearish.

Components

1.1 Momentum Score (weight: 35%)

Measures the direction-neutral momentum strength of constituent industries using a log-sigmoid function on multi-timeframe weighted returns (1W 10%, 1M 25%, 3M 35%, 6M 30%).

Formula:

weighted_return = perf_1w * 0.10 + perf_1m * 0.25 + perf_3m * 0.35 + perf_6m * 0.30
momentum_score = 100 / (1 + exp(-2.0 * (ln(1 + |weighted_return|) - ln(16))))

Midpoint at |15%| weighted return. Log transform compresses extreme values for better mid-range separation.

Data Source: FINVIZ industry performance (1W, 1M, 3M, 6M change %)

Scoring Scale:

Note: Absolute value is used so both strong bullish and strong bearish moves generate high heat.

1.2 Volume Score (weight: 20%)

Measures abnormal volume activity across the theme.

Formula:

volume_score = normalize(avg_relative_volume, scale=[0.5, 3.0] -> [0, 100])

avg_relative_volume = MEAN(stock_volume / stock_avg_volume) for all stocks in theme

Data Source: FINVIZ relative volume (volume / avg volume ratio)

Scoring Scale:

1.3 Uptrend Ratio Score (weight: 25%)

Measures the percentage of stocks in the theme that are in technical uptrends.

Formula:

uptrend_score = normalize(uptrend_ratio, scale=[0%, 100%] -> [0, 100])

uptrend_ratio = count(stocks_in_uptrend) / count(total_stocks)

Data Source: uptrend-dashboard output (3-point evaluation: price > 50-day SMA, 50-day SMA > 200-day SMA, 200-day SMA rising)

If uptrend data is unavailable: Use price-above-SMA200 as proxy from FINVIZ data. Score is reduced by 20% confidence penalty.

Scoring Scale:

Note: For bearish themes, invert the ratio (use downtrend ratio instead).

1.4 Breadth Score (weight: 20%)

Measures how broadly the theme is participating (not just a few large-cap names).

Formula:

breadth_score = normalize(participation_rate, scale=[0%, 100%] -> [0, 100])

participation_rate = count(stocks_moving_in_direction > 1%) / count(total_stocks)

Data Source: FINVIZ stock-level performance data

Scoring Scale:

Theme Heat Composite

theme_heat = (momentum_score * 0.35) + (volume_score * 0.20) + (uptrend_score * 0.25) + (breadth_score * 0.20)

Dimension 2: Lifecycle Maturity

Lifecycle assessment classifies a theme into one of five stages: Emerging, Accelerating, Trending, Mature, or Exhausting. This is critical for distinguishing emerging opportunities from crowded trades.

Components

2.1 Duration Score (weight: 25%)

How long the theme has been active (consecutive weeks of elevated heat).

Measurement: Count weeks where theme_heat >= 40.

Limitation: Duration tracking requires historical data. On first run, duration defaults to "Unknown" and lifecycle uses other factors only.

2.2 Extremity Clustering Score (weight: 25%)

Percentage of stocks in the theme near 52-week highs or lows.

Formula:

extremity_pct = count(within_5pct_of_52wk_high_or_low) / count(total_stocks)

Data Source: FINVIZ 52-week high/low data

2.3 Price Extreme Saturation Score (weight: 25%)

Proportion of stocks near 52-week extremes (within 5%).

Formula:

bullish: pct = count(dist_from_52w_high <= 0.05) / total
bearish: pct = count(dist_from_52w_low <= 0.05) / total
score = min(100, pct * 200)

2.4 Valuation Score (weight: 15%)

Average P/E ratio of theme constituents relative to S&P 500 P/E.

Formula:

relative_pe = avg_theme_pe / sp500_pe

Data Source: FMP API for P/E ratios (optional; uses FINVIZ forward P/E as fallback)

2.5 ETF Proliferation Score (weight: 10%)

Number of thematic ETFs tracking the theme. More ETFs indicate greater retail/institutional attention.

Source: thematic_etf_catalog.md (static reference)

Lifecycle Maturity Composite

maturity = (duration * 0.25) + (extremity * 0.25) + (price_extreme * 0.25) + (valuation * 0.15) + (etf_proliferation * 0.10)

Lifecycle Stage Classification

The lifecycle stage is classified from the maturity score:

Note: Media/Narrative Saturation is not included in the automated maturity calculation. Claude's WebSearch narrative confirmation can be used to qualitatively adjust the lifecycle assessment.

Dimension 3: Confidence (Low / Medium / High)

Confidence measures how reliable the theme detection is, based on data quality and confirmation signals.

Layers

3.1 Quantitative Layer (base)

Based on data breadth and consistency:

3.2 Breadth Layer (modifier)

Cross-sector participation adds confidence:

3.3 Narrative Layer (modifier, applied in Step 4)

WebSearch confirmation adjusts confidence:

Final Confidence

confidence = apply_modifiers(quantitative_base, breadth_modifier, narrative_modifier)
confidence = clamp(confidence, Low, High)

Direction Detection

Theme direction (leading vs. lagging) is determined by relative rank, not absolute price change:

Algorithm

1. Each industry gets a rank_direction based on its position in the momentum-ranked list: top half = "bullish" (leading), bottom half = "bearish" (lagging).
2. Theme direction is the majority vote of its constituent industries' rank_direction.

# Industry-level (industry_ranker.py)
rank_direction = "bullish" if rank <= len(industries) // 2 else "bearish"

# Theme-level (theme_classifier.py)
direction = majority_vote([ind.rank_direction for ind in theme.industries])

Important: Relative, Not Absolute

LEAD/LAG direction is relative. A "lagging" theme may still have positive absolute returns — it simply underperforms relative to other themes. This means:

• LEAD themes: Suitable for overweight / new positions
• LAG themes: Candidates for underweight reduction, not short signals

Data Sources

Primary: FINVIZ

Elite Mode (recommended):

• CSV export endpoint: https://elite.finviz.com/export.ashx?v=151&f=ind_{code},cap_smallover,...&auth=KEY
• Provides: ticker, company, sector, industry, market cap, P/E, change%, volume, avg volume, 52wk high/low, RSI, SMA20/50/200
• Rate limit: 0.5s between requests
• Coverage: Full stock universe per industry

Public Mode (fallback):

• HTML scraping: https://finviz.com/screener.ashx?v=151&f=ind_{code},cap_smallover
• Provides: Same fields but limited to page 1 (~20 stocks per industry)
• Rate limit: 2.0s between requests (aggressive scraping may trigger blocks)
• Coverage: Top 20 stocks per industry by market cap

Secondary: FMP API (optional)

• P/E ratios for valuation analysis
• Not required; FINVIZ forward P/E used as fallback
• Useful for more granular valuation metrics

Tertiary: uptrend-dashboard (optional)

• CSV output from the uptrend-dashboard skill
• Provides 3-point technical evaluation per stock
• Significantly improves uptrend ratio accuracy
• If unavailable, FINVIZ price-vs-SMA200 is used as proxy

Quaternary: WebSearch (narrative layer)

• Used for narrative confirmation in Step 4
• Not automated; Claude performs searches during workflow
• Subjective assessment of media coverage and analyst sentiment

Integration with uptrend-dashboard

When uptrend-dashboard data is available, the theme detector uses it for enhanced 3-point evaluation:

3-Point Evaluation Criteria:

1. Price > 50-day SMA (short-term trend)
2. 50-day SMA > 200-day SMA (medium-term trend, golden/death cross)
3. 200-day SMA is rising (long-term trend confirmation)

Stocks meeting all 3 points = in confirmed uptrend Stocks meeting 0 points = in confirmed downtrend

This provides more accurate uptrend ratios than simple price-above-SMA200 proxy.

Output Schema

JSON Output Structure

{
  "metadata": {
    "date": "2026-02-16",
    "mode": "elite",
    "themes_analyzed": 14,
    "industries_scanned": 145,
    "total_stocks": 5200,
    "uptrend_data_available": true,
    "execution_time_seconds": 150
  },
  "themes": [
    {
      "name": "AI & Semiconductors",
      "direction": "Bullish",
      "direction_strength": "Strong",
      "theme_heat": 82,
      "heat_components": {
        "momentum": 90,
        "volume": 75,
        "uptrend": 85,
        "breadth": 70
      },
      "lifecycle": {
        "stage": "Late",
        "duration_weeks": 12,
        "extremity_pct": 45,
        "relative_pe": 1.8,
        "etf_proliferation_score": 100,
        "etf_count": 11
      },
      "confidence": "High",
      "confidence_components": {
        "quantitative": "High",
        "breadth_modifier": "+1",
        "narrative_modifier": "pending"
      },
      "industries": [
        {
          "name": "Semiconductors",
          "change_pct": 4.2,
          "avg_relative_volume": 1.8,
          "uptrend_ratio": 0.75,
          "stock_count": 35
        }
      ],
      "top_stocks": [
        {"ticker": "NVDA", "change_pct": 6.5, "relative_volume": 2.1},
        {"ticker": "AVGO", "change_pct": 4.8, "relative_volume": 1.9}
      ],
      "proxy_etfs": ["SMH", "SOXX", "AIQ", "BOTZ"]
    }
  ],
  "industry_rankings": {
    "top_10": [...],
    "bottom_10": [...]
  },
  "sector_summary": {
    "Technology": {"uptrend_ratio": 0.65, "avg_change_pct": 2.1},
    "Energy": {"uptrend_ratio": 0.40, "avg_change_pct": -1.5}
  }
}

Known Limitations

1. Static theme definitions: Cross-sector themes are predefined in cross_sector_themes.md. New themes (e.g., a sudden meme-stock theme) are not automatically detected.

2. Industry granularity: FINVIZ industry classification may not perfectly map to investment themes. Some industries span multiple themes.

3. Single-stock dominance: Large-cap stocks (e.g., NVDA in AI) can skew theme-level metrics. Market-cap weighting amplifies this effect.

4. Temporal lag: Weekly performance data does not capture intraday or same-day momentum shifts.

5. ETF catalog staleness: The thematic ETF catalog is manually maintained and may not reflect recent ETF launches or closures.

6. Public mode data limits: Only ~20 stocks per industry are captured, which may underrepresent small/mid-cap participation.

7. Duration tracking: First-run analysis cannot assess theme duration without historical baseline data.

8. Narrative subjectivity: Confidence adjustment from WebSearch is inherently subjective and depends on search result quality.

9. Survivorship bias: Analysis only covers currently listed stocks and active ETFs, missing delisted or closed instruments.

10. FINVIZ data delays: Public FINVIZ data is 15-minute delayed; Elite provides real-time during market hours.