Theme Detection Methodology

Theme Detection Methodology

Overview

The Theme Detector uses a 3-Dimensional Scoring Model to identify, rank, and assess market themes. Unlike single-score ranking systems, this approach separates the intensity of a theme (heat), its maturity stage (lifecycle), and the reliability of the signal (confidence) into independent dimensions.

Dimension 1: Theme Heat (0-100)

Theme Heat measures the direction-neutral strength of a theme. A high heat score means the theme is generating significant market activity, regardless of whether it is bullish or bearish.

Components

1.1 Momentum Score (weight: 30%)

Measures the average performance of constituent industries.

Formula:

momentum_score = normalize(weighted_avg_change_pct, scale=[-5%, +5%] -> [0, 100])

weighted_avg_change_pct = SUM(industry_change_pct * industry_market_cap) / SUM(industry_market_cap)

Data Source: FINVIZ industry performance (1-week change %)

Scoring Scale:

Note: Absolute value is used so both strong bullish and strong bearish moves generate high heat.

1.2 Volume Score (weight: 25%)

Measures abnormal volume activity across the theme.

Formula:

volume_score = normalize(avg_relative_volume, scale=[0.5, 3.0] -> [0, 100])

avg_relative_volume = MEAN(stock_volume / stock_avg_volume) for all stocks in theme

Data Source: FINVIZ relative volume (volume / avg volume ratio)

Scoring Scale:

1.3 Uptrend Ratio Score (weight: 25%)

Measures the percentage of stocks in the theme that are in technical uptrends.

Formula:

uptrend_score = normalize(uptrend_ratio, scale=[0%, 100%] -> [0, 100])

uptrend_ratio = count(stocks_in_uptrend) / count(total_stocks)

Data Source: uptrend-dashboard output (3-point evaluation: price > 50-day SMA, 50-day SMA > 200-day SMA, 200-day SMA rising)

If uptrend data is unavailable: Use price-above-SMA200 as proxy from FINVIZ data. Score is reduced by 20% confidence penalty.

Scoring Scale:

Note: For bearish themes, invert the ratio (use downtrend ratio instead).

1.4 Breadth Score (weight: 20%)

Measures how broadly the theme is participating (not just a few large-cap names).

Formula:

breadth_score = normalize(participation_rate, scale=[0%, 100%] -> [0, 100])

participation_rate = count(stocks_moving_in_direction > 1%) / count(total_stocks)

Data Source: FINVIZ stock-level performance data

Scoring Scale:

Theme Heat Composite

theme_heat = (momentum_score * 0.30) + (volume_score * 0.25) + (uptrend_score * 0.25) + (breadth_score * 0.20)

Dimension 2: Lifecycle Maturity

Lifecycle assessment classifies a theme into one of four stages: Early, Mid, Late, or Exhaustion. This is critical for distinguishing emerging opportunities from crowded trades.

Components

2.1 Duration Score (weight: 25%)

How long the theme has been active (consecutive weeks of elevated heat).

Measurement: Count weeks where theme_heat >= 40.

Limitation: Duration tracking requires historical data. On first run, duration defaults to "Unknown" and lifecycle uses other factors only.

2.2 Extremity Clustering Score (weight: 20%)

Percentage of stocks in the theme near 52-week highs or lows.

Formula:

extremity_pct = count(within_5pct_of_52wk_high_or_low) / count(total_stocks)

Data Source: FINVIZ 52-week high/low data

2.3 Valuation Score (weight: 20%)

Average P/E ratio of theme constituents relative to S&P 500 P/E.

Formula:

relative_pe = avg_theme_pe / sp500_pe

Data Source: FMP API for P/E ratios (optional; uses FINVIZ forward P/E as fallback)

2.4 ETF Proliferation Score (weight: 20%)

Number of thematic ETFs tracking the theme. More ETFs indicate greater retail/institutional attention.

Source: thematic_etf_catalog.md (static reference)

2.5 Media/Narrative Saturation (weight: 15%)

Assessed via WebSearch during narrative confirmation step.

Lifecycle Stage Classification

The final lifecycle stage is determined by majority vote across the five component signals:

stage_votes = [duration_signal, extremity_signal, valuation_signal, etf_signal, media_signal]
lifecycle_stage = mode(stage_votes)  # most frequent stage

If tied, use the more mature stage (conservative approach to protect against crowded trade risk).

Dimension 3: Confidence (Low / Medium / High)

Confidence measures how reliable the theme detection is, based on data quality and confirmation signals.

Layers

3.1 Quantitative Layer (base)

Based on data breadth and consistency:

3.2 Breadth Layer (modifier)

Cross-sector participation adds confidence:

3.3 Narrative Layer (modifier, applied in Step 4)

WebSearch confirmation adjusts confidence:

Final Confidence

confidence = apply_modifiers(quantitative_base, breadth_modifier, narrative_modifier)
confidence = clamp(confidence, Low, High)

Direction Detection

Theme direction (bullish vs. bearish) is determined separately from heat:

Algorithm

weighted_perf = sum(industry_change_pct * industry_mcap) / sum(industry_mcap)
uptrend_ratio = count(uptrend_stocks) / count(total_stocks)  # if available

if weighted_perf > 0.5 and (uptrend_ratio > 0.5 or uptrend_data_unavailable):
    direction = "Bullish"
elif weighted_perf < -0.5 and (uptrend_ratio < 0.5 or uptrend_data_unavailable):
    direction = "Bearish"
else:
    direction = "Neutral"

Direction Strength

Data Sources

Primary: FINVIZ

Elite Mode (recommended):

• CSV export endpoint: https://elite.finviz.com/export.ashx?v=151&f=ind_{code},cap_smallover,...&auth=KEY
• Provides: ticker, company, sector, industry, market cap, P/E, change%, volume, avg volume, 52wk high/low, RSI, SMA20/50/200
• Rate limit: 0.5s between requests
• Coverage: Full stock universe per industry

Public Mode (fallback):

• HTML scraping: https://finviz.com/screener.ashx?v=151&f=ind_{code},cap_smallover
• Provides: Same fields but limited to page 1 (~20 stocks per industry)
• Rate limit: 2.0s between requests (aggressive scraping may trigger blocks)
• Coverage: Top 20 stocks per industry by market cap

Secondary: FMP API (optional)

• P/E ratios for valuation analysis
• Not required; FINVIZ forward P/E used as fallback
• Useful for more granular valuation metrics

Tertiary: uptrend-dashboard (optional)

• CSV output from the uptrend-dashboard skill
• Provides 3-point technical evaluation per stock
• Significantly improves uptrend ratio accuracy
• If unavailable, FINVIZ price-vs-SMA200 is used as proxy

Quaternary: WebSearch (narrative layer)

• Used for narrative confirmation in Step 4
• Not automated; Claude performs searches during workflow
• Subjective assessment of media coverage and analyst sentiment

Integration with uptrend-dashboard

When uptrend-dashboard data is available, the theme detector uses it for enhanced 3-point evaluation:

3-Point Evaluation Criteria:

1. Price > 50-day SMA (short-term trend)
2. 50-day SMA > 200-day SMA (medium-term trend, golden/death cross)
3. 200-day SMA is rising (long-term trend confirmation)

Stocks meeting all 3 points = in confirmed uptrend Stocks meeting 0 points = in confirmed downtrend

This provides more accurate uptrend ratios than simple price-above-SMA200 proxy.

Output Schema

JSON Output Structure

{
  "metadata": {
    "date": "2026-02-16",
    "mode": "elite",
    "themes_analyzed": 14,
    "industries_scanned": 145,
    "total_stocks": 5200,
    "uptrend_data_available": true,
    "execution_time_seconds": 150
  },
  "themes": [
    {
      "name": "AI & Semiconductors",
      "direction": "Bullish",
      "direction_strength": "Strong",
      "theme_heat": 82,
      "heat_components": {
        "momentum": 90,
        "volume": 75,
        "uptrend": 85,
        "breadth": 70
      },
      "lifecycle": {
        "stage": "Late",
        "duration_weeks": 12,
        "extremity_pct": 45,
        "relative_pe": 1.8,
        "etf_proliferation_score": 100,
        "etf_count": 11
      },
      "confidence": "High",
      "confidence_components": {
        "quantitative": "High",
        "breadth_modifier": "+1",
        "narrative_modifier": "pending"
      },
      "industries": [
        {
          "name": "Semiconductors",
          "change_pct": 4.2,
          "avg_relative_volume": 1.8,
          "uptrend_ratio": 0.75,
          "stock_count": 35
        }
      ],
      "top_stocks": [
        {"ticker": "NVDA", "change_pct": 6.5, "relative_volume": 2.1},
        {"ticker": "AVGO", "change_pct": 4.8, "relative_volume": 1.9}
      ],
      "proxy_etfs": ["SMH", "SOXX", "AIQ", "BOTZ"]
    }
  ],
  "industry_rankings": {
    "top_10": [...],
    "bottom_10": [...]
  },
  "sector_summary": {
    "Technology": {"uptrend_ratio": 0.65, "avg_change_pct": 2.1},
    "Energy": {"uptrend_ratio": 0.40, "avg_change_pct": -1.5}
  }
}

Known Limitations

1. Static theme definitions: Cross-sector themes are predefined in cross_sector_themes.md. New themes (e.g., a sudden meme-stock theme) are not automatically detected.

2. Industry granularity: FINVIZ industry classification may not perfectly map to investment themes. Some industries span multiple themes.

3. Single-stock dominance: Large-cap stocks (e.g., NVDA in AI) can skew theme-level metrics. Market-cap weighting amplifies this effect.

4. Temporal lag: Weekly performance data does not capture intraday or same-day momentum shifts.

5. ETF catalog staleness: The thematic ETF catalog is manually maintained and may not reflect recent ETF launches or closures.

6. Public mode data limits: Only ~20 stocks per industry are captured, which may underrepresent small/mid-cap participation.

7. Duration tracking: First-run analysis cannot assess theme duration without historical baseline data.

8. Narrative subjectivity: Confidence adjustment from WebSearch is inherently subjective and depends on search result quality.

9. Survivorship bias: Analysis only covers currently listed stocks and active ETFs, missing delisted or closed instruments.

10. FINVIZ data delays: Public FINVIZ data is 15-minute delayed; Elite provides real-time during market hours.