Kaggle Cognitive Cost Optimization

Kaggle Cognitive Cost Optimization: A Decision Framework

Problem

Kaggle has a hidden cost: quota + time + complexity. Most agents:

• Submit 5+ redundant times → 0 score gain
• Build elaborate custom models → lose to 0.8 × simple public kernel
• Re-submit 10× for "safety" → no benefit
• Spend 3 weeks on a problem that 1 public fork solves in 2 hours

Core Insight: The 0.6×BEST_PUBLIC Rule

After 20+ competitions (M8 Sports, S6E1-7, PTCG, ROGII, NeuroGolf, etc.):

For any Kaggle competition: choosing the best public kernel + ≤2 days tuning achieves ≥80% of the optimal result. The remaining 20% requires 10× more effort for marginal gain.

Evidence:

In 5+ competitions, custom work BEAT public by < 2% on average. Top 5% requires 10× more effort for 5% gain.

Context / Trigger Conditions

Use this skill when:

• First 48h of a new competition: spend ≥50% of time scanning top 10 public kernels
• Quota running low: prioritize 1 submission to best public > 3 to own variants
• Custom work plateauing: if your best OOF < public #1, the public is more likely the right answer
• Deadline approaching: stop tuning, submit the best combined
• Re-submit temptation: never re-submit within 48h of convergence start (see trueskill-simulation-competition-strategy)

Solution: The 3-Quota-First Strategy

For most competitions, allocate quota as:

Quota 1: Submit best public kernel EXACTLY (don't modify)
  → Establishes baseline. Validates 0.6×rule.
Quota 2 (optional): Submit public + minimal tuning (one new feature, one new param)
  → Tests if simple adaptation works.
Quota 3 (optional): Custom work if you have a fundamentally different angle
  → Only if evidence suggests the public is missing key insight.

Default: 1-2 quota submissions. Maximum 4 unless you're top 10% OOF clearly.

Cross-Modal Performance Optimization

Use when: 1 image AND text AND tabular data ALL available. The most robust approach is often:

1. Image model (ConvNeXt/EfficientNet) on raw pixels
2. Text model (BERT/DeBERTa) on descriptions
3. Tabular GBDT on metadata
4. Late fusion: weighted average of OOF predictions, weights chosen by stacking

Validation: each modality's OOF contribution must be positive; if a modality adds no OOF gain, remove it. Key trap: ensembling a weak model can hurt if the error pattern is random rather than uncorrelated with strong models (see ensemble-model-correlation-trap).

Anti-Patterns: What Wastes Quota

The Submission Strategy Decision Tree

Start
  │
  ├── Within first 48h of competition?
  │     ├── YES → Scan top 10 public kernels
  │     │         ├── Found >0.95 ratio to public → Fork best, minimal adapt
  │     │         ├── Found fundamental new angle → Custom work, use 1-2 quota
  │     │         └── Stagnant → Try public OVER different public (ensemble)
  │     └── NO → Is this a simulation (LB = TrueSkill)?
  │               ├── YES → Read trueskill-simulation-competition-strategy
  │               └── NO → Standard CV/LB validation
  │
  ├── Best OOF >> best public OOF?
  │     ├── NO → Custom work unlikely to win, submit best public
  │     └── YES → Submit 1, see if LB confirms
  │
  └── At deadline (final submission)?
        ├── Use 1-2 quota as high-roll duplicates of best agent
        └── Don't try new ideas (re-submit = reset convergence)

Exception Conditions (When 0.6×Rule Fails)

The 0.6×BEST_PUBLIC rule FAILS when:

• Specialized data access: drug/medical (domain jargon), scientific (custom formats)
• Code competition: fork requires finding dataset with specific artifacts (Biohub, ROGII, NeuroGolf) — the public kernel is the work
• Real-time streaming: latency-critical, not best-public-friendly
• Limited public activity: <50 public kernels → less proven baselines

In these cases, custom work has higher value (1.0× ratio more likely).

Evidence

• 20+ competitions over 4 months (Jun 2026 - Jul 2026)
• M8 Sports top 0.1407 vs 0.1390 public: own model ranked top 7%, public 1st
• S6E7 0.94942 vs 0.94999 public: 1pt behind public
• PTCG 967.8 vs 801.8 public: own better but eval panel misled (Marnie gap)
• ROGII 13.82 vs 5.26 public: own worse, rank 3161/4497
• Mean ratio 0.99: custom ≈ public across many competition types

When This Skill Is Required Reading

• First-time Kaggle agent setup
• Daily quota planning (before first submit)
• Deadline approaching (final 1-2 days)
• Custom work plateau (spend 2+ days, OOF < public)
• New competition type (simulation, code, quantitative)