Tracking List

AI News Tracking - Unified Specification

This document defines the shared specification for scoring, formatting, and validating AI news items.

Tracking Type Definitions

This tracking system categorizes AI industry updates into 4 types. Each record must be labeled with its type. The main Agent can exclude unwanted types via exclude_types (All tracked by default).

Type 1: Product

Feature/version updates for AI tools and platforms.

Source Priority: Changelog/Release Notes > GitHub Releases > Official Blog > Official X > Key People X

Key People Post Handling:

• Feature/version release → must cross-verify with Changelog
• Tips sharing ("I recently discovered...", "pro tip") → not included
• Announcement outside window but feature landed within window → use feature landing as primary, announcement as background

Type 2: Model

AI model releases, updates, and open-sourcing (including LLM, vision models, multimodal models).

Source Priority: Official Blog > Official X > API Changelog > GitHub/HuggingFace > Key People X > arXiv

Type 3: Benchmark

Benchmark leaderboard changes, benchmark results, academic papers, technical reports.

Source Priority: Benchmark institutions > arXiv > Official Research > HuggingFace Papers > KOL > Papers with Code > Reddit

Type 4: Funding

Major funding, acquisitions/mergers, strategic partnerships, milestone events.

Source Priority: Bilateral official confirmation > Crunchbase/PitchBook > Authoritative media (TechCrunch/The Verge)

Verification Rules:

Source Standards

Source Priority (General)

Factual Detail Verification (CRITICAL)

Core Rule: Every specific number or technical detail in the report MUST be traceable to an authoritative primary source. Never infer, extrapolate, or "fill in" details from memory or general knowledge.

Specific details that MUST be verified from primary sources before inclusion:

Verification protocol:

1. Check the primary source first — For models: read the HuggingFace model card or official blog. For products: read the changelog/release notes. For benchmarks: check the benchmark site.
2. If a detail cannot be verified — OMIT it entirely rather than guessing. Write "parameters not yet disclosed" or simply leave the metric out of the Key Data table. An absent detail is always better than a wrong one.
3. Never rely on your training data for specific numbers — Model knowledge may be outdated, conflated between similar models, or simply wrong. Always verify against a live source.
4. Cross-check confusable details — Many model families have similar names with different specs (e.g., Qwen2.5-72B vs Qwen3-235B). Verify that the number matches the exact model version being discussed.
5. Flag uncertainty explicitly — If a source gives conflicting numbers (e.g., blog says one thing, model card says another), note the discrepancy rather than picking one silently.

Prohibited Actions

• Using news sites as primary source (for lead discovery only)
• Using search results as main information source
• Including secondary source information without tracing verification
• Writing specific numbers (parameter counts, benchmark scores, pricing, etc.) without verifying from an authoritative primary source
• Filling in technical details from model memory/training data instead of checking the actual source

Cross-verification Rules

Timeliness Double-check Rules

For the following scenarios, a double-check is required (confirm the original event date through additional searches or official timelines):

1. No clear date annotation: Page content does not show a specific publish date → search "{product name} release date" or "{product name} announced" to confirm
2. Only third-party sources: Event only reported on third-party media/platforms, no official primary source → trace to official announcement to confirm date
3. Persistent state information: Leaderboard rankings, product pricing, feature availability, etc. → confirm the date of the first state change, not the current state
4. Vague time words like "recently"/"this month": Source uses "recently", "this month", etc. → cannot be accepted directly, must find precise date
5. Date anomaly: Event seems too major but wasn't covered by mainstream media within window → likely an old event, requires additional verification

Events where date cannot be confirmed → downgrade to skip, annotate "date cannot be confirmed"

Timeliness Validation Rules

Core Principle: Event date ≠ Page date

Key Rule: must confirm the "actual event date", not "page accessible date" or "page last updated date".

A page being currently accessible does not mean the event it describes occurred within the window. Common misjudgment scenarios:

Determination Process:

1. Find the event's original announcement (official blog, Release Notes, first X post)
2. Extract that announcement's publish date (not page last-modified)
3. Convert that date to UTC+8
4. Determine whether it falls within [time_window_start, time_window_end)
5. If the original date cannot be confirmed → do not include, record in skipped records and annotate "date cannot be confirmed"

Time Window

• Standard Window: [yesterday 08:00, today 08:00) UTC+8
• Window Length: 24 hours

Time Conversion

Date Extraction Rules by Source Type

Time Determination

Timeliness Check:
- ✅ Within window: Published Time ∈ [time_window_start, time_window_end)
- ❌ Outside window: Published Time < time_window_start or >= time_window_end

RT/Quote Handling

• Retweets (RT) and Quotes must trace to original post time
• Use original publish time for determination of whether it falls within window
• Retweet time within window ≠ original fact within window

Scoring Criteria

Two-Stage Scoring

Morning-AI uses a two-stage scoring pipeline:

Stage 1 — Automated scoring (collect.py → lib/score.py): Computes a 1-10 initial score from quantifiable metadata using 4 dimensions:

• Relevance (35%) — keyword/entity match strength from collector
• Engagement (30%) — platform-specific metrics (likes, stars, upvotes)
• Source Reliability (20%) — source tier weight (GitHub 0.9 → Reddit 0.5)
• Recency (15%) — date confidence level

Stage 2 — Agent evaluation (report generation): The agent reviews each item using the 5 qualitative dimensions below. The agent may adjust the Stage 1 score based on content understanding — e.g., a low-engagement but groundbreaking paper might be scored up, while a viral but trivial post might be scored down.

Scoring Dimensions (Stage 2 — Agent Evaluation)

Score Levels

Scoring Reference by Type

Model Scoring Reference

Product Scoring Reference

Benchmark/Paper Scoring Reference

Funding Scoring Reference

Scoring Factors

Draft Record Format

Valid Record Format

### {Entity name} - {Event description}

| Field | Value |
|-------|-------|
| **Type Label** | Product / Model / Benchmark / Funding |
| **Timeliness Check** | ✅ Within window / ❌ Outside window |
| **Published Time** | YYYY-MM-DD HH:MM UTC+8 |
| **Event Type** | New feature / New model / Version update / Capability upgrade / Open-source release / Leaderboard change / Academic paper / Funding / Acquisition / Major partnership / Milestone / ... |
| **Partner/Acquirer** | (for Funding type) XX Company |
| **Amount/Scale** | (for Funding type) $XM / Series X / XM users |
| **Source** | [Source Name](URL) |
| **Score** | X.X |

**Summary**:
- Key point 1 (include specific details: version numbers, parameter counts, percentage improvements, pricing, availability)
- Key point 2 (competitive comparison or positioning)
- Key point 3 (technical specs or architecture details)
- Key point 4 (availability, rollout timeline, or ecosystem impact)
- Key point 5 (additional context as needed)
- (9-10 scores: 5-8 bullet points; 7-8 scores: 4-6 bullet points; 5-6 scores: 3-4 bullet points — cover all important aspects)

**Why It Matters** (required for 7+ scores):
> 1-4 sentence analysis of industry impact, competitive significance, or user implications. For 9-10 scores use 2-4 sentences with strategic context; for 7-8 scores use 1-2 sentences. Explain what this changes for the industry or end users — don't just restate what happened.

**Key Data** (required for 7+ scores when quantitative data exists — include when quantitative metrics are available):
| Metric | Value |
|--------|-------|
| e.g. Benchmark score | e.g. 92.3% (+5.1% vs previous SOTA) |
| e.g. Parameters | e.g. 671B total / 37B active |
| e.g. Pricing | e.g. $3/M input, $15/M output (vs $5/M previous) |
| e.g. Context length | e.g. 1M tokens (+4x vs v3) |
| e.g. Funding amount | e.g. $500M Series C at $5B valuation |

**Multi-source Verification** (required for 7+):
- [Source 1](URL)
- [Source 2](URL)

Academic Paper Record Format (Benchmark type)

### {Entity name} - {Paper title}

| Field | Value |
|-------|-------|
| **Type Label** | Benchmark |
| **Timeliness Check** | ✅ Within window / ❌ Outside window |
| **Published Time** | YYYY-MM-DD HH:MM UTC+8 |
| **Event Type** | Academic paper / Technical report / Interpretability research |
| **arXiv** | [XXXX.XXXXX](https://arxiv.org/abs/XXXX.XXXXX) |
| **GitHub** | [Org/Repo](URL) (if available) |
| **Source** | [Source Name](URL) |
| **Score** | X.X |

**Core Innovation**:
> One-sentence description of the paper's core innovation

**Research Significance**:
> Who is impacted? What has changed? What does this enable that wasn't possible before? Include concrete implications for practitioners or downstream applications.

**Key Data**:
| Metric | Value |
|--------|-------|
| | |

Skipped Record Format

| Entity | Summary | Skip Reason | Source |
|--------|---------|-------------|--------|
| [Entity name] | [Content description] | Pure marketing / Unconfirmed rumor / Outside window / Duplicate coverage / Tips sharing | [link](url) |

Mid-Score Compact Format (5-6)

- **Entity** (X.X): Event description with specifics (version, capability, metric).
  - Detail 1: what changed, key numbers, comparison with previous version or competitors
  - Detail 2: additional context, availability, or technical specifics
  - Detail 3: implications or notable aspects
  Source: [Name](URL)

Lower-Score Compact Format (3-4)

Use compact table format. The Source column must contain clickable [Name](URL) links.

| Entity | Score | Event | Source |
|--------|-------|-------|--------|
| Entity name | X.X | Brief description with one key detail | [Name](URL) |

Workflow Specification

Data Collection Workflow

FOR each source:
    1️⃣ Check source (X account, Changelog/Blog, GitHub Releases, arXiv)
    2️⃣ Timeliness check (per time validation rules)
    3️⃣ Cross-verification (key people posts require cross-verification with official channels)
    4️⃣ Content classification → determine type label (Product/Model/Benchmark/Funding)
    5️⃣ Valid content → record with full format
    6️⃣ Irrelevant content → record in skipped items with reason
END FOR

Type Classification Guide

When a piece of information may belong to multiple types, classify by the following priority:

Key Checkpoints

• All source Checkboxes checked
• All records contain complete format and correct type labels
• 7+ scores have multi-source verification
• Funding events have bilateral confirmation
• Completion rate = 100%

Notes

1. Type label required - Each record must be labeled with type (Product/Model/Benchmark/Funding)
2. Record as you check - Append immediately upon discovery to the corresponding type section; don't backfill
3. Strict timeliness - Content outside window is not included
4. Cross-verification - 7+ must have multi-source confirmation
5. Funding events need bilateral confirmation - Unilateral announcement with no response from counterparty should be downgraded
6. Complete records - Including skipped content
7. 100% completion rate - All Checkboxes must be checked
8. Dynamic adaptation - Entity list and scoring references should evolve with the industry