Github Trending Scraper

GitHub Trending Scraper

Scrape GitHub Trending repositories across different time periods (daily, weekly, monthly) and save structured data to JSON files.

Usage

Run the scraper script to fetch trending repositories:

python scripts/github_trending_scraper.py

The script will:

1. Scrape trending repositories for all three periods (daily, weekly, monthly)
2. Extract project metadata: name, URL, description, language, stars, forks
3. Save results to github_trending.json in the current directory
4. Display statistics about scraped projects

Output Format

The generated JSON file contains:

{
  "update_time": "2026-02-09 10:30:00",
  "data": {
    "daily": [...],
    "weekly": [...],
    "monthly": [...]
  }
}

Each project includes:

• name: Repository full name (owner/repo)
• url: GitHub repository URL
• description: Project description
• language: Primary programming language
• stars: Total star count
• forks: Total fork count
• stars_daily/weekly/monthly: New stars in the period

Dependencies

Install required packages:

pip install requests beautifulsoup4

Or use the provided requirements file:

pip install -r requirements.txt

Customization

To scrape only specific periods, modify the periods list in the main() function of the script.

To change the output filename, modify the output_file variable in the script.