How do I install the Bigquery Historical Data Aggregator skill?

Run npx skillmds add majiayu000/bigquery-historical-data-aggregator in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Bigquery Historical Data Aggregator safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Bigquery Historical Data Aggregator?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Bigquery Historical Data Aggregator free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Bigquery Historical Data Aggregator?

majiayu000 (@majiayu000) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Bigquery Historical Data Aggregator by majiayu000

Name: Bigquery Historical Data Aggregator
Rating: 4.2 (5 reviews)
Author: majiayu000

Bigquery Historical Data Aggregator

Aggregates and analyzes historical data from multiple BigQuery tables with similar schemas. Queries multiple tables using UNION ALL, calculates aggregate metrics (averages, sums, counts), handles table discovery via INFORMATION_SCHEMA, and processes large datasets efficiently with batch queries.

majiayu000 dd62b31 2 files · 3.6 KB Updated Sep 9, 2026 567 repo stars

File contents

Instructions

Core Workflow

Discover Tables: First, query INFORMATION_SCHEMA.TABLES to identify all available tables within the target dataset. This ensures the skill adapts to the actual table names present.
Aggregate Historical Data: Construct a SQL query that uses UNION ALL to combine data from all identified tables. Calculate the required aggregate metric (e.g., AVG(score)) grouped by the relevant key (e.g., student_id, name).
Handle Large Results: For datasets returning many rows (>50), use a batched querying strategy (e.g., LIMIT and OFFSET or filtering by key ranges) to retrieve the complete result set without truncation.
Join with Latest Data: Read the latest data from the provided source (e.g., a local CSV file). Perform a join between the aggregated historical data and the latest data to enable comparative analysis.
Calculate Deltas & Filter: Compute the percentage change or difference between historical and latest values. Apply the user-specified threshold filter (e.g., drop_percentage > 0.25).
Output Results: Write the filtered results to the specified output file (e.g., bad_student.csv).
Trigger Critical Actions: For records exceeding a higher, critical threshold (e.g., drop_percentage > 0.45), execute immediate actions such as writing critical log entries to a designated logging service.

Key Techniques

Dynamic Table Inclusion: Use the list from INFORMATION_SCHEMA to build the UNION ALL query dynamically. Do not hardcode table names.
Efficient Batch Retrieval: When the final aggregated list or intermediate results are large, retrieve data in manageable chunks using WHERE clauses on sequential keys or LIMIT/OFFSET.
Precise Percentage Calculation: Ensure the percentage change formula is correct: (historical_value - latest_value) / historical_value.
Logging for Notification: When writing critical logs, include all necessary identifiers (e.g., name, ID) and context so downstream systems can trigger alerts or notifications.

Error Handling & Validation

Confirm the target dataset exists before querying.
Verify that source files (e.g., CSV) exist and are readable.
Validate that the log bucket or destination for critical alerts exists and is accessible.

Bundled Resources

scripts/aggregate_query_template.sql: A parameterized SQL template for the core aggregation logic.
references/schema_example.md: An example schema to illustrate the expected table structure.

---name: bigquery-historical-data-aggregatordescription: Aggregates and analyzes historical data from multiple BigQuery tables with similar schemas. Queries multiple tables using UNION ALL, calculates aggregate metrics (averages, sums, counts), handles table discovery via INFORMATION_SCHEMA, and processes large datasets efficiently with batch queries.---# Instructions ## Core Workflow 1. **Discover Tables:** First, query `INFORMATION_SCHEMA.TABLES` to identify all available tables within the target dataset. This ensures the skill adapts to the actual table names present. 2. **Aggregate Historical Data:** Construct a SQL query that uses `UNION ALL` to combine data from all identified tables. Calculate the required aggregate metric (e.g., `AVG(score)`) grouped by the relevant key (e.g., `student_id`, `name`). 3. **Handle Large Results:** For datasets returning many rows (>50), use a batched querying strategy (e.g., `LIMIT` and `OFFSET` or filtering by key ranges) to retrieve the complete result set without truncation. 4. **Join with Latest Data:** Read the latest data from the provided source (e.g., a local CSV file). Perform a join between the aggregated historical data and the latest data to enable comparative analysis. 5. **Calculate Deltas & Filter:** Compute the percentage change or difference between historical and latest values. Apply the user-specified threshold filter (e.g., `drop_percentage > 0.25`). 6. **Output Results:** Write the filtered results to the specified output file (e.g., `bad_student.csv`). 7. **Trigger Critical Actions:** For records exceeding a higher, critical threshold (e.g., `drop_percentage > 0.45`), execute immediate actions such as writing critical log entries to a designated logging service. ## Key Techniques * **Dynamic Table Inclusion:** Use the list from `INFORMATION_SCHEMA` to build the `UNION ALL` query dynamically. Do not hardcode table names. * **Efficient Batch Retrieval:** When the final aggregated list or intermediate results are large, retrieve data in manageable chunks using `WHERE` clauses on sequential keys or `LIMIT/OFFSET`. * **Precise Percentage Calculation:** Ensure the percentage change formula is correct: `(historical_value - latest_value) / historical_value`. * **Logging for Notification:** When writing critical logs, include all necessary identifiers (e.g., name, ID) and context so downstream systems can trigger alerts or notifications. ## Error Handling & Validation * Confirm the target dataset exists before querying. * Verify that source files (e.g., CSV) exist and are readable. * Validate that the log bucket or destination for critical alerts exists and is accessible. ## Bundled Resources * `scripts/aggregate_query_template.sql`: A parameterized SQL template for the core aggregation logic. * `references/schema_example.md`: An example schema to illustrate the expected table structure. ---

Frequently asked questions