# Chemical Reaction Network Evaluation

> Use when you have a rule-based reaction prediction system (like Pickaxe) with a large rule set (e.g., ~1221 MetaCyc generalized rules), and you need to understand the scaling relationship between rule count and reaction coverage to decide how many rules to deploy for a specific application.

- Skill: `holobiomicslab/chemical-reaction-network-evaluation` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/chemical-reaction-network-evaluation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/chemical-reaction-network-evaluation/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/chemical-reaction-network-evaluation

---


# chemical-reaction-network-evaluation

## Summary

Evaluate the empirical relationship between the number of generalized reaction rules applied and the coverage percentage of a reference reaction database (e.g., MetaCyc). This skill quantifies the trade-off between computational cost and reaction prediction completeness by systematically measuring how many distinct reactions can be represented as rule subsets grow.

## When to use

You have a rule-based reaction prediction system (like Pickaxe) with a large rule set (e.g., ~1221 MetaCyc generalized rules), and you need to understand the scaling relationship between rule count and reaction coverage to decide how many rules to deploy for a specific application. Use this when you want to balance prediction scope against runtime or memory constraints.

## When NOT to use

- You only have a small, domain-specific rule set (<50 rules) where coverage plateaus immediately; scaling analysis is less informative.
- Your reference database is not well-defined or lacks quantitative reaction annotations; you cannot reliably measure coverage.
- You need real-time or online rule selection; this is a one-time characterization study, not a live optimization approach.

## Inputs

- Full generalized reaction rule set (SMARTS strings, ~1221 rules for MetaCyc)
- Representative compound set (SMILES strings or structure identifiers)
- Reference reaction database (MetaCyc reactions to match against)

## Outputs

- Coverage table (TSV format): (number_of_rules, coverage_percent) pairs
- Scaling relationship plot or summary statistics
- Rule subset definitions (ranked or frequency-ordered)

## How to apply

Load the full generalized reaction rule set from a database (e.g., the ~70,000 MetaCyc reactions condensed into ~1221 generic SMARTS rules). Select ordered rule subsets of increasing size (e.g., 20, 100, 272, 500, 956, 1221 rules) using rule importance or frequency-based ranking. For each subset, apply the rules to a representative set of compounds from the target database and compute the percentage of total reactions in the reference database that can be represented. Aggregate the (number_of_rules, coverage_percent) pairs into a table and verify the coverage relationship follows an expected scaling curve (e.g., diminishing returns at higher rule counts). The scaling curve informs optimal rule selection for downstream applications.

## Related tools

- **RDKit** (Apply SMARTS reaction rules to compounds; compute molecular similarity and filter compounds by structure) — https://rdkit.org/docs/api-docs.html
- **Python** (Implement rule selection logic, iteration over rule subsets, coverage computation, and table aggregation)
- **MINE-Database (Pickaxe)** (Load and apply generalized reaction rules; predict reactions from starting compounds) — https://github.com/tyo-nu/MINE-Database
- **MongoDB or TSV export** (Store or serialize rule subsets and coverage results for analysis)

## Evaluation signals

- Coverage percentages increase monotonically with rule count (no non-monotonic dips).
- Coverage curve reaches 100% when all rules are included; no residual uncovered reactions.
- Intermediate coverage values match published reference values (e.g., 272 rules = 90% coverage for MetaCyc) to within ±2%.
- Rule subset sizes and coverage percentages are reproducible across multiple runs with the same ranking strategy.
- Output TSV file is well-formed with headers (number_of_rules, coverage_percent) and matches the row count of rule subsets tested.

## Limitations

- Scaling relationship is specific to the reference database and rule set used; results for MetaCyc may not generalize to KEGG or custom rule sets.
- Coverage measurement depends on the representativeness of the compound set tested; biased or incomplete compound selection will underestimate or overestimate coverage.
- Rule importance/frequency ranking is heuristic; different ranking strategies (e.g., by cofactor requirement, reaction class) may yield different scaling curves.
- The study characterizes average-case coverage; some compound subsets or reaction classes may have coverage profiles that deviate significantly from the aggregate curve.

## Evidence

- [intro] Scaling relationship between rule count and coverage: "20 rules provide 50% coverage, 100 rules provide 78% coverage, 272 rules provide 90% coverage, 500 rules provide 95% coverage, 956 rules provide 99% coverage, and all 1221 rules provide 100% coverage"
- [intro] MetaCyc rule set composition: "approximately 70,000 MetaCyc reactions condensed into generic rules"
- [intro] Rule application methodology: "Pickaxe applies reaction rules, representing reaction transformation patterns, to a list of user-specified compounds in order to predict reactions"
- [intro] Rule representation format: "Rules are generated using `SMARTS` which represent reactions in a string"
- [intro] Output format and workflow: "There are two ways to output data: 1. Writing to a mongo database that is specified by a `mongo uri`, either local or in mongo_uri.csv 2. Local .tsv files"