scoring-module-unit-testing
Summary
Validate the correctness of MS2Query scoring module outputs (InChIKey score and neighbourhood score) through automated unit tests that verify score computation, output record structure, and value ranges. This skill ensures that candidate match records are properly constructed and scores fall within expected bounds before deployment.
When to use
Apply this skill after implementing or modifying the scoring module that computes average InChIKey scores and neighbourhood scores for candidate matches, or when integrating new scoring logic into an existing MS2Query pipeline. Use it as a gate before writing scored candidates to the output file to prevent propagation of malformed or out-of-range scores into downstream analyses.
When NOT to use
- Scoring module has already been validated and deployed in production with stable, frozen parameters—unit tests are not a runtime skill.
- Input candidate matches lack InChIKey annotations or spectral metadata; tests will fail for missing data, not module logic.
- You are validating end-to-end MS2Query search performance (including ranking and recall); use integration tests or benchmark datasets instead.
Inputs
- Candidate matches from library-matching step (with match identifiers and InChIKey annotations)
- Spectral similarity metrics (MS2Deepscore embeddings or related scoring data)
- Mock or real MS2 spectra with known ground-truth InChIKey labels
Outputs
- Unit test results (pass/fail, coverage report)
- Validated candidate records with InChIKey score and neighbourhood score fields
- Test assertion logs documenting score range and structural validity
How to apply
Write unit tests using Python pytest or setuptools that: (1) load mock candidate matches with known InChIKey annotations and spectral similarity metrics; (2) invoke the scoring module to compute average InChIKey score and neighbourhood score; (3) verify that output records contain all standardized fields (candidate ID, InChIKey score, neighbourhood score) with correct data types; (4) assert that computed scores fall within the valid range [0, 1]; (5) validate internal consistency (e.g., no NaN or infinity values); (6) run the full test suite before committing changes via python setup.py test. The rationale is that scoring is a critical bottleneck for match reliability—tests catch bugs in score aggregation logic, neighbourhood density calculations, or edge cases (empty candidate sets, tied scores) before results reach users.
Related tools
- pytest (Framework for writing and executing unit tests to validate scoring module logic and output schema)
- setuptools (Python packaging and test runner for executing test suite via
python setup.py test)
- MS2Query (Source tool containing the scoring module being tested) — https://github.com/iomega/ms2query
Examples
python setup.py test
Evaluation signals
- All pytest/setuptools tests pass with no assertion errors or exceptions
- Output records contain exactly the expected fields: candidate ID, InChIKey score, neighbourhood score, with correct data types (numeric for scores)
- Computed InChIKey and neighbourhood scores are in range [0.0, 1.0] and contain no NaN, infinity, or negative values
- Score aggregation logic (averaging across InChIKey layers, density computation) produces consistent results on repeated runs with identical inputs
- Test coverage includes edge cases: empty candidate sets, single candidate, tied scores, missing InChIKey annotations
Limitations
- Unit tests validate internal correctness but do not guarantee that scores reflect true spectral similarity or analogue quality; integration testing against ground-truth matched libraries is needed for that.
- Tests depend on the availability and correctness of mock data or fixture candidate matches; if fixtures are outdated or incorrect, tests will produce false negatives.
- The provided README does not detail the exact formula for InChIKey score aggregation or neighbourhood density calculation, so test expectations must be inferred from code inspection or legacy documentation.
Evidence
- [readme] make sure the existing tests still work by running
python setup.py test: "make sure the existing tests still work by running python setup.py test"
- [other] Validate output record structure and score ranges using unit tests via Python pytest or setuptools.: "Validate output record structure and score ranges using unit tests via Python pytest or setuptools."
- [other] Compute average InChIKey score for each candidate by aggregating structural similarity metrics across matched InChIKey layers.: "Compute average InChIKey score for each candidate by aggregating structural similarity metrics across matched InChIKey layers."
- [other] Combine both scores into a single candidate record with standardized fields (candidate ID, InChIKey score, neighbourhood score).: "Combine both scores into a single candidate record with standardized fields (candidate ID, InChIKey score, neighbourhood score)."
- [readme] add your own tests (if necessary): "add your own tests (if necessary)"
1---2name: scoring-module-unit-testing3description: Use when after implementing or modifying the scoring module that computes average InChIKey scores and neighbourhood scores for candidate matches, or when integrating new scoring logic into an existing MS2Query pipeline.4license: CC-BY-4.05---67# scoring-module-unit-testing89## Summary1011Validate the correctness of MS2Query scoring module outputs (InChIKey score and neighbourhood score) through automated unit tests that verify score computation, output record structure, and value ranges. This skill ensures that candidate match records are properly constructed and scores fall within expected bounds before deployment.1213## When to use1415Apply this skill after implementing or modifying the scoring module that computes average InChIKey scores and neighbourhood scores for candidate matches, or when integrating new scoring logic into an existing MS2Query pipeline. Use it as a gate before writing scored candidates to the output file to prevent propagation of malformed or out-of-range scores into downstream analyses.1617## When NOT to use1819- Scoring module has already been validated and deployed in production with stable, frozen parameters—unit tests are not a runtime skill.20- Input candidate matches lack InChIKey annotations or spectral metadata; tests will fail for missing data, not module logic.21- You are validating end-to-end MS2Query search performance (including ranking and recall); use integration tests or benchmark datasets instead.2223## Inputs2425- Candidate matches from library-matching step (with match identifiers and InChIKey annotations)26- Spectral similarity metrics (MS2Deepscore embeddings or related scoring data)27- Mock or real MS2 spectra with known ground-truth InChIKey labels2829## Outputs3031- Unit test results (pass/fail, coverage report)32- Validated candidate records with InChIKey score and neighbourhood score fields33- Test assertion logs documenting score range and structural validity3435## How to apply3637Write unit tests using Python pytest or setuptools that: (1) load mock candidate matches with known InChIKey annotations and spectral similarity metrics; (2) invoke the scoring module to compute average InChIKey score and neighbourhood score; (3) verify that output records contain all standardized fields (candidate ID, InChIKey score, neighbourhood score) with correct data types; (4) assert that computed scores fall within the valid range [0, 1]; (5) validate internal consistency (e.g., no NaN or infinity values); (6) run the full test suite before committing changes via `python setup.py test`. The rationale is that scoring is a critical bottleneck for match reliability—tests catch bugs in score aggregation logic, neighbourhood density calculations, or edge cases (empty candidate sets, tied scores) before results reach users.3839## Related tools4041- **pytest** (Framework for writing and executing unit tests to validate scoring module logic and output schema)42- **setuptools** (Python packaging and test runner for executing test suite via `python setup.py test`)43- **MS2Query** (Source tool containing the scoring module being tested) — https://github.com/iomega/ms2query4445## Examples4647```48python setup.py test49```5051## Evaluation signals5253- All pytest/setuptools tests pass with no assertion errors or exceptions54- Output records contain exactly the expected fields: candidate ID, InChIKey score, neighbourhood score, with correct data types (numeric for scores)55- Computed InChIKey and neighbourhood scores are in range [0.0, 1.0] and contain no NaN, infinity, or negative values56- Score aggregation logic (averaging across InChIKey layers, density computation) produces consistent results on repeated runs with identical inputs57- Test coverage includes edge cases: empty candidate sets, single candidate, tied scores, missing InChIKey annotations5859## Limitations6061- Unit tests validate internal correctness but do not guarantee that scores reflect true spectral similarity or analogue quality; integration testing against ground-truth matched libraries is needed for that.62- Tests depend on the availability and correctness of mock data or fixture candidate matches; if fixtures are outdated or incorrect, tests will produce false negatives.63- The provided README does not detail the exact formula for InChIKey score aggregation or neighbourhood density calculation, so test expectations must be inferred from code inspection or legacy documentation.6465## Evidence6667- [readme] make sure the existing tests still work by running ``python setup.py test``: "make sure the existing tests still work by running ``python setup.py test``"68- [other] Validate output record structure and score ranges using unit tests via Python pytest or setuptools.: "Validate output record structure and score ranges using unit tests via Python pytest or setuptools."69- [other] Compute average InChIKey score for each candidate by aggregating structural similarity metrics across matched InChIKey layers.: "Compute average InChIKey score for each candidate by aggregating structural similarity metrics across matched InChIKey layers."70- [other] Combine both scores into a single candidate record with standardized fields (candidate ID, InChIKey score, neighbourhood score).: "Combine both scores into a single candidate record with standardized fields (candidate ID, InChIKey score, neighbourhood score)."71- [readme] add your own tests (if necessary): "add your own tests (if necessary)"
HolobiomicsLab