HolobiomicsLab probability-distribution-inverse-sampling
Summary
Use inverse complementary cumulative distribution function (CDF) sampling to probabilistically select a fixed number of compounds from a scored population, where compounds with higher scaled similarity scores have proportionally higher selection probability. This enables stochastic filtering that preserves high-quality compounds while maintaining diversity across generations of reaction network expansion.
When to use
When you need to reduce a large compound set generated by reaction rule application while (1) retaining compounds ranked highest by Tanimoto similarity to target structures, (2) avoiding hard thresholds that may lose borderline-useful compounds, and (3) maintaining stochastic diversity to explore multiple regions of chemical space across multiple generations.
When NOT to use
- You require deterministic, reproducible filtering: inverse sampling is stochastic and will produce different results across runs unless random seed is fixed.
- You must guarantee that all compounds above a known threshold are retained: inverse CDF sampling can exclude high-scoring compounds if sample_size is small relative to population size.
- You lack target compound fingerprints or Tanimoto similarity scores are not meaningful for your chemical space (e.g., scaffolds with no analogues in the target set will have near-zero scores and near-zero selection probability).
Inputs
- compound set (SMILES strings or RDKit molecule objects)
- target compound fingerprints (RDKit fingerprints, typically Morgan or similar)
- sample_size (integer, number of compounds to retain)
- weighting exponent (numeric, default 4 for T^4 scaling)
Outputs
- filtered compound set (subset of input, size = sample_size)
- selection probability distribution (scaled similarities used for sampling)
How to apply
For each compound in the population, compute its maximum Tanimoto similarity score to a target compound set using RDKit fingerprints. Scale each similarity score using a weighting function (default T^4, where T is the Tanimoto coefficient) to emphasize high-similarity compounds. Compute the complementary CDF (1 - CDF) of the scaled distribution, then use inverse CDF sampling to draw sample_size compounds without replacement. Compounds with higher scaled scores have higher cumulative probability mass and thus higher selection probability, but lower-ranked compounds retain a nonzero chance of selection proportional to their score.
Related tools
- RDKit (compute Tanimoto similarity scores between compound fingerprints and apply molecular fingerprinting) — https://rdkit.org/docs/api-docs.html
- pytest (write and execute unit tests to verify correct similarity calculation, weight scaling, and inverse CDF sampling behavior) — https://docs.pytest.org/en/stable/
- Python (scipy or numpy) (implement inverse complementary CDF sampling and probability distribution operations)
Evaluation signals
- Verify that compounds with higher maximum Tanimoto similarity scores have higher selection probability under the T^4 weighting function.
- Confirm that exactly sample_size compounds are retained after filtering (no duplicates, no under/over-selection).
- Check that the complementary CDF is correctly computed from scaled scores (i.e., CDF values are monotonically increasing from 0 to 1).
- Validate that inverse sampling respects the scaled distribution: compounds selected multiple times across independent runs should show frequency proportional to their scaled score.
- Test edge cases: compounds with zero similarity, empty target set, sample_size >= population size.
Limitations
- Performance degrades for very large compound sets (>1M) due to pairwise fingerprint similarity computation; consider batch processing or approximate nearest-neighbor methods.
- The T^4 default weighting function is arbitrary and may require tuning for different chemical spaces; weak discriminators (e.g., highly similar target set) may compress the distribution and reduce effective filtering.
- Stochastic sampling introduces variance across runs; reproducibility requires fixed random seed, which may complicate parallel or distributed reaction network generation.
- Inverse CDF sampling requires fully sorted or cumulative probability computation; not suitable for streaming or online filtering scenarios.
Evidence
- [other] scales each compound's maximum Tanimoto similarity score using a weighting function (default T^4): "scales each compound's maximum Tanimoto similarity score using a weighting function (default T^4)"
- [other] apply inverse complementary CDF sampling to select N compounds (sample_size) from the scaled distribution for the next generation: "apply inverse complementary CDF sampling to select N compounds (sample_size) from the scaled distribution for the next generation"
- [intro] This tanimoto score is scaled and then the distribution is sampled by inverse complementary distribution function sampling to select N compounds: "This tanimoto score is scaled and then the distribution is sampled by inverse complementary distribution function sampling to select N compounds"
- [other] compute the maximum Tanimoto similarity for each compound to the target set using RDKit fingerprints: "compute the maximum Tanimoto similarity for each compound to the target set using RDKit fingerprints"
- [intro] Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded: "Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded"