descriptor-deduplication-across-toolkits
License: restricted — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution.
Summary
Remove redundant molecular descriptors that exist in both RDKit and mordred libraries, retaining only the RDKit version to create a unified, non-redundant descriptor feature set for machine learning. This deduplication step is essential when combining multiple descriptor calculation toolkits to avoid feature duplication and reduce dimensionality in molecular featurization pipelines.
When to use
When computing molecular descriptors from SMILES strings using both RDKit and mordred libraries simultaneously, and you need to combine their outputs into a single feature matrix without redundant features. Specifically, apply this skill after calculating descriptors from both libraries but before concatenating them into a unified feature array for downstream machine learning tasks.
When NOT to use
- Input is already a single-source descriptor matrix (only RDKit or only mordred, not both) — deduplication is unnecessary
- Downstream analysis requires traceability of descriptor source or comparison of toolkit-specific outputs — concatenating deduplicated descriptors may obscure source provenance
- Memory or storage constraints do not permit retaining both RDKit and mordred descriptors in any form — use only one toolkit instead
Inputs
- SMILES strings (input molecular identifiers)
- RDKit descriptor array (NumPy array of descriptor values computed by RDKit)
- mordred descriptor array (NumPy array of descriptor values computed by mordred Calculator)
- Descriptor name lists from both RDKit and mordred libraries
Outputs
- Deduplicated mordred descriptor indices or array
- Unified descriptor feature matrix (concatenated RDKit + non-duplicate mordred descriptors)
- Descriptor name mapping (list of final descriptor identifiers post-deduplication)
How to apply
After calculating descriptor sets from both RDKit (using RDKit's descriptor module) and mordred (using mordred's Calculator with the full descriptor set), identify descriptors that appear in both libraries by name. Remove the mordred versions of these duplicate descriptors using a deduplication function (e.g., remove_mordred_duplicates), keeping only the RDKit versions since RDKit descriptors are the preferred source. This approach preserves chemical information coverage while eliminating collinearity and reducing memory footprint. The rationale is that RDKit is the more established and widely validated toolkit in cheminformatics; retaining its descriptors prevents potential method-specific bias from mordred while maintaining comprehensive molecular property coverage. Finally, concatenate the retained RDKit descriptors with the deduplicated (non-duplicate) mordred descriptors into a single feature matrix.
Related tools
- RDKit (Computes molecular descriptors from SMILES; serves as the preferred descriptor source for deduplicated features)
- mordred (Computes alternative set of molecular descriptors; sources are filtered to remove duplicates of RDKit descriptors)
- NumPy (Handles array manipulation, concatenation, and storage of descriptor matrices in .npz compressed format)
- pandas (Optional: facilitates descriptor metadata management and export to .csv/.txt formats for validation)
Examples
cleared_mordred_descriptors, duplicate_indices = remove_mordred_duplicates(mordred_descriptors_array, rdkit_descriptors_array); final_features = np.concatenate([rdkit_descriptors_array, cleared_mordred_descriptors], axis=1)
Evaluation signals
- Verify that the output descriptor count equals (unique RDKit descriptors) + (non-duplicate mordred descriptors) — check no descriptors appear twice
- Confirm all RDKit descriptor names are present in the final deduplicated list and all mordred names in the final list have no RDKit counterpart by exact string matching
- Validate output array shape: rows = number of molecules, columns = count of unique descriptors across both libraries
- Check that deduplicated mordred descriptor indices match expected positions after removal (e.g., if 50 of 500 mordred descriptors are duplicates, retain 450)
- Spot-check a sample of deduplicated descriptor values to ensure they match the original RDKit and mordred arrays at their respective source indices
Limitations
- Duplicate detection relies on exact name string matching; descriptors with similar names but different calculations (e.g., 'NumRotatableBonds' vs 'RotatableBonds') may not be identified as duplicates
- Retaining only RDKit versions of duplicates assumes RDKit implementations are correct and comparable; this assumption may fail if RDKit and mordred use different normalization or calculation logic for the same descriptor
- No mechanism is provided in the article to handle conflicting or discrepant descriptor values if both libraries produce different results for the same molecular property
- The deduplication function (
remove_mordred_duplicates) implementation details are not fully specified, leaving interpretation of matching logic to the practitioner
Evidence
- [results] Calculate RDKit descriptors using RDKit's descriptor module. 3. Calculate mordred descriptors using the mordred Calculator with the full descriptor set. 4. Remove mordred descriptors that duplicate RDKit descriptors using remove_mordred_duplicates function.: "Calculate RDKit descriptors using RDKit's descriptor module. 3. Calculate mordred descriptors using the mordred Calculator with the full descriptor set. 4. Remove mordred descriptors that duplicate"
- [results] The Calc_Descriptors routine retrieves descriptor names from both RDKit and mordred libraries, removes duplicate descriptors that exist in both libraries (keeping RDKit versions), and combines them with extra custom headers to create a unified descriptor feature set: "removes duplicate descriptors that exist in both libraries (keeping RDKit versions), and combines them with extra custom headers to create a unified descriptor feature set"
- [results] Removal of Mordred descriptors that duplicate RDKit descriptors [section=results; evidence='cleared_mordred_descriptors, duplicate_indeces = remove_mordred_duplicates(cleared_mordred_descriptors, rdkit_descriptors)']: "cleared_mordred_descriptors, duplicate_indeces = remove_mordred_duplicates(cleared_mordred_descriptors, rdkit_descriptors)"
- [results] Concatenate RDKit and deduplicated mordred descriptor arrays into a single feature matrix.: "Concatenate RDKit and deduplicated mordred descriptor arrays into a single feature matrix."
1---2name: descriptor-deduplication-across-toolkits3description: Use when when computing molecular descriptors from SMILES strings using both RDKit and mordred libraries simultaneously, and you need to combine their outputs into a single feature matrix without redundant features.4license: CC-BY-4.05---67# descriptor-deduplication-across-toolkits89> **License: restricted** — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->10## Summary1112Remove redundant molecular descriptors that exist in both RDKit and mordred libraries, retaining only the RDKit version to create a unified, non-redundant descriptor feature set for machine learning. This deduplication step is essential when combining multiple descriptor calculation toolkits to avoid feature duplication and reduce dimensionality in molecular featurization pipelines.1314## When to use1516When computing molecular descriptors from SMILES strings using both RDKit and mordred libraries simultaneously, and you need to combine their outputs into a single feature matrix without redundant features. Specifically, apply this skill after calculating descriptors from both libraries but before concatenating them into a unified feature array for downstream machine learning tasks.1718## When NOT to use1920- Input is already a single-source descriptor matrix (only RDKit or only mordred, not both) — deduplication is unnecessary21- Downstream analysis requires traceability of descriptor source or comparison of toolkit-specific outputs — concatenating deduplicated descriptors may obscure source provenance22- Memory or storage constraints do not permit retaining both RDKit and mordred descriptors in any form — use only one toolkit instead2324## Inputs2526- SMILES strings (input molecular identifiers)27- RDKit descriptor array (NumPy array of descriptor values computed by RDKit)28- mordred descriptor array (NumPy array of descriptor values computed by mordred Calculator)29- Descriptor name lists from both RDKit and mordred libraries3031## Outputs3233- Deduplicated mordred descriptor indices or array34- Unified descriptor feature matrix (concatenated RDKit + non-duplicate mordred descriptors)35- Descriptor name mapping (list of final descriptor identifiers post-deduplication)3637## How to apply3839After calculating descriptor sets from both RDKit (using RDKit's descriptor module) and mordred (using mordred's Calculator with the full descriptor set), identify descriptors that appear in both libraries by name. Remove the mordred versions of these duplicate descriptors using a deduplication function (e.g., `remove_mordred_duplicates`), keeping only the RDKit versions since RDKit descriptors are the preferred source. This approach preserves chemical information coverage while eliminating collinearity and reducing memory footprint. The rationale is that RDKit is the more established and widely validated toolkit in cheminformatics; retaining its descriptors prevents potential method-specific bias from mordred while maintaining comprehensive molecular property coverage. Finally, concatenate the retained RDKit descriptors with the deduplicated (non-duplicate) mordred descriptors into a single feature matrix.4041## Related tools4243- **RDKit** (Computes molecular descriptors from SMILES; serves as the preferred descriptor source for deduplicated features)44- **mordred** (Computes alternative set of molecular descriptors; sources are filtered to remove duplicates of RDKit descriptors)45- **NumPy** (Handles array manipulation, concatenation, and storage of descriptor matrices in .npz compressed format)46- **pandas** (Optional: facilitates descriptor metadata management and export to .csv/.txt formats for validation)4748## Examples4950```51cleared_mordred_descriptors, duplicate_indices = remove_mordred_duplicates(mordred_descriptors_array, rdkit_descriptors_array); final_features = np.concatenate([rdkit_descriptors_array, cleared_mordred_descriptors], axis=1)52```5354## Evaluation signals5556- Verify that the output descriptor count equals (unique RDKit descriptors) + (non-duplicate mordred descriptors) — check no descriptors appear twice57- Confirm all RDKit descriptor names are present in the final deduplicated list and all mordred names in the final list have no RDKit counterpart by exact string matching58- Validate output array shape: rows = number of molecules, columns = count of unique descriptors across both libraries59- Check that deduplicated mordred descriptor indices match expected positions after removal (e.g., if 50 of 500 mordred descriptors are duplicates, retain 450)60- Spot-check a sample of deduplicated descriptor values to ensure they match the original RDKit and mordred arrays at their respective source indices6162## Limitations6364- Duplicate detection relies on exact name string matching; descriptors with similar names but different calculations (e.g., 'NumRotatableBonds' vs 'RotatableBonds') may not be identified as duplicates65- Retaining only RDKit versions of duplicates assumes RDKit implementations are correct and comparable; this assumption may fail if RDKit and mordred use different normalization or calculation logic for the same descriptor66- No mechanism is provided in the article to handle conflicting or discrepant descriptor values if both libraries produce different results for the same molecular property67- The deduplication function (`remove_mordred_duplicates`) implementation details are not fully specified, leaving interpretation of matching logic to the practitioner6869## Evidence7071- [results] Calculate RDKit descriptors using RDKit's descriptor module. 3. Calculate mordred descriptors using the mordred Calculator with the full descriptor set. 4. Remove mordred descriptors that duplicate RDKit descriptors using remove_mordred_duplicates function.: "Calculate RDKit descriptors using RDKit's descriptor module. 3. Calculate mordred descriptors using the mordred Calculator with the full descriptor set. 4. Remove mordred descriptors that duplicate"72- [results] The Calc_Descriptors routine retrieves descriptor names from both RDKit and mordred libraries, removes duplicate descriptors that exist in both libraries (keeping RDKit versions), and combines them with extra custom headers to create a unified descriptor feature set: "removes duplicate descriptors that exist in both libraries (keeping RDKit versions), and combines them with extra custom headers to create a unified descriptor feature set"73- [results] Removal of Mordred descriptors that duplicate RDKit descriptors [section=results; evidence='cleared_mordred_descriptors, duplicate_indeces = remove_mordred_duplicates(cleared_mordred_descriptors, rdkit_descriptors)']: "cleared_mordred_descriptors, duplicate_indeces = remove_mordred_duplicates(cleared_mordred_descriptors, rdkit_descriptors)"74- [results] Concatenate RDKit and deduplicated mordred descriptor arrays into a single feature matrix.: "Concatenate RDKit and deduplicated mordred descriptor arrays into a single feature matrix."
HolobiomicsLab