tools-only Detecting Data Anomalies
Overview
This skill provides automated assistance for the described functionality.
Prerequisites
Before using this skill, ensure you have:
- Dataset in accessible format (CSV, JSON, or database)
- Python environment with scikit-learn or similar ML libraries
- Understanding of data distribution and expected patterns
- Sufficient data volume for statistical significance
- Knowledge of domain-specific normal behavior
- Data preprocessing capabilities for cleaning and scaling
Instructions
- Load dataset using Read tool
- Inspect data structure and identify relevant features
- Clean data by handling missing values and inconsistencies
- Normalize or scale features as appropriate for algorithm
- Split temporal data if time-series analysis is needed
- Apply selected algorithm using Bash tool
- Generate anomaly scores for each data point
- Classify points as normal or anomalous based on threshold
- Extract characteristics of identified anomalies
See {baseDir}/references/implementation.md for detailed implementation guide.
Output
- Total data points analyzed
- Number of anomalies detected
- Contamination rate (percentage of anomalies)
- Algorithm used and configuration parameters
- Confidence scores for detected anomalies
- Record identifier and timestamp (if applicable)
Error Handling
See {baseDir}/references/errors.md for comprehensive error handling.
Examples
See {baseDir}/references/examples.md for detailed examples.
Resources
- Isolation Forest documentation and implementation examples
- One-Class SVM for novelty detection
- Local Outlier Factor (LOF) for density-based detection
- Autoencoder-based anomaly detection for deep learning approaches
- scikit-learn anomaly detection module