# Scikit Learn

> Machine learning library providing simple and efficient tools for predictive data analysis including classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.

- Skill: `ffsshhttiikk/scikit-learn` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ffsshhttiikk/scikit-learn`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ffsshhttiikk/scikit-learn/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: ffsshhttiikk (https://skillmd.com/u/ffsshhttiikk)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ffsshhttiikk/scikit-learn

---


# Scikit-learn

## What I do

I provide comprehensive machine learning capabilities for building predictive models. I offer tools for supervised learning (classification, regression), unsupervised learning (clustering, dimensionality reduction), model evaluation, selection, and preprocessing. My consistent API design makes it easy to experiment with different algorithms and build production-ready ML pipelines.

## When to use me

- Building classification models (logistic regression, SVM, random forest, neural networks)
- Building regression models (linear, ridge, lasso, tree-based, ensemble)
- Clustering data (k-means, hierarchical, DBSCAN, GMM)
- Dimensionality reduction (PCA, t-SNE, UMAP, factor analysis)
- Feature selection and extraction
- Model evaluation with cross-validation
- Hyperparameter tuning
- Building reproducible ML pipelines

## Core Concepts

### Estimators
- **Supervised**: `Classifier` (predicts class labels) and `Regressor` (predicts continuous values)
- **Unsupervised**: `Clusterer` (groups data), `DimensionalityReducer` (reduces features)
- **Consistent API**: `fit()`, `predict()`, `transform()` methods

### Model Selection
- **Cross-validation**: K-fold, stratified, leave-one-out
- **Splitting**: `train_test_split()`, `KFold`, `StratifiedKFold`
- **Hyperparameter tuning**: `GridSearchCV`, `RandomizedSearchCV`
- **Metrics**: Accuracy, precision, recall, F1, ROC-AUC, MSE, R²

### Preprocessing
- **Scaling**: `StandardScaler`, `MinMaxScaler`, `RobustScaler`
- **Encoding**: `OneHotEncoder`, `LabelEncoder`, `OrdinalEncoder`
- **Imputation**: `SimpleImputer`, `KNNImputer`, `IterativeImputer`
- **Feature extraction**: `CountVectorizer`, `TfidfVectorizer`

### Pipelines
- **Pipeline**: Chain transformations and model
- **ColumnTransformer**: Apply different transforms to different columns
- **FeatureUnion**: Concatenate features from multiple transformers

### Ensemble Methods
- **Bagging**: `BaggingClassifier`, `RandomForest`
- **Boosting**: `AdaBoost`, `GradientBoosting`, `XGBoost` integration
- **Voting/Stacking**: `VotingClassifier`, `StackingClassifier`

## Code Examples (Python)

```python
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.preprocessing import StandardScaler, LabelEncoder, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.compose import ColumnTransformer
from sklearn.metrics import (accuracy_score, precision_score, recall_score, 
                             f1_score, roc_auc_score, confusion_matrix,
                             mean_squared_error, r2_score, classification_report)

from sklearn.linear_model import LogisticRegression, LinearRegression, Ridge
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
from sklearn.ensemble import (RandomForestClassifier, RandomForestRegressor,
                             GradientBoostingClassifier, AdaBoostClassifier)
from sklearn.svm import SVC, SVR
from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor
from sklearn.naive_bayes import GaussianNB
from sklearn.cluster import KMeans, AgglomerativeClustering, DBSCAN
from sklearn.decomposition import PCA, TruncatedSVD
from sklearn.manifold import TSNE

# Data preparation
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Preprocessing pipeline
numeric_features = ['age', 'income', 'score']
categorical_features = ['category', 'city']

preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features),
        ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_features)
    ])

# Create pipeline
pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])

# Fit and predict
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)
y_proba = pipeline.predict_proba(X_test)[:, 1]

# Evaluation
accuracy = accuracy_score(y_test, y_pred)
precision = precision_score(y_test, y_pred, average='weighted')
recall = recall_score(y_test, y_pred, average='weighted')
f1 = f1_score(y_test, y_pred, average='weighted')
roc_auc = roc_auc_score(y_test, y_proba)
cm = confusion_matrix(y_test, y_pred)
print(classification_report(y_test, y_pred))

# Cross-validation
cv_scores = cross_val_score(pipeline, X, y, cv=5, scoring='accuracy')

# Hyperparameter tuning
param_grid = {
    'classifier__n_estimators': [100, 200],
    'classifier__max_depth': [5, 10, None],
    'classifier__min_samples_split': [2, 5]
}
grid_search = GridSearchCV(pipeline, param_grid, cv=5, scoring='accuracy', n_jobs=-1)
grid_search.fit(X_train, y_train)
print(f"Best params: {grid_search.best_params_}")
print(f"Best CV score: {grid_search.best_score_}")

# Regression example
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
regression_pipeline = Pipeline([
    ('preprocessor', StandardScaler()),
    ('regressor', Ridge(alpha=1.0))
])
regression_pipeline.fit(X_train, y_train)
y_pred = regression_pipeline.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

# Clustering
kmeans = KMeans(n_clusters=3, random_state=42)
clusters = kmeans.fit_predict(X_scaled)
silhouette = silhouette_score(X_scaled, clusters)

# Dimensionality reduction
pca = PCA(n_components=0.95)  # Keep 95% variance
X_pca = pca.fit_transform(X_scaled)

# Using different classifiers
classifiers = {
    'Logistic Regression': LogisticRegression(max_iter=1000),
    'Decision Tree': DecisionTreeClassifier(max_depth=5),
    'Random Forest': RandomForestClassifier(n_estimators=100),
    'SVM': SVC(kernel='rbf', probability=True),
    'KNN': KNeighborsClassifier(n_neighbors=5),
    'Naive Bayes': GaussianNB()
}

for name, clf in classifiers.items():
    clf.fit(X_train_scaled, y_train)
    score = clf.score(X_test_scaled, y_test)
    print(f"{name}: {score:.4f}")
```

## Best Practices

1. **Always split data**: Use `train_test_split` before any preprocessing to avoid data leakage.

2. **Use pipelines**: Encapsulate preprocessing and modeling to prevent leakage and ensure reproducibility.

3. **Scale appropriately**: Normalize/standardize features for distance-based algorithms (SVM, KNN, neural networks).

4. **Handle class imbalance**: Use `class_weight='balanced'`, SMOTE, or appropriate metrics.

5. **Cross-validate**: Use CV for robust performance estimation, not single train-test split.

6. **Feature engineering**: Engineer features before feeding to models; pipelines make this easier.

7. **Avoid data leakage**: Never use test data for feature selection or model selection.

8. **Use appropriate metrics**: Accuracy is misleading for imbalanced data; use precision, recall, F1, AUC.

## Common Patterns

### Pattern 1: Complete ML Workflow
```python
def train_ml_model(X, y, model_type='classifier'):
    # Split data
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, random_state=42, stratify=y
    )
    
    # Create pipeline
    pipeline = Pipeline([
        ('scaler', StandardScaler()),
        ('imputer', SimpleImputer(strategy='median')),
        ('classifier', RandomForestClassifier(n_estimators=100))
    ])
    
    # Cross-validation
    cv_scores = cross_val_score(pipeline, X_train, y_train, cv=5)
    
    # Fit and evaluate
    pipeline.fit(X_train, y_train)
    y_pred = pipeline.predict(X_test)
    
    metrics = {
        'cv_mean': cv_scores.mean(),
        'cv_std': cv_scores.std(),
        'test_accuracy': accuracy_score(y_test, y_pred),
        'test_f1': f1_score(y_test, y_pred, average='weighted')
    }
    
    return pipeline, metrics
```

### Pattern 2: Model Selection with Multiple Algorithms
```python
def compare_models(X_train, y_train, X_test, y_test):
    models = {
        'Logistic Regression': LogisticRegression(max_iter=1000),
        'Random Forest': RandomForestClassifier(n_estimators=100),
        'Gradient Boosting': GradientBoostingClassifier(),
        'SVM': SVC(probability=True)
    }
    
    results = []
    for name, model in models.items():
        # CV score
        cv_scores = cross_val_score(model, X_train, y_train, cv=5)
        
        # Fit and test
        model.fit(X_train, y_train)
        y_pred = model.predict(X_test)
        
        results.append({
            'model': name,
            'cv_mean': cv_scores.mean(),
            'cv_std': cv_scores.std(),
            'test_accuracy': accuracy_score(y_test, y_pred)
        })
    
    return pd.DataFrame(results).sort_values('cv_mean', ascending=False)
```

### Pattern 3: Hyperparameter Tuning with Grid Search
```python
def tune_hyperparameters(X, y):
    pipeline = Pipeline([
        ('scaler', StandardScaler()),
        ('classifier', RandomForestClassifier(random_state=42))
    ])
    
    param_grid = {
        'classifier__n_estimators': [100, 200, 300],
        'classifier__max_depth': [5, 10, 15, None],
        'classifier__min_samples_split': [2, 5, 10],
        'classifier__min_samples_leaf': [1, 2, 4]
    }
    
    grid_search = GridSearchCV(
        pipeline, param_grid, cv=5, 
        scoring='f1_weighted', n_jobs=-1, verbose=1
    )
    grid_search.fit(X, y)
    
    return grid_search.best_estimator_, grid_search.best_params_
```