scikit-learn

What this skill does

# scikit-learn - Machine Learning in Python

A robust library for classical machine learning. It features a uniform API: all objects share the same interface for fitting, transforming, and predicting.

## When to Use

- **Classification**: Detecting categories (Spam vs. Ham, Disease diagnosis).
- **Regression**: Predicting continuous values (House prices, Stock trends).
- **Clustering**: Grouping similar objects (Market segmentation, Image compression).
- **Dimensionality Reduction**: Reducing feature count while keeping info (PCA, Visualization).
- **Model Selection**: Comparing models and tuning hyperparameters (Cross-validation, Grid search).
- **Preprocessing**: Transforming raw data into features (Scaling, Encoding, Imputation).

## Reference Documentation

**Official docs**: https://scikit-learn.org/stable/  
**User Guide**: https://scikit-learn.org/stable/user_guide.html  
**Search patterns**: `sklearn.pipeline.Pipeline`, `sklearn.model_selection`, `sklearn.ensemble`, `sklearn.preprocessing`

## Core Principles

### The "Estimator" Interface

- **Estimators**: Implement `fit(X, y)`. They learn from data.
- **Transformers**: Implement `transform(X)` (and `fit_transform(X)`). They modify data.
- **Predictors**: Implement `predict(X)`. They provide estimates for new data.

### Use scikit-learn For

- Tabular data (Excel-like, CSVs).
- Traditional ML (Random Forests, SVMs, Linear Models).
- Feature engineering and pipeline automation.
- Small to medium-sized datasets.

### Do NOT Use For

- Deep Learning / Neural Networks (use PyTorch or TensorFlow).
- Natural Language Processing at scale (use spaCy or HuggingFace).
- Large-scale "Big Data" (use Spark MLlib or Dask-ML).
- Real-time streaming predictions (consider specialized inference engines).

## Quick Reference

### Installation

```bash
pip install scikit-learn
```

### Standard Imports

```python
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.metrics import classification_report, mean_squared_error
```

### Basic Pattern - Train/Predict

```python
from sklearn.ensemble import RandomForestClassifier

# 1. Prepare data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# 2. Instantiate and fit
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)

# 3. Predict and evaluate
predictions = model.predict(X_test)
print(classification_report(y_test, predictions))
```

## Critical Rules

### ✅ DO

- **Split before anything** - Always use `train_test_split` before looking at data properties.
- **Use Pipelines** - Combine preprocessing and modeling to prevent data leakage.
- **Scale your data** - Models like SVM, KNN, and Linear Regression require feature scaling.
- **Check for Imbalance** - Use `stratify=y` in `train_test_split` for classification.
- **Cross-Validate** - Don't trust a single train/test split; use `cross_val_score`.
- **Handle Missing Values** - Use `SimpleImputer` or similar before fitting models.
- **Standardize Categories** - Use `OneHotEncoder` for nominal or `OrdinalEncoder` for ordinal data.

### ❌ DON'T

- **Fit on test data** - Never call `.fit()` or `.fit_transform()` on the test set.
- **Use Categorical data as-is** - Scikit-learn requires numerical input; encode strings first.
- **Ignore Class Imbalance** - Accuracy is misleading for imbalanced datasets; use F1-score or AUC.
- **Overfit** - Don't keep tuning hyperparameters until the test score is perfect.
- **Ignore Random State** - Set `random_state` for reproducibility during experiments.

## Anti-Patterns (NEVER)

```python
# ❌ BAD: Data Leakage (Fitting scaler on the whole dataset)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X) # Data from "future" test set leaks into training!
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y)

# ✅ GOOD: Fit scaler only on training data
X_train, X_test, y_train, y_test = train_test_split(X, y)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test) # Use training mean/std

# ❌ BAD: Repeating preprocessing manually
# (Error-prone and hard to maintain)

# ✅ GOOD: Use Pipelines (Automates everything safely)
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('model', RandomForestClassifier())
])
pipe.fit(X_train, y_train)
```

## Preprocessing (sklearn.preprocessing)

### Scaling and Encoding

```python
from sklearn.preprocessing import StandardScaler, MinMaxScaler, OneHotEncoder

# Scaling numerical data
scaler = StandardScaler()
X_num_scaled = scaler.fit_transform(X_numeric)

# Encoding categorical data
encoder = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
X_cat_encoded = encoder.fit_transform(X_categorical)

# Handling missing values
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy='median')
X_imputed = imputer.fit_transform(X_with_nan)
```

### Column Transformer (The Pro Way)

```python
from sklearn.compose import ColumnTransformer

numeric_features = ['age', 'salary']
categorical_features = ['city', 'job_type']

preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numeric_features),
        ('cat', OneHotEncoder(), categorical_features)
    ])

# Now use this in a pipeline
pipeline = Pipeline([
    ('prep', preprocessor),
    ('clf', LogisticRegression())
])
```

## Classification

### Common Algorithms

```python
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.ensemble import GradientBoostingClassifier

# Logistic Regression (Baseline)
log_reg = LogisticRegression(max_iter=1000)

# Support Vector Machine
svm = SVC(kernel='rbf', probability=True)

# Gradient Boosting
gb = GradientBoostingClassifier(n_estimators=100, learning_rate=0.1)
```

## Regression

### Common Algorithms

```python
from sklearn.linear_model import Ridge, Lasso, ElasticNet
from sklearn.ensemble import RandomForestRegressor

# Regularized Linear Models
ridge = Ridge(alpha=1.0) # L2
lasso = Lasso(alpha=0.1) # L1

# Non-linear Regression
rf_reg = RandomForestRegressor(n_estimators=100, max_depth=10)
```

## Model Evaluation

### Metrics

```python
from sklearn.metrics import accuracy_score, f1_score, roc_auc_score, r2_score, mean_absolute_error

# Classification
acc = accuracy_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred, average='weighted')

# Regression
r2 = r2_score(y_true, y_pred)
mae = mean_absolute_error(y_true, y_pred)
```

### Cross-Validation

```python
from sklearn.model_selection import cross_val_score

scores = cross_val_score(pipeline, X, y, cv=5, scoring='f1_macro')
print(f"Mean F1: {scores.mean():.4f} (+/- {scores.std():.4f})")
```

## Hyperparameter Tuning

### Grid Search and Randomized Search

```python
from sklearn.model_selection import GridSearchCV

param_grid = {
    'clf__n_estimators': [50, 100, 200],
    'clf__max_depth': [None, 10, 20],
    'clf__min_samples_split': [2, 5]
}

grid_search = GridSearchCV(pipeline, param_grid, cv=3, scoring='accuracy', n_jobs=-1)
grid_search.fit(X_train, y_train)

print(f"Best params: {grid_search.best_params_}")
best_model = grid_search.best_estimator_
```

## Dimensionality Reduction

### PCA (Principal Component Analysis)

```python
from sklearn.decomposition import PCA

# Reduce to 2 components for visualization
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

print(f"Explained variance ratio: {pca.explained_variance_ratio_}")
```

## Clustering

### K-Means and DBSCAN

```python
from sklearn.cluster import KMeans, DBSCAN

# K-Means (Requires specifying K)
kmeans = KMeans(n_clusters=3, n_init='auto')
clusters = kmeans.fit_predict(X)

# DBSCAN (Density-based, finds K automatically)
dbscan = DBSCAN(eps=0.5, min_samples=5)
clusters = dbscan.fit_predict