# Ds Maximum Likelihood > Implements maximum likelihood estimation, likelihood functions, and optimization methods for parameter estimation in probabilistic models - Skill: `paulpas/ds-maximum-likelihood` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-maximum-likelihood` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-maximum-likelihood/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-maximum-likelihood --- # Maximum Likelihood Estimation Comprehensive guide to maximum likelihood estimation in machine learning and data science workflows. ## When to Use This Skill - Solving real-world statistical inference problems - Building machine learning pipelines with maximum likelihood estimation - Implementing best practices for maximum likelihood estimation - Optimizing model performance using maximum likelihood estimation techniques - Learning industry-standard approaches to maximum likelihood estimation ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require maximum likelihood estimation rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Maximum Likelihood Estimation is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Maximum Likelihood Estimation ```python import numpy as np from scipy.optimize import minimize from scipy.stats import norm def negative_log_likelihood(params: np.ndarray, data: np.ndarray) -> float: """Calculate negative log-likelihood for Gaussian distribution.""" mu, sigma = params if sigma <= 0: return np.inf return -np.sum(norm.logpdf(data, loc=mu, scale=sigma)) def basic_mle(data: np.ndarray) -> dict: """ Perform basic Maximum Likelihood Estimation for Gaussian parameters. Args: data: 1D array of observations Returns: Dictionary containing estimated parameters and optimization status """ if data is None or len(data) == 0: raise ValueError("Input data cannot be empty") initial_guess = np.array([np.mean(data), np.std(data)]) result = minimize( negative_log_likelihood, initial_guess, args=(data,), method='Nelder-Mead' ) if not result.success: raise RuntimeError(f"MLE optimization failed: {result.message}") return { 'mu': result.x[0] 'sigma': result.x[1] 'log_likelihood': -result.fun 'converged': result.success } # Generate sample data and run estimation np.random.seed(42) sample_data = np.random.normal(loc=5.0, scale=2.0, size=1000) mle_results = basic_mle(sample_data) print(f"Estimated mu: {mle_results['mu']:.4f}, sigma: {mle_results['sigma']:.4f}") ``` ### Pattern 2: Production-Ready Maximum Likelihood Estimation ```python import logging import pandas as pd import numpy as np from scipy.optimize import minimize from scipy.stats import norm from typing import Any, Dict, List logger = logging.getLogger(__name__) class MaximumLikelihoodEstimator: """Production implementation of Maximum Likelihood Estimation for Gaussian distributions.""" def __init__(self, method: str = 'Nelder-Mead', tol: float = 1e-6): self.method = method self.tol = tol self.results_: Dict[str, Any] = {} def _negative_log_likelihood(self, params: List[float], data: np.ndarray) -> float: mu, sigma = params if sigma <= 0: return np.inf return -np.sum(np.log(norm.pdf(data, loc=mu, scale=sigma))) def fit(self, data: pd.DataFrame, column: str) -> 'MaximumLikelihoodEstimator': if column not in data.columns: raise ValueError(f"Column '{column}' not found in DataFrame") values = data[column].dropna().values if len(values) == 0: raise ValueError("No valid data points found in specified column") initial_guess = [np.mean(values), np.std(values)] result = minimize( self._negative_log_likelihood, initial_guess, args=(values,), method=self.method tol=self.tol ) if not result.success: logger.warning(f"MLE optimization did not converge: {result.message}") self.results_ = { 'parameters': {'mu': result.x[0], 'sigma': result.x[1]} 'log_likelihood': -result.fun 'converged': result.success 'iterations': result.nit } return self def execute(self, data: pd.DataFrame) -> Dict[str, Any]: """Return the fitted results. Requires fit() to be called first.""" if not self.results_: raise RuntimeError("Model has not been fitted. Call fit() first.") return self.results_ ``` ### Pattern 3: BAD vs GOOD Implementation (DRY Principle) ```python # BAD: Repeated likelihood calculations, no error handling, magic numbers def bad_mle(data): mu = sum(data) / len(data) sigma = sum((x - mu)**2 for x in data) / len(data) return mu, sigma # Ignores optimization, uses MLE closed-form incorrectly for small samples # GOOD: Modular, validated, uses scipy optimizer, follows DRY principle def good_mle(data: np.ndarray) -> Dict[str, float]: if len(data) < 2: raise ValueError("Need at least 2 points for reliable estimation") def nll(p): return -np.sum(norm.logpdf(data, loc=p[0], scale=p[1])) res = minimize(nll, [np.mean(data), np.std(data)], method='L-BFGS-B') return {'mu': res.x[0], 'sigma': res.x[1], 'll': -res.fun} ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Validate all data preprocessing steps are fit-only on training data, never on validation or test sets - Implement reproducible pipelines with fixed random seeds and deterministic operations where possible - Report model performance with confidence intervals via bootstrapping or cross-validation across multiple runs - Log all experiments with parameters, metrics, and artifacts using MLflow or equivalent tracking system ### MUST NOT DO - Do not evaluate a model on the same data used for training — always hold out a proper test set - Avoid overfitting to the validation set by limiting hyperparameter search iterations - Never use features that can only be computed at inference time (look-ahead bias) - Do not report single-run accuracy without statistical significance testing or error bars ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Maximum Likelihood Estimation — Wikipedia](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) - [MLE with SciPy Optimization](https://docs.scipy.org/doc/scipy/tutorial/stats/mle.html) - [Statistical Inference — Coursera (Johns Hopkins)](https://www.coursera.org/learn/statistical-inference) - [MLE Guide (Stanford Statistics 312)](https://web.stanford.edu/class/stats312/) - [Optimization Methods in SciPy](https://docs.scipy.org/doc/scipy/tutorial/optimize.html#maximum-likelihood-estimation)