# Ds Bayesian Inference > "Applies Bayesian methods for prior selection, posterior estimation and probabilistic inference in machine learning models" - Skill: `paulpas/ds-bayesian-inference` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-bayesian-inference` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-bayesian-inference/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-bayesian-inference --- # Bayesian Inference Comprehensive guide to bayesian inference in machine learning and data science workflows. ## When to Use This Skill - Solving real-world statistical inference problems - Building machine learning pipelines with bayesian inference - Implementing best practices for bayesian inference - Optimizing model performance using bayesian inference techniques - Learning industry-standard approaches to bayesian inference ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require bayesian inference rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Bayesian Inference is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Bayesian Inference ```python import numpy as np import pandas as pd from scipy import stats from typing import Tuple, Dict def bayesian_linear_regression( X: np.ndarray, y: np.ndarray, prior_precision: float = 1.0 ) -> Dict[str, np.ndarray]: """ Perform Bayesian Linear Regression using conjugate Normal-Gamma priors. Computes posterior distribution over regression weights analytically. Args: X: Feature matrix of shape (n_samples, n_features) y: Target vector of shape (n_samples,) prior_precision: Precision (inverse variance) of the prior on weights Returns: Dictionary containing posterior mean, covariance, and noise variance """ if X.ndim == 1: X = X.reshape(-1, 1) n_samples, n_features = X.shape X_b = np.column_stack([np.ones(n_samples), X]) # Initialize prior covariance matrix prior_cov = np.eye(n_features + 1) / prior_precision # Estimate initial noise variance from data beta_mle = np.linalg.lstsq(X_b, y, rcond=None)[0] residuals = y - X_b @ beta_mle sigma2_init = np.sum(residuals**2) / n_samples # Compute posterior precision and covariance posterior_precision = np.linalg.inv(prior_cov) + (1.0 / sigma2_init) * (X_b.T @ X_b) posterior_cov = np.linalg.inv(posterior_precision) # Compute posterior mean posterior_mean = posterior_cov @ ( np.linalg.inv(prior_cov) @ np.zeros(n_features + 1) + (1.0 / sigma2_init) * (X_b.T @ y) ) return { 'posterior_mean': posterior_mean 'posterior_cov': posterior_cov 'noise_variance': sigma2_init 'n_samples': n_samples 'n_features': n_features } ``` ### Pattern 2: Production-Ready Bayesian Inference ```python import logging import numpy as np import pandas as pd from typing import Any, Dict, Optional from scipy import stats logger = logging.getLogger(__name__) class BayesianInferenceEngine: """Production-grade Bayesian Inference engine for regression tasks.""" def __init__(self, prior_precision: float = 1.0, confidence_level: float = 0.95) -> None: self.prior_precision = prior_precision self.confidence_level = confidence_level logger.info("BayesianInferenceEngine initialized with prior_precision=%.2f, confidence=%.2f", prior_precision, confidence_level) def _validate_input(self, data: pd.DataFrame) -> None: if data is None or data.empty: raise ValueError("Input data cannot be None or empty") if not isinstance(data, pd.DataFrame): raise TypeError("Input must be a pandas DataFrame") if data.shape[0] < 2: raise ValueError("Insufficient data for inference: requires at least 2 samples") def execute(self, data: pd.DataFrame, target_col: str = 'y') -> Dict[str, Any]: """Execute Bayesian inference on provided data and return structured results.""" self._validate_input(data) logger.info("Running Bayesian inference on %d samples", len(data)) X = data.drop(columns=[target_col]).values y = data[target_col].values n, d = X.shape X_b = np.column_stack([np.ones(n), X]) prior_cov = np.eye(d + 1) / self.prior_precision sigma2 = np.var(y - np.mean(y)) post_prec = np.linalg.inv(prior_cov) + (1.0 / sigma2) * (X_b.T @ X_b) post_cov = np.linalg.inv(post_prec) post_mean = post_cov @ (np.linalg.inv(prior_cov) @ np.zeros(d + 1) + (1.0 / sigma2) * (X_b.T @ y)) y_pred = X_b @ post_mean pred_var = sigma2 * (1 + np.sum(X_b * (post_cov @ X_b.T), axis=1)) ci_half_width = stats.norm.ppf((1 + self.confidence_level) / 2) * np.sqrt(pred_var) return { 'posterior_mean': post_mean 'posterior_cov': post_cov 'predictions': y_pred 'lower_bound': y_pred - ci_half_width 'upper_bound': y_pred + ci_half_width 'noise_variance': sigma2 'status': 'success' } ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Validate all data preprocessing steps are fit-only on training data, never on validation or test sets - Implement reproducible pipelines with fixed random seeds and deterministic operations where possible - Report model performance with confidence intervals via bootstrapping or cross-validation across multiple runs - Log all experiments with parameters, metrics, and artifacts using MLflow or equivalent tracking system ### MUST NOT DO - Do not evaluate a model on the same data used for training — always hold out a proper test set - Avoid overfitting to the validation set by limiting hyperparameter search iterations - Never use features that can only be computed at inference time (look-ahead bias) - Do not report single-run accuracy without statistical significance testing or error bars ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [PyMC Documentation](https://docs.pymc.io/) - [Bayesian Inference — Wikipedia](https://en.wikipedia.org/wiki/Bayesian_inference) - [Bayesian Methods for Hackers](http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/) - [Stan User's Guide](https://mc-stan.org/docs/stan-users-guide/index.html) - [Bayesian A/B Testing (Optimizely)](https://blog.optimizely.com/a-b-testing-best-practices/), | - [Statistical Rethinking — Richard McElreath](https://github.com/rmcelreath/rethinking)