# Ds Online Experiments > "Implements multi-armed bandits, contextual bandits, exploration-exploitation tradeoff, and online learning algorithms" - Skill: `paulpas/ds-online-experiments` (Agent Skill) - Install (CLI): `npx skillmds@latest add paulpas/ds-online-experiments` - Raw SKILL.md: https://api.skillmd.com/api/skills/paulpas/ds-online-experiments/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: MIT - Author: paulpas (https://skillmd.com/u/paulpas) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/paulpas/ds-online-experiments --- # Online Experiments Comprehensive guide to online experiments in machine learning and data science workflows. ## When to Use This Skill - Solving real-world experimentation & a/b testing problems - Building machine learning pipelines with online experiments - Implementing best practices for online experiments - Optimizing model performance using online experiments techniques - Learning industry-standard approaches to online experiments ## When NOT to Use This Skill - When using pre-built libraries without understanding underlying concepts - For toy problems that don't require online experiments rigor - When domain expertise in specific problem requires different approach - If your problem doesn't require the complexity this skill provides ## Purpose and Key Concepts Online Experiments is a critical component of the machine learning workflow. This skill covers: 1. **Theoretical foundations** — Mathematical principles and statistical concepts 2. **Practical implementation** — Working code examples and patterns 3. **Common pitfalls** — Mistakes to avoid and how to recover from them 4. **Best practices** — Industry-standard approaches and optimization techniques ## Core Workflow 1. **Understand the problem** — Clearly define what you're solving for 2. **Select approach** — Choose the right technique for your data and constraints 3. **Implement solution** — Write clean, tested code following best practices 4. **Validate results** — Verify your implementation with tests and validation 5. **Optimize performance** — Improve efficiency and accuracy incrementally ## Implementation Patterns ### Pattern 1: Basic Online Experiments ```python import numpy as np from typing import Dict, List, Tuple class EpsilonGreedyBandit: """Basic epsilon-greedy multi-armed bandit implementation.""" def __init__(self, n_arms: int, epsilon: float = 0.1) -> None: if n_arms <= 0: raise ValueError("n_arms must be a positive integer") self.n_arms = n_arms self.epsilon = max(0.0, min(1.0, epsilon)) self.q_values = np.zeros(n_arms) self.arm_counts = np.zeros(n_arms) def select_arm(self) -> int: if np.random.random() < self.epsilon: return int(np.random.randint(self.n_arms)) return int(np.argmax(self.q_values)) def update(self, arm: int, reward: float) -> None: if not 0 <= arm < self.n_arms: raise ValueError(f"Arm index {arm} out of range [0, {self.n_arms})") self.arm_counts[arm] += 1 n = self.arm_counts[arm] old_q = self.q_values[arm] self.q_values[arm] = old_q + (1.0 / n) * (reward - old_q) def get_statistics(self) -> Dict[str, List[float]]: return { 'avg_reward': [float(np.mean(self.q_values))] 'arm_counts': self.arm_counts.tolist() 'q_values': self.q_values.tolist() } ``` ### Pattern 2: Production-Ready Online Experiments ```python import logging import numpy as np import pandas as pd from typing import Any, Dict, List, Optional logger = logging.getLogger(__name__) class UCBBandit: """Upper Confidence Bound bandit for production online experiments.""" def __init__(self, n_arms: int, confidence: float = 2.0) -> None: if n_arms <= 0: raise ValueError("n_arms must be positive") self.n_arms = n_arms self.confidence = confidence self.q_values = np.zeros(n_arms) self.arm_counts = np.zeros(n_arms) self.total_pulls = 0 def select_arm(self) -> int: if self.total_pulls < self.n_arms: return int(np.argmin(self.arm_counts)) ucb_values = self.q_values + self.confidence * np.sqrt( np.log(self.total_pulls) / self.arm_counts ) return int(np.argmax(ucb_values)) def update(self, arm: int, reward: float) -> None: self.arm_counts[arm] += 1 self.total_pulls += 1 n = self.arm_counts[arm] old_q = self.q_values[arm] self.q_values[arm] = old_q + (1.0 / n) * (reward - old_q) def execute(self, data: pd.DataFrame) -> Dict[str, Any]: required_cols = {'reward', 'arm'} if not required_cols.issubset(data.columns): raise ValueError(f"DataFrame must contain columns: {required_cols}") results: List[Dict[str, Any]] = [] for _, row in data.iterrows(): arm = int(row['arm']) reward = float(row['reward']) self.update(arm, reward) results.append({ 'arm': arm, 'reward': reward, 'q_estimate': float(self.q_values[arm]) }) logger.info(f"Processed {len(results)} observations across {self.n_arms} arms") return { 'final_q_values': self.q_values.tolist() 'arm_counts': self.arm_counts.tolist() 'total_pulls': self.total_pulls 'history': results } ``` ### Pattern 3: BAD vs GOOD Implementation ```python # BAD: Hardcoded values, no type hints, bypasses validation, monolithic function def bad_bandit(data): q = [0.0, 0.0, 0.0] for i in range(len(data)): if i < 3: arm = i else: arm = q.index(max(q)) q[arm] = q[arm] + 0.1 * (data[i] - q[arm]) return q # GOOD: Parameterized, typed, validated, decomposed, follows SOLID principles import numpy as np from typing import List, Dict def good_bandit(rewards: List[float], n_arms: int = 3, alpha: float = 0.1) -> Dict[str, List[float]]: if n_arms <= 0 or not rewards: raise ValueError("Invalid parameters") q_estimates = np.zeros(n_arms) arm_counts = np.zeros(n_arms) for t, reward in enumerate(rewards): if t < n_arms: arm = t else: arm = int(np.argmax(q_estimates)) arm_counts[arm] += 1 n = arm_counts[arm] q_estimates[arm] += alpha * (reward - q_estimates[arm]) return { 'estimated_means': q_estimates.tolist() 'pull_counts': arm_counts.tolist() } ``` ## Best Practices - ✅ Always validate your implementation on test data - ✅ Document your assumptions and methodology - ✅ Use version control for reproducibility - ✅ Monitor performance metrics in production - ✅ Periodically review and update your approach - ✅ Test with edge cases and outliers - ✅ Log all significant operations for debugging - ✅ Follow SOLID principles for class design and DRY for utility functions ## Common Pitfalls | Pitfall | Problem | Solution | | --- --- ## Constraints ### MUST DO - Validate all data preprocessing steps are fit-only on training data, never on validation or test sets - Implement reproducible pipelines with fixed random seeds and deterministic operations where possible - Report model performance with confidence intervals via bootstrapping or cross-validation across multiple runs - Log all experiments with parameters, metrics, and artifacts using MLflow or equivalent tracking system ### MUST NOT DO - Do not evaluate a model on the same data used for training — always hold out a proper test set - Avoid overfitting to the validation set by limiting hyperparameter search iterations - Never use features that can only be computed at inference time (look-ahead bias) - Do not report single-run accuracy without statistical significance testing or error bars ## Live References > Authoritative documentation links for this skill's domain. The model follows markdown links at load time to resolve external references and inline content. - [Online Controlled Experiments (Microsoft Research)](https://www.microsoft.com/en-us/research/uploads/prod/2018/03/online-a-b-testing.pdf) - [Optimizely Online Experimentation Guide](https://blog.optimizely.com/a-b-testing-best-practices/) - [VWO A/B Testing Best Practices](https://vwo.com/blog/ab-testing-best-practices/) - [Experimentation at Scale — Netflix Eng Blog](https://netflixtechblog.com/experimentation-at-scale-netflix-and-the-evolution-of-distributed-decision-making-a1f7b2e45a0e) - [Google Optimize Documentation](https://support.google/optimize/)