Data Science Expert
Comprehensive data science frameworks for analytics, machine learning, and data-driven decision making.
Data Strategy
Data Maturity Model
| Level |
Name |
Characteristics |
| 1 |
Ad Hoc |
Manual, inconsistent, siloed |
| 2 |
Opportunistic |
Some automation, point solutions |
| 3 |
Systematic |
Defined processes, governance emerging |
| 4 |
Differentiating |
Data-driven decisions, advanced analytics |
| 5 |
Transformative |
AI-first, competitive advantage |
Analytics Value Chain
DATA → INFORMATION → INSIGHT → ACTION → VALUE
PROGRESSION:
Descriptive: What happened?
Diagnostic: Why did it happen?
Predictive: What will happen?
Prescriptive: What should we do?
Autonomous: Self-optimizing systems
Statistical Analysis
Descriptive Statistics
CENTRAL TENDENCY:
- Mean: Sum / Count (sensitive to outliers)
- Median: Middle value (robust to outliers)
- Mode: Most frequent value
DISPERSION:
- Range: Max - Min
- Variance: Average squared deviation
- Standard Deviation: √Variance
- IQR: Q3 - Q1 (robust)
DISTRIBUTION SHAPE:
- Skewness: Asymmetry (0 = symmetric)
- Kurtosis: Tail heaviness (3 = normal)
For detailed inferential statistics and hypothesis testing, see Statistical Methods Reference.
Machine Learning
Algorithm Selection
| Task |
Algorithms |
When to Use |
| Classification |
Logistic Regression, Random Forest, XGBoost, Neural Networks |
Categorical outcomes |
| Regression |
Linear Regression, Ridge/Lasso, Random Forest, XGBoost |
Continuous outcomes |
| Clustering |
K-Means, Hierarchical, DBSCAN |
Group discovery |
| Dimensionality Reduction |
PCA, t-SNE, UMAP |
Feature reduction, visualization |
| Anomaly Detection |
Isolation Forest, One-Class SVM, Autoencoders |
Outlier detection |
| Time Series |
ARIMA, Prophet, LSTM |
Sequential data |
| Recommendation |
Collaborative Filtering, Content-Based, Matrix Factorization |
Personalization |
| NLP |
Transformers, BERT, GPT |
Text understanding/generation |
For detailed ML pipelines, feature engineering, and model monitoring, see ML Pipelines Reference.
Data Governance
Data Governance Framework
GOVERNANCE PILLARS:
POLICIES:
- Data ownership
- Data classification
- Data retention
- Data access
- Data quality standards
ROLES:
- Data Owner: Accountable for data domain
- Data Steward: Day-to-day quality management
- Data Custodian: Technical implementation
- Data Consumer: End user
PROCESSES:
- Data cataloging
- Metadata management
- Data lineage
- Issue resolution
- Change management
METRICS:
- Data quality scores
- Policy compliance
- Data access requests
- Issue resolution time
Data Quality Dimensions
| Dimension |
Definition |
Measurement |
| Accuracy |
Correct representation of reality |
% records matching source |
| Completeness |
All required data present |
% non-null values |
| Consistency |
Same across systems |
% matching across sources |
| Timeliness |
Available when needed |
Latency, freshness |
| Validity |
Conforms to format/rules |
% passing validation |
| Uniqueness |
No unwanted duplicates |
Duplicate rate |
Business Intelligence
BI Architecture
ARCHITECTURE LAYERS:
DATA SOURCES:
- Operational systems
- External data
- IoT/streaming
DATA INTEGRATION:
- ETL/ELT pipelines
- Data lakes
- Data warehouses
SEMANTIC LAYER:
- Business definitions
- Calculated metrics
- Hierarchies
- Relationships
PRESENTATION:
- Dashboards
- Reports
- Ad-hoc analysis
- Embedded analytics
Dashboard Design Principles
DESIGN PRINCIPLES:
PURPOSE:
- One clear objective per dashboard
- Know your audience
- Enable decisions
LAYOUT:
- Most important top-left
- Related items grouped
- Progressive disclosure
- Whitespace for clarity
VISUALS:
- Right chart for data type
- Consistent formatting
- Minimal decoration
- Color with purpose
INTERACTIVITY:
- Filters for exploration
- Drill-down capability
- Cross-filtering
- Tooltip details
Metric Design
METRIC DEFINITION TEMPLATE:
NAME: [Metric name]
DEFINITION: [Clear business definition]
FORMULA: [Precise calculation]
OWNER: [Responsible person]
DATA SOURCE: [Where it comes from]
GRAIN: [Level of detail]
FREQUENCY: [Update cadence]
DIMENSIONS: [Slicing attributes]
TARGETS: [Goals/benchmarks]
RELATED: [Related metrics]
Predictive Modeling
Use Case Framework
| Use Case |
Business Application |
Approach |
| Churn Prediction |
Retention programs |
Classification |
| Demand Forecasting |
Inventory planning |
Time series |
| Lead Scoring |
Sales prioritization |
Classification |
| Price Optimization |
Revenue management |
Regression/RL |
| Fraud Detection |
Risk mitigation |
Anomaly detection |
| Recommendation |
Personalization |
Collaborative filtering |
| Customer Segmentation |
Marketing targeting |
Clustering |
| Lifetime Value |
Customer investment |
Regression |
Data Ethics & Privacy
Ethical AI Framework
PRINCIPLES:
FAIRNESS:
- No discriminatory outcomes
- Bias testing across groups
- Regular auditing
ACCOUNTABILITY:
- Clear ownership
- Decision audit trails
- Escalation process
TRANSPARENCY:
- Explainable decisions
- Clear documentation
- User communication
PRIVACY:
- Data minimization
- Consent management
- Security controls
Bias Detection
BIAS TYPES:
HISTORICAL: Reflects past discrimination
REPRESENTATION: Training data not representative
MEASUREMENT: Proxy variables correlate with protected attributes
AGGREGATION: Single model for diverse populations
EVALUATION: Inappropriate benchmarks
FAIRNESS METRICS:
- Demographic Parity: Equal positive rates
- Equalized Odds: Equal TPR and FPR
- Individual Fairness: Similar inputs, similar outputs
- Calibration: Equal accuracy across groups
Analytics Team Structure
Team Roles
| Role |
Focus |
Skills |
| Data Engineer |
Pipelines, infrastructure |
SQL, Python, Spark, Cloud |
| Data Analyst |
Reporting, ad-hoc analysis |
SQL, BI tools, Statistics |
| Data Scientist |
Modeling, ML |
Python/R, ML, Statistics |
| ML Engineer |
Model deployment |
MLOps, Software Engineering |
| Analytics Engineer |
Data modeling |
dbt, SQL, Data Modeling |
Operating Models
| Model |
Description |
Best For |
| Centralized |
Single analytics team |
Consistency, efficiency |
| Decentralized |
Embedded in business units |
Business alignment |
| Hub & Spoke |
Central CoE + embedded |
Balance of both |
| Federated |
Shared platform, domain teams |
Scale with autonomy |
References
- ML Pipelines Reference - Detailed ML pipeline, feature engineering, model development
- Statistical Methods Reference - Inferential statistics, hypothesis testing, evaluation metrics
See Also
1---2name: data-science3description: Data science and analytics expertise for statistical analysis, machine learning pipelines, data governance, business intelligence, predictive modeling, and analytics strategy. Use when building ML models, analyzing data, creating dashboards, or designing data architectures.4---5
6# Data Science Expert
7
8Comprehensive data science frameworks for analytics, machine learning, and data-driven decision making.
9
10## Data Strategy
11
12### Data Maturity Model
13
14| Level | Name | Characteristics |
15| ----- | ------------------- | ----------------------------------------- |
16| 1 | **Ad Hoc** | Manual, inconsistent, siloed |
17| 2 | **Opportunistic** | Some automation, point solutions |
18| 3 | **Systematic** | Defined processes, governance emerging |
19| 4 | **Differentiating** | Data-driven decisions, advanced analytics |
20| 5 | **Transformative** | AI-first, competitive advantage |
21
22### Analytics Value Chain
23
24```
25DATA → INFORMATION → INSIGHT → ACTION → VALUE
26
27PROGRESSION:
28Descriptive: What happened?
29Diagnostic: Why did it happen?
30Predictive: What will happen?
31Prescriptive: What should we do?
32Autonomous: Self-optimizing systems
33```
34
35## Statistical Analysis
36
37### Descriptive Statistics
38
39```
40CENTRAL TENDENCY:
41- Mean: Sum / Count (sensitive to outliers)
42- Median: Middle value (robust to outliers)
43- Mode: Most frequent value
44
45DISPERSION:
46- Range: Max - Min
47- Variance: Average squared deviation
48- Standard Deviation: √Variance
49- IQR: Q3 - Q1 (robust)
50
51DISTRIBUTION SHAPE:
52- Skewness: Asymmetry (0 = symmetric)
53- Kurtosis: Tail heaviness (3 = normal)
54```
55
56For detailed inferential statistics and hypothesis testing, see [Statistical Methods Reference](references/statistical-methods.md).
57
58## Machine Learning
59
60### Algorithm Selection
61
62| Task | Algorithms | When to Use |
63| ---------------------------- | ------------------------------------------------------------ | -------------------------------- |
64| **Classification** | Logistic Regression, Random Forest, XGBoost, Neural Networks | Categorical outcomes |
65| **Regression** | Linear Regression, Ridge/Lasso, Random Forest, XGBoost | Continuous outcomes |
66| **Clustering** | K-Means, Hierarchical, DBSCAN | Group discovery |
67| **Dimensionality Reduction** | PCA, t-SNE, UMAP | Feature reduction, visualization |
68| **Anomaly Detection** | Isolation Forest, One-Class SVM, Autoencoders | Outlier detection |
69| **Time Series** | ARIMA, Prophet, LSTM | Sequential data |
70| **Recommendation** | Collaborative Filtering, Content-Based, Matrix Factorization | Personalization |
71| **NLP** | Transformers, BERT, GPT | Text understanding/generation |
72
73For detailed ML pipelines, feature engineering, and model monitoring, see [ML Pipelines Reference](references/ml-pipelines.md).
74
75## Data Governance
76
77### Data Governance Framework
78
79```
80GOVERNANCE PILLARS:
81
82POLICIES:
83- Data ownership
84- Data classification
85- Data retention
86- Data access
87- Data quality standards
88
89ROLES:
90- Data Owner: Accountable for data domain
91- Data Steward: Day-to-day quality management
92- Data Custodian: Technical implementation
93- Data Consumer: End user
94
95PROCESSES:
96- Data cataloging
97- Metadata management
98- Data lineage
99- Issue resolution
100- Change management
101
102METRICS:
103- Data quality scores
104- Policy compliance
105- Data access requests
106- Issue resolution time
107```
108
109### Data Quality Dimensions
110
111| Dimension | Definition | Measurement |
112| ---------------- | --------------------------------- | ------------------------- |
113| **Accuracy** | Correct representation of reality | % records matching source |
114| **Completeness** | All required data present | % non-null values |
115| **Consistency** | Same across systems | % matching across sources |
116| **Timeliness** | Available when needed | Latency, freshness |
117| **Validity** | Conforms to format/rules | % passing validation |
118| **Uniqueness** | No unwanted duplicates | Duplicate rate |
119
120## Business Intelligence
121
122### BI Architecture
123
124```
125ARCHITECTURE LAYERS:
126
127DATA SOURCES:
128- Operational systems
129- External data
130- IoT/streaming
131
132DATA INTEGRATION:
133- ETL/ELT pipelines
134- Data lakes
135- Data warehouses
136
137SEMANTIC LAYER:
138- Business definitions
139- Calculated metrics
140- Hierarchies
141- Relationships
142
143PRESENTATION:
144- Dashboards
145- Reports
146- Ad-hoc analysis
147- Embedded analytics
148```
149
150### Dashboard Design Principles
151
152```
153DESIGN PRINCIPLES:
154
155PURPOSE:
156- One clear objective per dashboard
157- Know your audience
158- Enable decisions
159
160LAYOUT:
161- Most important top-left
162- Related items grouped
163- Progressive disclosure
164- Whitespace for clarity
165
166VISUALS:
167- Right chart for data type
168- Consistent formatting
169- Minimal decoration
170- Color with purpose
171
172INTERACTIVITY:
173- Filters for exploration
174- Drill-down capability
175- Cross-filtering
176- Tooltip details
177```
178
179### Metric Design
180
181```
182METRIC DEFINITION TEMPLATE:
183
184NAME: [Metric name]
185DEFINITION: [Clear business definition]
186FORMULA: [Precise calculation]
187OWNER: [Responsible person]
188DATA SOURCE: [Where it comes from]
189GRAIN: [Level of detail]
190FREQUENCY: [Update cadence]
191DIMENSIONS: [Slicing attributes]
192TARGETS: [Goals/benchmarks]
193RELATED: [Related metrics]
194```
195
196## Predictive Modeling
197
198### Use Case Framework
199
200| Use Case | Business Application | Approach |
201| ------------------------- | -------------------- | ----------------------- |
202| **Churn Prediction** | Retention programs | Classification |
203| **Demand Forecasting** | Inventory planning | Time series |
204| **Lead Scoring** | Sales prioritization | Classification |
205| **Price Optimization** | Revenue management | Regression/RL |
206| **Fraud Detection** | Risk mitigation | Anomaly detection |
207| **Recommendation** | Personalization | Collaborative filtering |
208| **Customer Segmentation** | Marketing targeting | Clustering |
209| **Lifetime Value** | Customer investment | Regression |
210
211## Data Ethics & Privacy
212
213### Ethical AI Framework
214
215```
216PRINCIPLES:
217
218FAIRNESS:
219- No discriminatory outcomes
220- Bias testing across groups
221- Regular auditing
222
223ACCOUNTABILITY:
224- Clear ownership
225- Decision audit trails
226- Escalation process
227
228TRANSPARENCY:
229- Explainable decisions
230- Clear documentation
231- User communication
232
233PRIVACY:
234- Data minimization
235- Consent management
236- Security controls
237```
238
239### Bias Detection
240
241```
242BIAS TYPES:
243
244HISTORICAL: Reflects past discrimination
245REPRESENTATION: Training data not representative
246MEASUREMENT: Proxy variables correlate with protected attributes
247AGGREGATION: Single model for diverse populations
248EVALUATION: Inappropriate benchmarks
249
250FAIRNESS METRICS:
251- Demographic Parity: Equal positive rates
252- Equalized Odds: Equal TPR and FPR
253- Individual Fairness: Similar inputs, similar outputs
254- Calibration: Equal accuracy across groups
255```
256
257## Analytics Team Structure
258
259### Team Roles
260
261| Role | Focus | Skills |
262| ---------------------- | -------------------------- | --------------------------- |
263| **Data Engineer** | Pipelines, infrastructure | SQL, Python, Spark, Cloud |
264| **Data Analyst** | Reporting, ad-hoc analysis | SQL, BI tools, Statistics |
265| **Data Scientist** | Modeling, ML | Python/R, ML, Statistics |
266| **ML Engineer** | Model deployment | MLOps, Software Engineering |
267| **Analytics Engineer** | Data modeling | dbt, SQL, Data Modeling |
268
269### Operating Models
270
271| Model | Description | Best For |
272| ----------------- | ----------------------------- | ----------------------- |
273| **Centralized** | Single analytics team | Consistency, efficiency |
274| **Decentralized** | Embedded in business units | Business alignment |
275| **Hub & Spoke** | Central CoE + embedded | Balance of both |
276| **Federated** | Shared platform, domain teams | Scale with autonomy |
277
278## References
279
280- [ML Pipelines Reference](references/ml-pipelines.md) - Detailed ML pipeline, feature engineering, model development
281- [Statistical Methods Reference](references/statistical-methods.md) - Inferential statistics, hypothesis testing, evaluation metrics
282
283## See Also
284
285- [Fortune 50 Product Management](../fortune50-product-management/SKILL.md)
286- [Fortune 50 Business Strategy](../fortune50-business-strategy/SKILL.md)
287- [Fortune 50 Finance](../fortune50-finance/SKILL.md)
majiayu000