majiayu000 Data Analysis for Science
Data analysis is where observations become evidence. Raw measurements are not conclusions -- they are the raw material from which conclusions are carefully extracted. This skill covers the full pipeline from recording measurements to drawing valid conclusions, with emphasis on honest reporting, appropriate statistical reasoning, and the discipline of knowing where the data end and interpretation begins.
Agent affinity: wu (precision and error analysis), mcclintock (data interpretation)
Concept IDs: sci-measurement-units, sci-data-tables-graphs, sci-error-analysis, sci-evidence-conclusions
The Analysis Pipeline
| Stage | What happens | Key question |
|---|---|---|
| 1. Measurement | Quantitative observation with instruments | What is being measured, with what tool, to what precision? |
| 2. Recording | Data captured in organized form | Are the data recorded systematically and in real time? |
| 3. Description | Summary statistics and visualization | What do the data look like? |
| 4. Inference | Statistical tests | Are observed patterns real or noise? |
| 5. Conclusion | Interpretation and limitation | What can the data tell us, and what can they not? |
Stage 1 -- Measurement
SI Units and Dimensional Consistency
All scientific measurements use SI units (International System of Units) or units derived from them:
| Quantity | SI unit | Symbol |
|---|---|---|
| Length | meter | m |
| Mass | kilogram | kg |
| Time | second | s |
| Temperature | kelvin | K |
| Amount | mole | mol |
| Electric current | ampere | A |
| Luminous intensity | candela | cd |
Dimensional analysis catches errors by checking that units are consistent across an equation. If you are calculating a speed and your answer has units of kg/m, something is wrong.
Significant Figures
The number of significant figures in a measurement reflects the precision of the instrument:
- A ruler marked in millimeters gives measurements to 3-4 significant figures (e.g., 12.3 cm).
- A digital balance reading 4.5678 g gives 5 significant figures.
- Calculations cannot increase precision. The result has the same number of significant figures as the least precise input.
The false precision trap: Reporting a calculated result as "3.141592654 grams" when the original measurement was "3.1 grams" creates an illusion of precision that does not exist. Wu's significant figures audit catches this.
Accuracy vs. Precision
Accuracy: How close a measurement is to the true value. A scale that consistently reads 0.5 g too high is precise but inaccurate.
Precision: How close repeated measurements are to each other. A scale that reads 4.2, 4.3, 4.1, 4.2, 4.2 is precise. If the true value is 4.2, it is also accurate. If the true value is 5.0, it is precise but not accurate.
Both matter. Precision without accuracy is systematically wrong. Accuracy without precision is unreliable.
Stage 2 -- Recording
Data Tables
Data should be recorded in structured tables at the time of measurement, not reconstructed from memory afterward.
Table conventions:
- Independent variable in the left column
- Dependent variable(s) in subsequent columns
- Units in column headers, not in individual cells
- One measurement per row
- Include a column for notes/observations (qualitative data captured alongside quantitative)
Example:
| Light wavelength (nm) | Plant height day 14 (cm) | Notes |
|---|---|---|
| 450 (blue) | 8.2 | Slight yellowing on leaf tips |
| 450 (blue) | 7.9 | |
| 450 (blue) | 8.5 | |
| 650 (red) | 12.1 | |
| 650 (red) | 11.8 | |
| 650 (red) | 12.4 | Tallest plant in tray |
Data Integrity
- Record in real time. Do not fill in data tables from memory.
- Record anomalies. If a measurement looks odd, record it and flag it. Do not silently discard it.
- Use ink, not pencil (or digital timestamps). The record should show what was measured, not what was expected.
- Preserve raw data. Processed data (means, ratios) are derived from raw data. If the raw data are lost, reanalysis is impossible.
Stage 3 -- Description
Descriptive Statistics
| Statistic | What it tells you | When to use |
|---|---|---|
| Mean (average) | Central tendency | Symmetric distributions |
| Median | Central tendency, robust to outliers | Skewed distributions or when outliers are present |
| Standard deviation (SD) | Spread of individual measurements | Describing variability in raw data |
| Standard error of the mean (SEM) | Precision of the estimated mean | Comparing group means |
| Range | Total spread (min to max) | Quick sense of variability |
| Interquartile range (IQR) | Middle 50% spread | Robust alternative to SD for skewed data |
SD vs. SEM: SD describes how variable the individual data points are. SEM describes how precisely the mean is estimated. SEM = SD / sqrt(n). As sample size increases, SEM shrinks (the mean estimate gets more precise) but SD does not (individual variability is unchanged). Use SD when describing the data. Use SEM when comparing group means.
Graphical Representation
| Graph type | Best for | Example |
|---|---|---|
| Bar chart | Comparing means across discrete categories | Mean plant height by light color |
| Scatter plot | Showing relationship between two continuous variables | Height vs. temperature |
| Line graph | Showing change over time or continuous IV | Growth rate over 14 days |
| Histogram | Showing distribution of a single variable | Distribution of test scores |
| Box plot | Comparing distributions across groups | Height distribution by treatment |
Error bars: Every bar chart or line graph comparing group means should include error bars. Use SEM or 95% confidence intervals (report which one). Error bars without a label are meaningless.
The cardinal rule of graphs: A graph should be interpretable without reading the surrounding text. This requires: descriptive title, labeled axes with units, legend if multiple series, and error bars if applicable.
Stage 4 -- Inference
The Logic of Statistical Testing
Statistical inference answers: "Is the observed pattern likely to be real, or could it be explained by random chance?"
The framework:
- State the null hypothesis (H0): there is no effect / no difference.
- State the alternative hypothesis (H1): there is an effect / a difference.
- Choose a significance level (alpha, typically 0.05).
- Calculate a test statistic from the data.
- Determine the p-value: the probability of observing data at least as extreme as the actual data, assuming H0 is true.
- If p < alpha, reject H0. If p >= alpha, fail to reject H0.
What "fail to reject H0" means: It does NOT mean H0 is true. It means the data are insufficient to distinguish from H0. Absence of evidence is not evidence of absence.
Common Tests
| Test | Use when | Example |
|---|---|---|
| t-test (independent) | Comparing means of 2 independent groups | Treatment vs. control |
| t-test (paired) | Comparing means of 2 related measurements | Before vs. after in same subjects |
| ANOVA | Comparing means of 3+ groups | Three light wavelengths |
| Chi-square | Testing association between categorical variables | Is survival related to treatment group? |
| Correlation (Pearson) | Measuring linear association between 2 continuous variables | Height vs. weight |
| Linear regression | Modeling the relationship between variables | Predicting growth rate from temperature |
Effect Size
Statistical significance tells you whether an effect is real. Effect size tells you whether it matters. Report both.
Cohen's d for comparing two means: d = (mean1 - mean2) / pooled SD. Conventions: d = 0.2 (small), d = 0.5 (medium), d = 0.8 (large).
r-squared for regression: the proportion of variance in the DV explained by the IV. r^2 = 0.01 (1% explained, trivial), r^2 = 0.25 (25%, moderate), r^2 = 0.64 (64%, large).
Stage 5 -- Conclusion
Drawing Valid Conclusions
A valid conclusion:
- States what the data support. "Plants grown under blue light were significantly shorter than those grown under red light (t(28) = 4.2, p < 0.001, d = 1.1)."
- Quantifies the evidence. Effect size and confidence interval, not just the p-value.
- Acknowledges limitations. "This study tested only two wavelengths. Intermediate wavelengths were not tested."
- Distinguishes correlation from causation. Controlled experiments can support causal claims. Observational studies can establish correlation and temporal precedence but not causation.
- Identifies what the data do NOT show. "The data do not address whether these effects persist beyond 14 days."
The Absence of Evidence Clause
When an experiment finds no significant effect, the correct conclusion is NOT "there is no effect." The correct conclusion is: "The experiment did not detect an effect. This may mean there is no effect, or the effect may be too small to detect with this sample size (power = X), or the measurement may lack sufficient precision."
Cross-References
- wu agent: Primary agent for precision measurement and error analysis. Draws on this skill for statistical methodology.
- mcclintock agent: Data interpretation in the context of experimental design.
- feynman-s agent: Evaluates whether data analysis methods are appropriate for the claims being made.
- scientific-method skill: The overarching inquiry cycle within which data analysis is one stage.
- experimental-design-sci skill: The design that determines what data are collected.
References
- Moore, D. S., McCabe, G. P., & Craig, B. A. (2017). Introduction to the Practice of Statistics. 9th edition. W.H. Freeman.
- Tufte, E. R. (2001). The Visual Display of Quantitative Information. 2nd edition. Graphics Press.
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd edition. Lawrence Erlbaum.
- Wasserstein, R. L., & Lazar, N. A. (2016). "The ASA Statement on p-Values." The American Statistician, 70(2), 129-133.
- Cumming, G. (2012). Understanding the New Statistics. Routledge.