ArviZ Python
Turn posterior artifacts into correctly labeled inference data, preserve chain
and draw structure, diagnose sampling and prediction, and compare only models
whose observations and log-likelihood semantics are compatible.
Boundary
Use this skill when the input is completed or partially completed Bayesian
draws and the code directly uses ArviZ. Use the modeling framework's skill to
build or sample a model. A plot request alone does not trigger this skill unless
the plot represents posterior, diagnostic, predictive, or comparison results.
Know the objects
| Object |
Meaning |
Use it for |
ArviZ 1.x DataTree / accepted idata-like container |
Named inference groups, each holding labeled xarray variables. Legacy integrations may still call the artifact InferenceData. |
The durable boundary for posterior analysis. |
| Group |
A semantic dataset such as posterior, sample_stats, log_likelihood, posterior_predictive, predictions, observed_data, or constant_data. |
Keeping quantities with different meaning separate. |
| Variable |
A named tensor with dimensions and coordinates. |
One parameter, statistic, observation, or prediction. |
chain |
Independent MCMC run dimension. |
Between-chain convergence diagnostics. |
draw |
Iteration within a chain. |
Within-chain sample sequence. |
| Model-comparison result |
Pointwise predictive score summaries, uncertainty, diagnostics, and weights. |
Comparing compatible predictive models, not proving truth. |
Dimension names determine semantics; axis order is secondary. Sample dimensions
must not be confused with event/data dimensions. observed_data has no chain or
draw axis; posterior predictive values normally have both plus observation
dimensions. Read the inference-data contract before
converting arrays, concatenating results, or selecting coordinates.
Ordered workflow
- Establish provenance: model, sampler/guide, package versions, warmup policy,
number of chains/draws, seed, observation identity, and log-likelihood target.
- Inspect available groups, variables, dimensions, coordinates, dtypes, and
missing values before computing a statistic. On ArviZ 1.x, treat the
DataTree children as the group registry instead of assuming 0.x
InferenceData attribute behavior.
- Convert raw arrays with explicit group, variable, dims, and coords mappings.
Preserve chain and draw separately; reject ambiguous flattened samples.
- Select variables/coordinates intentionally. Exclude warmup or transformed
helper variables only under a recorded policy.
- Diagnose sampling using multiple complementary measures and plots. Report
problematic variables and coordinates, not only a global maximum.
- Check posterior predictive adequacy with discrepancies tied to the modeling
objective and distinguish in-sample replicated data from out-of-sample
predictions.
- For model comparison, verify identical observations, likelihood target, data
preprocessing, and pointwise
log_likelihood; inspect Pareto diagnostics and
score uncertainty before ranking.
- Persist the labeled artifact and provenance when results must be reproducible.
Decision map
| Need |
Use |
Guard |
| Describe marginal posterior |
Summary/HDI plus MCSE and ESS |
Do not report precision beyond Monte Carlo accuracy. |
| Diagnose MCMC convergence |
Rank R-hat, bulk/tail ESS, MCSE, trace/rank/energy views, sampler stats |
Keep chains separate and inspect parameter coordinates. |
| Diagnose one-chain output |
ESS/MCSE and trace/autocorrelation evidence |
R-hat cannot supply between-chain evidence; obtain more chains where possible. |
| Assess model fit |
Posterior predictive checks |
Choose domain-relevant discrepancy, not only overlapping histograms. |
| Compare predictive models |
PSIS-LOO or justified alternative |
Same observations/target and valid pointwise log likelihood; inspect Pareto-k. |
| Combine independent chains |
Concatenate along chain only after schema/coordinate equality |
Do not concatenate posterior draws along an observation dimension. |
| Combine different groups |
Extend/merge by group under matching variables/coords |
Do not overwrite an existing group silently. |
Read diagnostics and comparison before
turning any single number into a pass/fail conclusion.
Canonical labeled conversion
import arviz as az
import numpy as np
def make_idata(
beta: np.ndarray,
log_likelihood: np.ndarray,
feature_names: list[str],
observation_ids: list[str],
):
if beta.ndim != 3 or log_likelihood.ndim != 3:
raise ValueError("expected chain x draw x domain arrays")
if beta.shape[:2] != log_likelihood.shape[:2]:
raise ValueError("posterior and log likelihood sample axes differ")
if beta.shape[2] != len(feature_names):
raise ValueError("feature coordinate length differs")
if log_likelihood.shape[2] != len(observation_ids):
raise ValueError("observation coordinate length differs")
if len(set(feature_names)) != len(feature_names):
raise ValueError("feature coordinates must be unique")
if len(set(observation_ids)) != len(observation_ids):
raise ValueError("observation coordinates must be unique")
return az.from_dict(
{
"posterior": {"beta": beta},
"log_likelihood": {"outcome": log_likelihood},
},
coords={"feature": feature_names, "observation": observation_ids},
dims={"beta": ["feature"], "outcome": ["observation"]},
)
This assumes arrays arrive as chain × draw × domain. Inspect the installed
converter rather than applying that assumption to every backend. Assert
coordinate lengths and uniqueness before conversion.
Diagnostic contract
- R-hat near one is necessary for many MCMC workflows but not sufficient. Use
rank-normalized and folded variants as supported, plus trace behavior.
- Bulk ESS concerns central estimates; tail ESS concerns quantiles/tails. MCSE
must be small relative to the precision needed for the reported estimand.
- Divergences, energy/BFMI, acceptance, and tree-depth fields live in
sample_stats when the sampler records them. Absence is unknown, not zero.
- Do not average chains before diagnostics or reshape chain × draw to one axis
and then recreate fake chains.
- Diagnostic thresholds are escalation rules, not proof of model validity.
Predictive adequacy and model assumptions remain separate.
Comparison contract
PSIS-LOO requires pointwise log likelihood for the same observation units. Do
not compare models fit to different filtered rows, likelihood factorizations,
response transformations, or weighting conventions without a justified mapping.
On ArviZ 1.x, call loo(..., pointwise=True, var_name=...) when the caller must
retain per-observation Pareto-k, then pass those ELPD results to compare.
compare no longer accepts the 0.x ic="loo" selector. Inspect Pareto-k and
uncertainty; address influential observations or use a more robust validation
plan rather than reporting weights mechanically. Comparison answers relative
predictive performance among candidates, not absolute fit, causality, or
scientific truth.
Use testing and version grounding because
ArviZ 1.x packaging and stats/plot APIs differ from many 0.x examples.
Completion gate
Do not declare completion until provenance is recorded; required groups exist;
each variable's dimensions and coordinates match semantic axes; chains and
draws remain separate; convergence reports combine R-hat, ESS, MCSE, traces,
and available sampler statistics; predictive checks target the stated use;
comparisons use identical observations and valid pointwise log likelihood;
Pareto and uncertainty warnings are surfaced; persisted output reloads with the
same groups/coords; and missing groups or skipped diagnostics are reported.
References
- Inference-data contract
- Diagnostics and comparison
- Testing and version grounding
1---2name: arviz-python3description: Use for writing, reviewing, debugging, or testing Python analysis of Bayesian inference results with ArviZ, including 1.x DataTree groups, legacy InferenceData inputs, xarray dimensions and coordinates, conversion, summaries, R-hat/ESS/MCSE diagnostics, posterior predictive checks, PSIS-LOO, Pareto-k, and model comparison. Trigger on chain/draw shape errors, mislabeled groups, flattened samples, missing log likelihood, or misleading diagnostic claims. Do not use to construct or sample PyMC, NumPyro, or Bambi models, for generic plotting, or for deterministic statistics without Bayesian draws.4argument-hint: "[ArviZ DataTree/InferenceData, diagnostics, predictive check, or comparison]"5metadata:6 source: https://skillmd.com/skills/schattenspiegel/arviz-python7---89# ArviZ Python1011Turn posterior artifacts into correctly labeled inference data, preserve chain12and draw structure, diagnose sampling and prediction, and compare only models13whose observations and log-likelihood semantics are compatible.1415## Boundary1617Use this skill when the input is completed or partially completed Bayesian18draws and the code directly uses ArviZ. Use the modeling framework's skill to19build or sample a model. A plot request alone does not trigger this skill unless20the plot represents posterior, diagnostic, predictive, or comparison results.2122## Know the objects2324| Object | Meaning | Use it for |25|---|---|---|26| ArviZ 1.x `DataTree` / accepted idata-like container | Named inference groups, each holding labeled xarray variables. Legacy integrations may still call the artifact `InferenceData`. | The durable boundary for posterior analysis. |27| Group | A semantic dataset such as `posterior`, `sample_stats`, `log_likelihood`, `posterior_predictive`, `predictions`, `observed_data`, or `constant_data`. | Keeping quantities with different meaning separate. |28| Variable | A named tensor with dimensions and coordinates. | One parameter, statistic, observation, or prediction. |29| `chain` | Independent MCMC run dimension. | Between-chain convergence diagnostics. |30| `draw` | Iteration within a chain. | Within-chain sample sequence. |31| Model-comparison result | Pointwise predictive score summaries, uncertainty, diagnostics, and weights. | Comparing compatible predictive models, not proving truth. |3233Dimension names determine semantics; axis order is secondary. Sample dimensions34must not be confused with event/data dimensions. `observed_data` has no chain or35draw axis; posterior predictive values normally have both plus observation36dimensions. Read [the inference-data contract](references/object-model.md) before37converting arrays, concatenating results, or selecting coordinates.3839## Ordered workflow40411. Establish provenance: model, sampler/guide, package versions, warmup policy,42 number of chains/draws, seed, observation identity, and log-likelihood target.432. Inspect available groups, variables, dimensions, coordinates, dtypes, and44 missing values before computing a statistic. On ArviZ 1.x, treat the45 `DataTree` children as the group registry instead of assuming 0.x46 `InferenceData` attribute behavior.473. Convert raw arrays with explicit group, variable, dims, and coords mappings.48 Preserve chain and draw separately; reject ambiguous flattened samples.494. Select variables/coordinates intentionally. Exclude warmup or transformed50 helper variables only under a recorded policy.515. Diagnose sampling using multiple complementary measures and plots. Report52 problematic variables and coordinates, not only a global maximum.536. Check posterior predictive adequacy with discrepancies tied to the modeling54 objective and distinguish in-sample replicated data from out-of-sample55 predictions.567. For model comparison, verify identical observations, likelihood target, data57 preprocessing, and pointwise `log_likelihood`; inspect Pareto diagnostics and58 score uncertainty before ranking.598. Persist the labeled artifact and provenance when results must be reproducible.6061## Decision map6263| Need | Use | Guard |64|---|---|---|65| Describe marginal posterior | Summary/HDI plus MCSE and ESS | Do not report precision beyond Monte Carlo accuracy. |66| Diagnose MCMC convergence | Rank R-hat, bulk/tail ESS, MCSE, trace/rank/energy views, sampler stats | Keep chains separate and inspect parameter coordinates. |67| Diagnose one-chain output | ESS/MCSE and trace/autocorrelation evidence | R-hat cannot supply between-chain evidence; obtain more chains where possible. |68| Assess model fit | Posterior predictive checks | Choose domain-relevant discrepancy, not only overlapping histograms. |69| Compare predictive models | PSIS-LOO or justified alternative | Same observations/target and valid pointwise log likelihood; inspect Pareto-k. |70| Combine independent chains | Concatenate along `chain` only after schema/coordinate equality | Do not concatenate posterior draws along an observation dimension. |71| Combine different groups | Extend/merge by group under matching variables/coords | Do not overwrite an existing group silently. |7273Read [diagnostics and comparison](references/diagnostics-comparison.md) before74turning any single number into a pass/fail conclusion.7576## Canonical labeled conversion7778```python79import arviz as az80import numpy as np818283def make_idata(84 beta: np.ndarray,85 log_likelihood: np.ndarray,86 feature_names: list[str],87 observation_ids: list[str],88):89 if beta.ndim != 3 or log_likelihood.ndim != 3:90 raise ValueError("expected chain x draw x domain arrays")91 if beta.shape[:2] != log_likelihood.shape[:2]:92 raise ValueError("posterior and log likelihood sample axes differ")93 if beta.shape[2] != len(feature_names):94 raise ValueError("feature coordinate length differs")95 if log_likelihood.shape[2] != len(observation_ids):96 raise ValueError("observation coordinate length differs")97 if len(set(feature_names)) != len(feature_names):98 raise ValueError("feature coordinates must be unique")99 if len(set(observation_ids)) != len(observation_ids):100 raise ValueError("observation coordinates must be unique")101 return az.from_dict(102 {103 "posterior": {"beta": beta},104 "log_likelihood": {"outcome": log_likelihood},105 },106 coords={"feature": feature_names, "observation": observation_ids},107 dims={"beta": ["feature"], "outcome": ["observation"]},108 )109```110111This assumes arrays arrive as chain × draw × domain. Inspect the installed112converter rather than applying that assumption to every backend. Assert113coordinate lengths and uniqueness before conversion.114115## Diagnostic contract116117- R-hat near one is necessary for many MCMC workflows but not sufficient. Use118 rank-normalized and folded variants as supported, plus trace behavior.119- Bulk ESS concerns central estimates; tail ESS concerns quantiles/tails. MCSE120 must be small relative to the precision needed for the reported estimand.121- Divergences, energy/BFMI, acceptance, and tree-depth fields live in122 `sample_stats` when the sampler records them. Absence is unknown, not zero.123- Do not average chains before diagnostics or reshape chain × draw to one axis124 and then recreate fake chains.125- Diagnostic thresholds are escalation rules, not proof of model validity.126 Predictive adequacy and model assumptions remain separate.127128## Comparison contract129130PSIS-LOO requires pointwise log likelihood for the same observation units. Do131not compare models fit to different filtered rows, likelihood factorizations,132response transformations, or weighting conventions without a justified mapping.133On ArviZ 1.x, call `loo(..., pointwise=True, var_name=...)` when the caller must134retain per-observation Pareto-k, then pass those ELPD results to `compare`.135`compare` no longer accepts the 0.x `ic="loo"` selector. Inspect Pareto-k and136uncertainty; address influential observations or use a more robust validation137plan rather than reporting weights mechanically. Comparison answers relative138predictive performance among candidates, not absolute fit, causality, or139scientific truth.140141Use [testing and version grounding](references/testing-version.md) because142ArviZ 1.x packaging and stats/plot APIs differ from many 0.x examples.143144## Completion gate145146Do not declare completion until provenance is recorded; required groups exist;147each variable's dimensions and coordinates match semantic axes; chains and148draws remain separate; convergence reports combine R-hat, ESS, MCSE, traces,149and available sampler statistics; predictive checks target the stated use;150comparisons use identical observations and valid pointwise log likelihood;151Pareto and uncertainty warnings are surfaced; persisted output reloads with the152same groups/coords; and missing groups or skipped diagnostics are reported.153154## References155156- [Inference-data contract](references/object-model.md)157- [Diagnostics and comparison](references/diagnostics-comparison.md)158- [Testing and version grounding](references/testing-version.md)