Bayesian Optical Flow Eval

bayesian-optical-flow-eval

Bayesian Optical Flow with Uncertainty Quantification — Sun et al. (2016) (arXiv:1611.01230, 2016)

What this evaluates

Evaluates a Bayesian statistical inversion method for estimating optical flow fields and quantifying their uncertainty from image pairs, compared against deterministic baselines.

Datasets

• Synthetic benchmark flow fields — total 5; splits: test (5)
• Middlebury dataset — total 6; splits: test (6)

Metrics

• reconstruction accuracy (primary) — range: other
  • Quantified via scatter plot correlation between the reconstructed second image ($\hat{G}$) derived from the estimated mean flow, and both the noisy reference image ($G$) and the noiseless ground-truth image ($\bar{G}$). Evaluated qualitatively by visual comparison of mean flow fields against true flows.

Input / output format

Input: Grayscale image pair (F, G) at 30x30 (synthetic) or 60x60 (real) resolution, with pixel intensities normalized to [0,1].

Output: Posterior distribution of flow fields p(U,V) via MCMC sampling; mean flow field estimate and pixel-wise uncertainty confidence regions.

Scoring recipe

# 1. Sample posterior p(U,V) using MCMC-Gibbs
samples = mcmc_gibbs_sample(image_pair, priors, hyperpriors)
# 2. Compute mean flow field
mean_flow = np.mean(samples, axis=0)
# 3. Reconstruct second image from mean flow
G_hat = reconstruct_second_image(F, mean_flow)
# 4. Compare G_hat to G (noisy) and G_bar (noiseless) via scatter plot correlation
corr_noisy = pearsonr(G_hat.flatten(), G.flatten())
corr_noiseless = pearsonr(G_hat.flatten(), G_bar.flatten())

Common pitfalls

• MCMC sampling may fail to converge to a stationary distribution; requires restarting from different initial seeds.
• Uncertainty quantification relies on fitting a 2D normal distribution to MCMC samples after discarding initial transients.
• Evaluation is primarily qualitative/visual rather than relying on a single standardized scalar metric.

Evidence (verbatim from paper)

To test the usefulness of the reconstructed flow field, we use it together with the first image F to obtain an estimated second image \hat{G} from equation (38), setting the noise term to be zero. The estimated second image \hat{G} is shown for each example in the third column of Fig.[7] through Fig.[12]. Interestingly, the estimated \hat{G} seems to not only resemble the given noisy image G, but even more similar to the noiseless second image \bar{G}. This observation is confirmed quantitatively, as shown in the last column of Fig.[7] through Fig.[12], where the values of \hat{G}{ij} are plotted against both those of G{ij} and of \bar{G}_{ij}.

Citation

@misc{sun2016bayesian,
  title={Bayesian Optical Flow with Uncertainty Quantification},
  author={Sun et al. (2016)},
  year={2016},
  note={arXiv:1611.01230}
}

• arXiv: 1611.01230