Tessera

Herman

@herman-tesso.bsky.social

Doctoral Researcher at Aalto University's Computer Science department. My research has to see with computational probabilistic modelling.

4 Posts12 Follower 7 Folgt

Herman @herman-tesso.bsky.social · 6mo ago

If you have been using LOO-PIT, this is a must read for you; "LOO-PIT predictive model checking" by me and @avehtari.bsky.social , doi.org/10.48550/arX.... 1/4

Title: LOO-PIT predictive model checking ,

Authors: Herman Tesso and Aki Vehtari ,

Abstract: We consider predictive checking for Bayesian model assessment using leave-one-out
probability integral transform (LOO-PIT). LOO-PIT values are conditional cumulative predictive
probabilities given LOO predictive distributions and corresponding left out observations. For a
well-calibrated model, LOO-PIT values should be near uniformly distributed, but in the finite sample
case they are not independent, due to LOO predictive distributions being determined by nearly the
same data (all but one observation). We prove that this dependency is non-negligible in the finite
case and depends on model complexity. We propose three testing procedures that can be used for
continuous and discrete dependent uniform values. We also propose an automated graphical method
for visualizing local departures from the null. Extensive numerical experiments on simulated and real
datasets demonstrate that the proposed tests achieve competitive performance overall and have much
higher power than standard uniformity tests based on the independence assumption that inevitably
lead to lower than expected rejection rate