Aki Vehtari

@avehtari.bsky.social

Professor in computational Bayesian modeling, Aalto University, Finland. Co-author of Bayesian Data Analysis 3rd ed, Regression and Other Stories, Active Statistics and Bayesian Workflow. #mcmc_stan and #arviz developer. https://users.aalto.fi/ave/

For examples of the priorsense package in use, see e.g. chapter 17 case study ("sleep study") from the Bayesian Workflow website: avehtari.github.io/Bayesian-Wor...

figure 10 from the linked page, showing prior sensitivity checks for three parameters
Aki Vehtari@avehtari.bsky.social · last wk.

priorsense R package has now JOSS paper "priorsense: Efficient prior and likelihood sensitivity checks for Bayesian models in R" you can cite, too doi.org/10.21105/jos... with Noa Kallioinen, Topi Paananen, and @paulbuerkner.com priorsense achieved also a gold badge from @ropensci.org review!

📘 With the release of our textbook "Bayesian Workflow" (avehtari.github.io/Bayesian-Wor...), I figured I'd also share the content of my graduate course on the topic at UBC. 🌎 charlesm93.github.io/stat547/ The course contains overlapping and complementary material, homeworks and reading.

Bayesian Workflow book: Website – Bayesian Workflow book

Website for the Bayesian Workflow book by Gelman, Vehtari, McElreath, et al. — case studies, code, and exercises in R and Stan.

avehtari.github.io

"We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and.. need for selecting simpler models can depend on prior choice..predictively consistent priors, which keep prior predictive implications stable as model complexity increases." 🧪

Figure 3. Illustrative example: Part 1. Schematic illustration of model performance relative to the oracle model for increasing true effect size in the finite-data regime, comparing the full (true-structure) model (M1), the intercept-only model (M0), Bayesian model averaging and stacking, as well as selecting a model with Bayes factor or PSIS elpdloo. This is a simplified summary based on the results of simulated experiments in Appendix D. For small true effect sizes, the true-structure model can overfit, but, over the considered range, the full model has the best expected performance.
Aki Vehtari@avehtari.bsky.social · last mo.

New paper "To select or not to select: predictively consistent priors instead of model selection" with Anna Elisabeth Riha, Leevi Lindgren, @davidkohns.bsky.social, @paulbuerkner.com arxiv.org/abs/2606.22850 Model selection is not a substitute for building good models in the first place 1/

To select or not to select: predictively
consistent priors instead of model selection

Anna Elisabeth Riha, Leevi Lindgren, David Kohns, Paul-Christian Bürkner, Aki Vehtari

Bayesian modelling workflows often consider multiple candidate models of varying complexity. Model selection is commonly used to navigate potential trade-offs between model complexity and generalisability to new data. We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and find that the need for selecting simpler models can depend on prior choice. We formalise predictively consistent priors, which keep prior predictive implications stable as model complexity increases. Across examples and numerical experiments, including adding covariates in linear and logistic regression, forward variable selection, and nonlinear modelling, flexible models with predictively consistent priors typically match or outperform selected simpler models in out-of-sample predictive performance. When selection helps, it can indicate poor joint prior implications, such as excessive prior mass on implausible predictive values. Based on our findings, we propose replacing the notion of sparsity or parsimony at the level of model components with specifying priors that remain sensible in predictive space as models become more complex.