Tessera

Chad Loder@chadloder.dev · 4mo ago

i prefer the term "podcast intellectuals" all gotcha hot takes, no technical depth

Oh now Thiel is talking about the "reproducibility crisis" in science, amazing Wait these guys are all myna birds, aren't they. They hear a few words said by someone else and they just grab them and repeat them without understanding. Now I see why they love LLMs so much.

Aki Vehtari @avehtari.bsky.social · 4mo ago

New paper "To select or not to select: predictively consistent priors instead of model selection" with Anna Elisabeth Riha, Leevi Lindgren, @davidkohns.bsky.social, @paulbuerkner.com arxiv.org/abs/2606.22850 Model selection is not a substitute for building good models in the first place 1/

To select or not to select: predictively
consistent priors instead of model selection

Anna Elisabeth Riha, Leevi Lindgren, David Kohns, Paul-Christian Bürkner, Aki Vehtari

Bayesian modelling workflows often consider multiple candidate models of varying complexity. Model selection is commonly used to navigate potential trade-offs between model complexity and generalisability to new data. We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and find that the need for selecting simpler models can depend on prior choice. We formalise predictively consistent priors, which keep prior predictive implications stable as model complexity increases. Across examples and numerical experiments, including adding covariates in linear and logistic regression, forward variable selection, and nonlinear modelling, flexible models with predictively consistent priors typically match or outperform selected simpler models in out-of-sample predictive performance. When selection helps, it can indicate poor joint prior implications, such as excessive prior mass on implausible predictive values. Based on our findings, we propose replacing the notion of sparsity or parsimony at the level of model components with specifying priors that remain sensible in predictive space as models become more complex.

Aaron Fisher@aaronjfisher.bsky.social · 4mo ago

bring back old academic twitter controversies to bsky 😈

Sickos Haha Yes

ALT: Sickos Haha Yes

static.klipy.com

"Priority will be placed on studies that rely on indices of emotion that move beyond self-report methodology alone." whut??

Henrik Singmann @singmann.bsky.social · 4mo ago

Finally out in Psychological Review (psycnet.apa.org/doi/10.1037/...), our update to Signal Detection Theory. We show that contrary to the prevailing Gaussian assumption, evidence distributions in recognition memory are likely minimum extreme Gumbel!

Figure 7: Illustration of the Gumbel-min Signal Detection Model
Figure consists of three panels.
Bottom-left panel: The Gumbel-min latent-strength distributions associated with SIGNAL (old) and NOISE (new) items. Top-left panel: The log likelihood ratio (log-LR) for the two latent-strength distributions. Right panel: The receiver operating characteristic function produced by the two latent-strength distributions. pH = hit probabilities; pFA = false-alarm probabilities.

Henrik Singmann@singmann.bsky.social · last yr.

Honey, we fixed Signal Detection Theory (SDT)! In this preprint, Constantin Meyer-Grant, David Kellen, Sam Harding, and I critically evaluate the (unequal-variance) Gaussian SDT model in recognition memory and pursue the Gumbel-min model as a principled alternative: doi.org/10.31234/osf... 🧵

Ricardo Rey-Sáez (β)@ricardoreysaez.bsky.social · 4mo ago

Finally, our paper is out! Together with @aliciafrancomnez.bsky.social, Javier Revuelta, and @mavadillo.bsky.social, we extend hierarchical factor models for skewed RTs and task validation, now in Psychological Methods: doi.org/10.1037/met0... Let me briefly explain what we did and why it matters!

doi.org

Jonas Dora @jonasdora.bsky.social · 4mo ago

I'm really excited to share a new preprint! Lab studies have consistently shown that stress and negative emotions make people drink alcohol, but studies of daily life (EMA) keep failing to find it. What does that mean for the shared theoretical prediction both designs test? osf.io/preprints/ps...

Figure summarizing new EMA data from our lab

Fij @mrfijiwiji.bsky.social · 4mo ago

Historically the word "salad" simply refers to a mixture of chopped cold foods combined with a dressing Therefore, a bowl of cereal with milk is a salad. I call it wheat salad with a milk dressing.

i have a fun one cooking, stay tuned this next week 🤓

John Avocado 宇杰@supercroup.com · 4mo ago

dip-leer

I'm at an #rstats conference today, and the main thing I am learning is the very large number of ways people pronounce "dplyr".

Aki Vehtari @avehtari.bsky.social · 4mo ago

Two recommendations for cmdstanr in case studies: 1) use sample(..., refresh=0) to avoid 80 lines of MCMC progress reporting 2) cmdstanr::print_stan_file() in Quarto with output asis to get color formatting ``` #| output: asis print_stan_file(stan_file) ``` e.g., avehtari.github.io/Bayesian-Wor...

$print_stan_file(bioassay_stan_file) data { int<lower=0> J; vector[J] x; array[J] int<lower=0> N; array[J] int<lower=0, upper=N> y; } parameters { real a; real<lower=0> b; } model { {a, b} ~ normal(0, 5); y ~ binomial_logit(N, a + b * x); }$

Nathaniel Haines@natehaines.bsky.social · 4mo ago

here is the full sequence 🤓

giving a talk later this year and this is gonna be so fun

Thùy Vy T Nguyễn @thuyvytnguyen.bsky.social · 8mo ago

Happy New Year! It is the Year of the Fire Horse. I want to use this turn of the year to share that as of today, I have stepped down from my position as Associate Professor at @durhampsych.bsky.social and will now serve full time as director of my non-profit, @thesolitudelab.bsky.social,...

giving a talk later this year and this is gonna be so fun

Dan Quintana@dsquintana.bsky.social · 5mo ago

the "methods" section is the only thing that separates science from journalism

As long as a paper's methods are described *somewhere* I don't think it matters where they are located in a manuscript

Meme using Norman Rockwell's painting "Freedom of Speech" (1943), showing a working-class man standing up to speak at a town meeting while suited men around him look on. Text reads: "IT IS OK TO HAVE METHODS AT THE END OF A MANUSCRIPT / THIS DOES NOT MEAN THAT METHODS ARE LESS IMPORTANT"

reach out if you or someone you knows wants to chat about doing some modeling together 🤓 my focus is primarily on modeling behavioral paradigms and self-report measures—either together, separately, or even longitudinally bayesianbeginnings.com/?v=2

Bayesian Beginnings — Consulting for Bayesian & Computational Modeling

Helping researchers design, fit, and validate Bayesian and computational models — and build the reproducible pipelines that make the work last.

bayesianbeginnings.com

some big news! hBayesDM has now been fully ported to cmdstan/cmdstanr/cmdstanpy check out the PR below, or the changelog for an overview: PR: github.com/CCS-Lab/hBay... Changelog: ccs-lab.github.io/hBayesDM/new... stay tuned for some additional features coming soon, including covariate support 🤓

feature: hBayesDM version 2.0 by Nathaniel-Haines · Pull Request #182 · CCS-Lab/hBayesDM

hBayesDM 2.0 — modernize stack & toolchain Top-to-bottom refactor of both the R and Python packages onto current Stan tooling, plus the supporting work to keep behavior, docs, and CI in sync. H...

github.com

404 Media@404media.co · 5mo ago

i like the approach here tbh it's rather extreme, but warranted if you can't be bothered to do some basic due diligence on papers whose purpose is to spread knowledge there are plenty of forums for non fact checked, sloppy writing anyway

NEW: ArXiv, the open-access repository of preprint academic research, will ban authors of papers for a year if they submit obviously AI-generated work. www.404media.co/new-arxiv-ru...

Boo!-per Lund@cooperlund.online · 2y ago

same but influencer -> psych researcher 4 humors -> big 5

Someone pointed out that influencers use serotonin/dopamine/cortisol/oxytocin like they’re the four humors and now I can’t stop seeing it

Riccardo Fusaroli @fusaroli.eurosky.social · 5mo ago

even if llms are only good at local interpolation, i think it's easy to lose sight of how vast (and sparse) the sample space is, and therefore how useful even minor interpolation can be

some Bayesian cognitive modeling students are having trouble seeing the industry-relevance of the methods (my worked out examples are research based, since it's what I do). Tips for good industry examples? @ajordannafa.com @natehaines.bsky.social @rmcelreath.bsky.social @avehtari.bsky.social

Berna Devezer @devezer.bsky.social · 6mo ago

📣 The difference between replicable and not replicable is not itself scientifically replicable. 📣 New work with Erkan Buzbas, showing that verdicts such as "X% of results replicated" are based on an inferential machinery that doesn't work. arxiv.org/abs/2604.26268

Screenshot of a paper's title page. Title: "The Difference Between 'Replicable' and 'Not replicable' is not Itself Scientifically Replicable". Authors: Berna Devezer and Erkan O. Buzbas, both at the University of Idaho — Devezer in the Department of Business and the Institute for Modeling Collaboration and Innovation, Buzbas in the Department of Mathematics and Statistical Science. Authors contributed equally. Corresponding author: bdevezer@uidaho.edu.
Abstract: Replication studies estimate the replicability rate of scientific results by aggregating binary verdicts of experiments. Exact replications are rarely attainable, so most replication sequences are non-exact. Experiments differ in ways that matter and do not share a single common data-generating process. We formalize two statistical interpretations of this non-exactness. In a shared latent rate model (benchmark), experiments are exchangeable and depend on a common random replicability rate. In a conditionally independent rates model (operational), each experiment has its own replicability rate drawn independently from a population distribution. Under the shared latent rate model, even small variability among replicability rates induces an irreducible variance floor on the estimated mean replicability rate that cannot be eliminated by adding more replications. Under the conditionally independent rates model, the degree of non-exactness is not identifiable from standard replication data, because one binary verdict per experiment contains no information about between-experiment heterogeneity. Researchers therefore cannot tell which precision regime they are operating in or whether high- and low-replicability sequences can be distinguished in principle. As a result, the usual data structure of one binary verdict per experiment cannot support reliable demarcation between "replicable" and "not replicable" results and systematically understates uncertainty, making high- and low-replicability sequences appear discrim…

disqus just started lighting up my blog comment section with ads unfortunate probably going to just delete the comment section this weekend 😬

Katie Mack@astrokatie.com · 6mo ago

TBF this is a trend that has been going on for quite some time. Statistical software has become so user friendly that you can do some rather complex analyses these days without truly understanding your model Is this a net bad? Hard to say. It’s on users to do their due diligence IMO

"The real threat is a slow, comfortable drift toward not understanding what you're doing. Not a dramatic collapse. Not Skynet. Just a generation of researchers who can produce results but can't produce understanding."

a man in a blue shirt stands next to a woman and a man in a plaid shirt ..

ALT: a man in a blue shirt stands next to a woman and a man in a plaid shirt ..

media.tenor.com

Part 2 of my shrinkage estimator series is out! Part 1 covered the univariate case, but now we dive into multivariate shrinkage 🤓 We cover Spearman's classic correlation disattenuation formula, multivariate James-Stein estimators, and hierarchical methods too haines-lab.com/post/how-to-...

one day I will convince folks to actually use their reliability estimates and not simply report them... one day...

Kevin O’Neill @kevingoneill.github.io · 7mo ago

Oh and can't forget to highlight a (not so?) subtle dig at the recipes academic researchers blindly follow... 😁

it’s hard to overstate how important this is for quantitative research (especially psychology/neuroscience)- computing individual-level stats independently often robs you of hard earned statistical power. highly recommended reading *both* of these wonderful blog posts!

Nathaniel Haines @natehaines.bsky.social · 7mo ago

And in what is my favorite figure ever, we look into how shrinkage impacts person-level means in both the univariate and multivariate case:

Nathaniel Haines @natehaines.bsky.social · 7mo ago

How to Estimate a Correlation, and What It Means for Science | Computational Psychology

Introduction In Part 1 of this series, we showed that six different methods for estimating individual means—James-Stein, classical true score estimation, empirical Bayes, ridge regression, and hierarc...

haines-lab.com