Anne-Laure Boulesteix
@boulesteixlaure.bsky.social
Statistician and metascientist. Professor of biometrics at LMU Munich Medical and Mathematical Faculties, committed to open science, member of the Munich Center of Machine Learning. Opinions are mine.
Workshop on “Evidence & Uncertainty in Science: Methodological, Philosophical and Meta-Scientific Issues” 10th & 11th June, Uni of Tübingen, in-person only uni-tuebingen.de/de/281247#c2... With @cruwelli.bsky.social, @boulesteixlaure.bsky.social, @babeheim.bsky.social, & @hendriks.bsky.social
NEW (METASCIENTIFIC) PREPRINT on the exploratory/confirmatory distinction: Title: On "confirmatory" methodological research in statistics and related fields by @iamjulianlange.bsky.social J. Wilcke, @sabinehoffmann11.bsky.social M. Herrmann and myself arxiv.org/abs/2503.08124
On "confirmatory" methodological research in statistics and related fields
Empirical substantive research, such as in the life or social sciences, is commonly categorized into the two modes exploratory and confirmatory, both of which are essential to scientific progress. The...
arxiv.org
NEW METASCIENTIFIC PREPRINT: "The impact of the storytelling fallacy on real data examples in methodological research" by M. Mandl et al: arxiv.org/html/2503.03... or why it is misleading to argue in favor of a method just because one can tell a nice story on the results obtained for n=1 dataset.
The impact of the storytelling fallacy on real data examples in methodological research
arxiv.org
NEW PAPER: Confidence intervals for (e.g., cross-validation) prediction error by H. Schulz-Kümpel, S. Fischer et al. "Constructing confidence Intervals for “the” Generalization Error – a Comprehensive Benchmark Study" openreview.net/pdf?id=x7kCj...
openreview.net
This Registered Report masterpiece just dropped at BMC Biology, brilliantly led by a great team with the help of 300+ analysts & reviewers Same question, same data: go figure! tl;dr: Substantial heterogeneity among results comes from differences among analytical choices 🔗 doi.org/10.1186/s129...
IMO it’s a mistake to give stats to 1st year med students. It’s not why they chose medicine & they resent it. Better to wait until they have developed some curiosity for it. I argued unsuccessfully for this during my time at UCL.
Does #randomization ensures balance of risk factors between groups? Consider this: In Denmark 860 individuals were randomly allocated to either intervention or control. Individuals were unaware of their allocation. No intervention took place. Mortality was higher in the intervention group (p=0.003)
So happy to see you all here! As the @lmu-osc.bsky.social coordinator, I just started a new project: coaching individual research groups so members can switch together to #OpenResearch - a tailored pedagogical intervention to maximise chances of sustainable adoption in the group, and a lot of fun! 🧵
Something that's been bugging me for a while in bioinformatics data analysis is this overreliance on packages, workflows and what's been called "cargo cult science". Can we have more conceptual thinking, more theory? Asking for what we really want to achieve and what we need to do gets us there.
Happy to have been involved in this exciting project on the registration, design and reporting of statistical simulation studies, with @bsiepe.bsky.social @timpmorris.bsky.social et al., appeared in Psychological Methods:
Björn Siepe@bsiepe.bsky.social · 2y agoJournal version at doi.org/10.1037/met0..., final openly available version at doi.org/10.31234/osf.... I have learned a lot from this collaboration, which started with a cold email at the beginning of my PhD - very much worth it
AI researchers: hold my beer — Old model performed slightly not worse when data were generated by the new generative AI method I just made up
Simulation studies are essential for methods research. How well are they conducted & reported? How can we improve their quality? Out now in Psychological Methods, see 🧵 below. With @fbartos.bsky.social, @timpmorris.bsky.social, @boulesteixlaure.bsky.social, @danielheck.bsky.social & Samuel Pawel
Björn Siepe@bsiepe.bsky.social · 3y agoWe reviewed 100 psych. simulation studies & find room for improvement in planning/reporting. As a remedy, we (František Bartoš, @timpmorris.bsky.social Anne-Laure Boulesteix, @danielheck.bsky.social & Samuel Pawel) present ADEMP-PreReg, a simulation study preregistration & reporting template 🧵/1
MEMTAB 2025 pre-conference courses news! 2 options: - An Introduction to Clinical Prediction Models and Sample Size Calculations for Model Development & Evaluation - Systematic Reviews of Prognosis Studies Just £50 when registering for conference! Details 👇 uobevents.eventsair.com/memtab-2025/...
Pre-conference Courses - MEMTAB 2025
uobevents.eventsair.com
We must stand against the arbitrary categorization of continuous variables! ... and that's why I'm proud to announce my support of abolishing time zones in favor of the time gradient
Independent GroupLeader / PI position in AI in Genome Biology, Multimodal Omics at European Molecular Biology Lab (EMBL), Heidelberg! www.embl.org/jobs/positio... Looking in particular for researchers with quantitative/ methodological background who want to dive into leading edge biology research.
For all the new followers here: welcome! Here some recent highlights: (1/4) My eclectic and subjective list of scientific writing tips
Scientific writing tips – Huber Group @ EMBL
An eclectic and subjective list
huber.embl.de
Bias in the evaluation of female academics is hard to remove www.forbes.com/sites/kimels...
College Professors Tried To Reduce Gender Bias In Evaluations—But Couldn’t
Hamilton College researchers tested strategies to reduce gender bias in student evaluations, but their efforts failed, highlighting how deeply ingrained bias remains.
forbes.com
Created a new group replacing and updating a list I enjoyed following on the bird site. Mostly people posting on medical stats and DS/AI ICYI: go.bsky.app/ArqEz36
Medical stats/ds/ai
Join the conversation
go.bsky.app
And there, look out! A meteor! In the form of the BMJ that code sharing will become mandatory. We told you this was coming. Scrutiny of your research data practices. And now it's here. www.bmj.com/content/384/...
Mandatory data and code sharing for research published by The BMJ
New policy requires authors to share analytic codes from all studies and data from all trials The case for sharing data from clinical research is strong.12 Clinical study data include all informatio...
bmj.com
Heinze G, Boulesteix A-L, Kammer M, Morris TP, White IR. ‘Phases of methodological research in biostatistics—Building the evidence base for new methods.’ Proposes a ‘methods-development pipeline’… 10/ doi.org/10.1002/bimj...
Phases of methodological research in biostatistics—Building the evidence base for new methods
Although new biostatistical methods are published at a very high rate, many of these developments are not trustworthy enough to be adopted by the scientific community. We propose a framework to think...
doi.org
Friedrich S, Friede T. ‘On the role of benchmarking data sets and simulations in method comparison studies.’ Nice discussion of simulation studies vs. benchmarking datasets for prediction/classification methodology work. 9/ doi.org/10.1002/bimj...
On the role of benchmarking data sets and simulations in method comparison studies
Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the...
doi.org
Strobl C, Leisch F. ‘Against the “one method fits all data sets” philosophy for comparison studies in methodological research.’ Makes the argument for simulation/comparison studies to ask ‘which methods work well when’ instead of ‘which method is best on average’. 8/ doi.org/10.1002/bimj...
Against the “one method fits all data sets” philosophy for comparison studies in methodological research
Many methodological comparison studies aim at identifying a single or a few “best performing” methods over a certain range of data sets. In this paper we take a different viewpoint by asking whether ...
doi.org
Morris TP, White IR, Cro S, Bartlett JW, Carpenter JR, Pham TM. ‘Comment on Oberman & Vink: Should we fix or simulate the complete data in simulation studies evaluating missing data methods?’ Says when we can fix the complete data and gives many cautions against doing so. 7/ doi.org/10.1002/bimj...
Comment on Oberman & Vink: Should we fix or simulate the complete data in simulation studies evaluating missing data methods?
For simulation studies that evaluate methods of handling missing data, we argue that generating partially observed data by fixing the complete data and repeatedly simulating the missingness indicator...
doi.org
Oberman HI, Vink G. ‘Toward a standardized evaluation of imputation methodology’ Loads of useful and practical advice for simulation studies comparing missing data methods (particular relevance to missing covariate values in observational research). 6/ @oberman.bsky.social doi.org/10.1002/bimj...
Toward a standardized evaluation of imputation methodology
Developing new imputation methodology has become a very active field. Unfortunately, there is no consensus on how to perform simulation studies to evaluate the properties of imputation methods. In pa...
doi.org
Pawel S, Kook L, Reeve K. Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method. Nice table of QRPs, and recommendations to avoid them. 5/ doi.org/10.1002/bimj...
Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method
Comparative simulation studies are workhorse tools for benchmarking statistical methods. As with other empirical studies, the success of simulation studies hinges on the quality of their design, exec...
doi.org
Nießl C, Hoffmann S, Ullmann T, Boulesteix AL. ’Explaining the optimistic performance evaluation of newly proposed methods: A cross-design validation experiment’ Shows how method comparison studies can over-claim and gives some meta-evidence… @boulesteixlaure.bsky.social 3/ doi.org/10.1002/bimj...
Explaining the optimistic performance evaluation of newly proposed methods: A cross‐design validation experiment
The constant development of new data analysis methods in many fields of research is accompanied by an increasing awareness that these new methods often perform better in their introductory paper than....
doi.org