Ben Van Calster
@benvancalster.bsky.social
Medical Statistician at KU Leuven. My brain is like a snail but it gets there in the end (or not).
Regarding metric- vs content-based science, I see 3 general positions: - Idealists: contents matter 100%; highly needed but most disappear (frustration, don't get promotions) - Pragmatists: focus on content but use metrics to survive - Conformists: metrics matter 100%
Happy to see this in print! doi 10.1146/annurev-statistics-042324-123749 @maartenvsmeden.bsky.social @laurewynants.bsky.social @vanamsterdam.bsky.social and Ewout Steyerberg
Doug Altman was an internationally renowned statistician who served as The BMJ’s chief statistical adviser. Read about life and work that made this statistician a "citation millionaire" #BMJChristmas www.bmj.com/content/391/...
Our guidance regarding performance measures for medical AI models is finally out! - Stop bashing AUROC, although it does not settle things - Calibration and clinical utility are key - Show risk distributions - Classification statistics (e.g. F1) are improper www.thelancet.com/journals/lan...
Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance
Numerous measures have been proposed to illustrate the performance of predictive artificial intelligence (AI) models. Selecting appropriate performance measures is essential for predictive AI models i...
thelancet.com
Expertise is having fucked up in enough different ways that you become able to anticipate it.
In our latest work, we show that risk estimates for patients are HUGELY uncertain due to model, data, and population uncertainty. Even for well performing models (c statistic, calibration, utility) based on large N. @laure_wynants @ESteyerberg @lasaibarrenada.bsky.social arxiv.org/abs/2506.17141
The fundamental problem of risk prediction for individuals: health AI, uncertainty, and personalized medicine
Background: Clinical prediction models for a health condition are commonly evaluated regarding performance for a population, although decisions are made for individuals. The classic view relates uncer...
arxiv.org
Universities love open science Small print: unless money is involved
Huge variability documented in how publishers respond when informed about a problematic body of work by a research group. www.jclinepi.com/article/S089... #publishers #retractions
Multiple Imputation of Missing Covariates When Using the Fine–Gray Model. Edouard F. Bonneville, Jan Beyersmann, Ruth H. Keogh, Jonathan W. Bartlett, Tim P. Morris, Nicola Polverelli, Liesbeth C. de Wreede, Hein Putter. Statistics in Medicine. onlinelibrary.wiley.com/doi/10.1002/...
Multiple Imputation of Missing Covariates When Using the Fine–Gray Model
The Fine–Gray model for the subdistribution hazard is commonly used for estimating associations between covariates and competing risks outcomes. When there are missing values in the covariates includ...
onlinelibrary.wiley.com
Seconded. Any time someone uses this term, make sure they explain exactly what they mean. If they can't, they are obviously trying to bullshit you.
“Volume is a bad driver,” [Sir Mark Walport] said. “The incentive should be quality, not quantity. It’s about re-engineering the system in a way that encourages good research from beginning to end.” www.theguardian.com/science/2025...
Quality of scientific papers questioned as academics ‘overwhelmed’ by the millions published
Widespread mockery of AI-generated rat with giant penis in one paper brings problem to public attention
theguardian.com
What is common knowledge in your field, but shocks outsiders? Data isn't objective and researchers have innumerable ways to put their thumbs on the scale. Many don't understand statistics well enough to realize they're doing it.
Aaron Sofaer ✍️🏳️⚧️@aaronsofaer.bsky.social · last yr.What is common knowledge in your field, but shocks outsiders? Almost all of the bugs and problems and breakage in the software you use is known to the engineers, we just aren't allowed to fix it. Gotta ship new features.
**New Lancet DH paper** "Importance of sample size on the quality & utility of AI-based prediction models for healthcare" - for broad audience - explains why inadequate SS harms #AI model training, evaluation & performance - pushback to claims SS irrelevant to AI research 👇 tinyurl.com/yrje52fn
Importance of sample size on the quality and utility of AI-based prediction models for healthcare
Rigorous study design and analytical standards are required to generate reliable findings in healthcare from artificial intelligence (AI) research. On…
sciencedirect.com
I recently had a paper in a journal where the fee for open access was lower than the fee for closed access. Is that common? @grahamkendall.bsky.social
Oh, there's an English version as well! DW reports about misconduct at Max Planck Institutes: www.youtube.com/watch?v=n5nE...
How Germany's elite research institution fails young scientists | DW Documentary
YouTube video by DW Documentary
youtube.com
Julia M. Rohrer@dingdingpeng.the100.ci · last yr.TFW Max-Planck-Gesellschaft mal wieder in den Medien für Machtmissbrauch 👀 www.youtube.com/watch?v=cAL4...
PROBAST+AI is out! www.bmj.com/content/388/...
PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods
The Prediction model Risk Of Bias ASsessment Tool (PROBAST) is used to assess the quality, risk of bias, and applicability of prediction models or algorithms and of prediction model/algorithm studies....
bmj.com
I will be presenting our recent work on individual risk estimation uncertainty at ENAR in New Orleans. Come say hi! Work with @benvancalster.bsky.social @laurewynants.bsky.social #DoranneThomassen #EwoutSteyerberg
Quousque tandem? @kuleuvenuniversity.bsky.social @fwovlaanderen.bsky.social A lot of publications in MDPI journals also in Flanders, despite it being listed on predatoryjournals.org.
predatorypublishers.org
Eric Schares@eschares.bsky.social · 2y agoFigure 4 shows the stacked yearly breakdown by publisher and OA type, adjusted to 2023 USD. In 2019 the total stack was ~$910M. MDPI grows very quickly from high volume. Elsevier balances both gold and hybrid, Springer-Nature emphasizes gold, Wiley has more hybrid revenue. #AcademicPublishing
🚨 Preprint! We combine our recent open dataset of #APC prices with the article counts per journal-year from #OpenAlex to estimate how much the academic community has paid in APCs over the last 5 years. A. $8.349 billion ($8.968 B in 2023 USD) $2.5B in 2023 alone. arxiv.org/abs/2407.16551 #metasci
Estimating global article processing charges paid to six publishers for open access between 2019 and 2023
This study presents estimates of the global expenditure on article processing charges (APCs) paid to six publishers for open access between 2019 and 2023. APCs are fees charged for publishing in some ...
arxiv.org
my life would radically improve if i weren’t subjected to microsoft products..,. like my days would feel a 1000 x better
NEW PREPRINT 📊: We propose 3 methods to obtain flexible calibration plots while accounting for clustering: 1. Clustered Group Calibration (CG-C) 2. Two-Stage Meta-Analysis Calibration (2MA-C) 3. Mixed Model Calibration (MIX-C) Ready-to-use R code included!
Ben Van Calster@benvancalster.bsky.social · last yr.We tried to look at ways to obtain flexible calibration plots in clustered (e.g. multicenter) validation studies. Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social arxiv.org/abs/2503.08389
We tried to look at ways to obtain flexible calibration plots in clustered (e.g. multicenter) validation studies. Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social arxiv.org/abs/2503.08389
Clustered Flexible Calibration Plots For Binary Outcomes Using Random Effects Modeling
Evaluation of clinical prediction models across multiple clusters, whether centers or datasets, is becoming increasingly common. A comprehensive evaluation includes an assessment of the agreement betw...
arxiv.org
It is a good day today, I got the reviewer comment "please consult a statistician" again.
'Onze universiteit roept op tot evidence-based beleid, maar zelf volgt de rector zijn buikgevoel' www.veto.be/onderzoek/on...
'Onze universiteit roept op tot evidence-based beleid, maar zelf volgt de rector zijn buikgevoel'
Professor wetenschapsfilosofie Andreas De Block zat acht jaar lang de rectorale denktank rond basisfinanciering voor. Hij blikt teleurgesteld terug op het afgelegde traject: 'De uitkomst moest altijd ...
veto.be
Bad. But how bad? I once asked myself, while finding fakes in the Scientific Report @natureportfolio.bsky.social. So I decide to "do my our research" - I took 100 articles in a row from Physical sciences that contains diffraction. Result - 15 fakes out 100. Details in the 🧵 #ResearchIntegrity 1/x
NEW PREPRINT A detailed overview of 32 popular predictive performance metrics for prediction models arxiv.org/abs/2412.10288
New work in preprint! "Performance evaluation of predictive AI models to support medical decisions: Overview and guidance". Under the wings of the STRATOS initiative. But @maartenvsmeden.bsky.social said it better already 😜 arxiv.org/abs/2412.10288
Performance evaluation of predictive AI models to support medical decisions: Overview and guidance
A myriad of measures to illustrate performance of predictive artificial intelligence (AI) models have been proposed in the literature. Selecting appropriate performance measures is essential for predi...
arxiv.org
Is this something everybody knows already but me? The joy of EHR! www.youtube.com/watch?v=xB_t...
EHR State of Mind | An Electronic Medical Records Parody
YouTube video by ZDoggMD
youtube.com
