”Using vibe-coded, unvalidated forensic meta-science tools to evaluate the trustworthiness of published research risks holding authors vs. tools to unequal standards. The solution is less vibe-coding and more validation.”

Ian Hussey@ianhussey.mmmdata.io · last wk.

Our reflections on the recent "AI and Research Integrity" meeting in Montreal: We are somewhat worried that many AI trustworthiness assessment tools are not trustworthy. @lhdjung.bsky.social @jamiecummins.bsky.social trustworthy.scientific.claims/posts/trustw...

Many new forensic meta-science tools aim to detect trustworthiness problems in published research at scale. Yet how trustworthy are these tools themselves? We – @ianhussey.mmmdata.io, @jamiecummins.bsky.social, and I – have some concerns.

Ian Hussey@ianhussey.mmmdata.io · last wk.

Our reflections on the recent "AI and Research Integrity" meeting in Montreal: We are somewhat worried that many AI trustworthiness assessment tools are not trustworthy. @lhdjung.bsky.social @jamiecummins.bsky.social trustworthy.scientific.claims/posts/trustw...

Our reflections on the recent "AI and Research Integrity" meeting in Montreal: We are somewhat worried that many AI trustworthiness assessment tools are not trustworthy. @lhdjung.bsky.social @jamiecummins.bsky.social trustworthy.scientific.claims/posts/trustw...

Trustworthiness assessment tools must be trustworthy

trustworthy.scientific.claims

I sincerely hope this paper will be seen as a turning point in how the medical sciences think and approach the issue of trustworthiness assessment (and I hope the social sciences follow suit!). Congrats to @jdwilko.bsky.social and colleagues (incl. @ianhussey.mmmdata.io and @lhdjung.bsky.social)!

The Cochrane Collaboration@cochrane.org · 3w ago

INSPECT-SR – a tool for assessing the trustworthiness of randomized controlled trials – has been published today in @bmj.com, giving researchers a structured way to identify problematic studies before they're included in systematic reviews. 🔗 www.cochrane.org/about-us/new... #ResearchIntegrity

Fully expecting this to be based on a tiny self-selected sample predisposed to aggression, with participants coached on how to act, exaggerated performances for observers, all run by a host with a flair for showmanship but questionable ethical instincts. You know, a direct replication

DiscussingFilm@discussingfilm.net · last mo.

Netflix has announced a new reality series titled ‘THE NEW STANFORD PRISON EXPERIMENT’. The series follows 30 participants who enter a real prison and are randomly assigned as guards or prisoners. They must see if they conform to the roles or if morality wins.

Machine learning and generative LLMs are subject to many of the same drawbacks and issues as general empirical research in the typical publishing system. In this piece now published at NHB we argue that, like for empirical research, ML and LLMs can benefit from the use of Registered Reports.

Nature Human Behaviour@nathumbehav.nature.com · last mo.

How Registered Reports can improve trust in machine learning research

"Can a program that works in one school district succeed nationwide? That's the generalizability problem" That's an *instance* of the generalizability problem

@some-rss-feeds.bsky.social · 2mo ago

[some-subscribed-rss] New Post: The Generalizability Problem, the Natural Field Experiments Answer, by conversableeconomist https://conversableeconomist.com/2026/08/18/the-generalizability-problem-the-natural-field-experiments-answer/

My favorite Metascience conference will land in Copenhagen next year. The 9th (!) edition of the Perspectives on Science Error conference will be from January 27 to 29. Block your calendar, and see you there! errorsin.science

Today we launch the first stable release of RegCheck: v1.0.0. RegCheck makes it easier and quicker to compare study registrations to published papers for consistency - something we know is important in principle, but rarely done in practice. A 🧵 on what's new: regcheck.app

RegCheck

RegCheck is an AI tool to compare preregistrations with papers instantly.

regcheck.app

Beth continues to kick ass. So glad this kind of permanent academic position exists in metascience, and so glad @bethclarke.bsky.social is paving the way. Hopefully there will be lots more departments adding metasciencey people like her in the years to come! A pretty smart move if you ask me.

Beth Clarke@bethclarke.bsky.social · 4mo ago

Very excited to announce that I’ll be joining @zpid.bsky.social in Trier, Germany as a tenure-track Jun.-Prof. in Psychological Metascience in November! 🤸🏻‍♀️✨

I have managed to abstract the template for replies to critiques.

Our lab loved this paper when it came out as it combined two of our interests, technology effects and schmeasurement. Plus: Registered report, open data. We read it for our journal club and found issues in the Slovak sample that we now posted to PubPeer: pubpeer.com/publications...

Veli-Matti Karhulahti@mkarhulahti.bsky.social · 9mo ago

First post of the year, new paper out today: we present possibly the biggest case of systematic Measurement Schmeasurement in tech use. It seems that most studies on gaming (videogame) addiction/disorder haven't measured gaming after all. This research took years, so long 🧵 doi.org/10.1098/rsos...

What happens when researchers coordinate instead of compete? Our recent Nature World View article reflects on the experience of the Netherlands Metascience Coalition and what it might mean for the way we organize research. www.nature.com/articles/d41...

Don’t compete, collaborate: why collective funding applications are the future

Scientists with disparate expertise writing grants together can identify knowledge gaps and drive progress — but systems must change to incentivize them.

nature.com

Can you identify problems in a paper using only the reported statistics? @lhdjung.bsky.social joins us to discuss tools for assessing the trustworthiness of published research through summary statistics. Register free; open to all www.eventbrite.co.uk/e/the-kings-...

Pure rage bait (laudatory) by @posit.co opensource.posit.co/blog/2026-05...

In Defense of YAML

A common refrain among developers: YAML is bad and TOML is good. This post argues otherwise, tracing the history of configuration formats, examining what YAML 1.2 actually fixed, and introducing py-ya...

opensource.posit.co

If you have access to any the following stats software, would you be willing to run some chi squared tests on a handful of simulated datasets and send me results? I need to verify that that my R implementation of what other software does is correct. - Stata - SAS - StatXact - SPSS DM me :)

What riles me up about “people tried to shut down PP research” is that Cuddy’s own coauthor-turned-critic Carney guest-edited a whole special issue of Registered Reports on Power Posing (incl a meta-analysis), which shows strong evidence against PP even though most authors clearly weren’t sceptics.

Daniel Lakens@lakens.bsky.social · 5mo ago

New blog post: Evaluating Dr. Cuddy’s Claim that the Debunking of Power Posing is a Myth. daniellakens.blogspot.com/2026/05/eval... On an AI generated description of a non-existent study, incorrectly citing findings from studies, and the importance of scientific criticism.

I’m hiring a PhD student! The candidate will work alongside @zefreeman.bsky.social, who is joining our research group as postdoc. jobs.unibe.ch/job-vacancie...

PhD Student in Meta-Science and Clinical Psychology - Universität Bern

Universität Bern is looking for PhD Student in Meta-Science and Clinical Psychology

jobs.unibe.ch

Don't forget to sign up to attend one or multiple online ECR presentations next week! Ten ECR's will present their work in five lunch sessions Monday to Friday They would love your support and feedback! Full program: tinyurl.com/22v38up2 Sign up: forms.office.com/e/MkjYWqTvAw

Very nice INSPECT-SR website containing the guidance and editable template, made by @ianhussey.mmmdata.io inspect.sr

PYMS is organizing online lunch talks from 20th–24th April! 🥪📖 If you‘re an ECR and want to discuss your meta-scientific research (or ideas for it), sign up now! 🚀

Platform for Young Meta-Scientists (PYMS)@youngmetascience.bsky.social · 7mo ago

📢 New from PYMS! We're launching online lunch meetings. A chance to present your work to other metascience ECRs Share your work in any form: finished projects, preliminary results, a trial-run conference presentation, or ideas you'd love feedback on Want to present? Sign up using the form!

Should we use error checking tools like Statcheck and GRIM to automatically check for errors in studies in meta-analyses? I wrote a blog post with some insights from a hackathon where we scrutinized this idea. metaresearch.nl/blog/2026/3/...

Checking for Statistical Errors in Primary Studies: A Way to Improve Meta-Analytic Evidence? — Meta-Research Center

This blogpost was written by dr. Olmo van den Akker. He is a postdoc in our department working on two projects: a systematic review of studies that simulate p-hacking, and a randomized controlled tria...

metaresearch.nl

New report: Joel, Eastwick, & Finkel (2017) “Is Romantic Desire Predictable? Machine Learning Applied to Initial Romantic Attraction”. Based on the review by @florianpargent.bsky.social, we find Minor Errors that do not affect the core conclusions of the manuscript. osf.io/hxnum

I have a data integrity joke, but actually I don't

Sanjay Srivastava@sanjaysrivastava.com · 7mo ago

I have a p-hacking joke but I can't repeat it

python vs R? base R vs tidyverse? framework X vs Y? unless you're getting paid, use the one that brings you the most joy. life's too short to keep on writing code that you neither enjoy nor get paid for. #rstats

New preprint! 🎉 How much post-publication correction takes place in privacy and security research? Short answer: not so much. With Nele Borgert, Luisa Jansen, Theresa Halbritter (none on Bluesky), and @malte.the100.ci

Richard McElreath 🐈‍⬛@rmcelreath.bsky.social · 8mo ago

Yes please let's normalize errors, their discovery, their correction, and their integration into the literature. "Corrections Not Found: Post-Publication Integrity in Usable Security and Privacy Research" (osf.io/preprints/ps...)

I should think twice before having Claude comment on my drafts

The pipe wars, of course, are distinguished by the fact that one side is correct. #rstats

Michael Friendly@datavisfriendly.bsky.social · 8mo ago

If there's been a flare-up of #rstats pipe-wars, consider the war between Newton & Leibnitz over the invention of calculus Leibnitz won in the end because his NOTATION proved to generalize better. medium.com/@vplevris/ne...

My favorites from what I got: - “Nested ifelse() builds character” - "attach() gets a bad rap" - "setwd() isn't that bad" - "Excel is underrated"

Julia M. Rohrer@dingdingpeng.the100.ci · 8mo ago

Planning my next conversation starter

Lagging behind as an #rstats developer since I haven't implemented my own pipe operator