Always share a data dictionary along with your data so that people aren't guessing what a variable represents. There are many options for creating one (see thread for examples). Choose which one works best for you and share it alongside your data files. :)

Today we launch the first stable release of RegCheck: v1.0.0. RegCheck makes it easier and quicker to compare study registrations to published papers for consistency - something we know is important in principle, but rarely done in practice. A 🧵 on what's new: regcheck.app

RegCheck

RegCheck is an AI tool to compare preregistrations with papers instantly.

regcheck.app

In a new preprint with Max Littel, Stephanie Koolen-Maas, Hoda Fahimpour, Arthur Gautier and Wahideh Achbari, we revisit research on charitable giving. While replications and retractions are rare, we show how science can be self-correcting through replications. #metascience osf.io/preprints/me...

Progress! The metacheck team led by @lakens.bsky.social and @debruine.bsky.social has worked hard to develop software that runs automated checks of research outputs for best practices. Improve your preprint - fix references, links, p-values, and much more. renebekkers.wordpress.com/2026/07/08/r...

Research Transparency Check Update

Here’s an update on Research Transparency Check: a a software tool that provides automatic assessments of the transparency of data and methods in research reports before they are published an…

renebekkers.wordpress.com

Our next Leibniz Open Science Day is coming up on Nov 17 in Berlin! Submit your paper if you do research on science. We welcome submissions across all areas of meta-science. Keynote by @flxhlzmstr.bsky.social! www.zbw.eu/en/about-us/...

How can AI & LLMs improve how science works? FORRT is delighted to host the AI in Metascience online conference. Please join us! 🗓️ 28–30 Sep 2026 · online · free Talks, workshops & hackathons — all career stages welcome. 👉 Submit by 31 Jul: forrt.org/ai-metascience #Metascience #OpenScience

FORRT AI in Metascience Online Conference | FORRT - Framework for Open and Reproducible Research Training

Call for submissions — FORRT AI in Metascience Online Conference, 28–30 September 2026, online. Deadline 31 July 2026.

forrt.org

"Federal research funding has produced exceptional returns precisely because decisions about what to fund have rested on merit and independent expert judgment, rather than political alignment." @webmz.bsky.social comments on OMB's proposal rule for @cos.io www.cos.io/blog/what-om...

What OMB’s Proposed Grantmaking Revisions Could Mean for Research Integrity

The Center for Open Science (COS) submitted a public comment opposing several proposed revisions in the White House Office of Management and Budget’s proposed Regulation for Federal Financial Assistan...

cos.io

The emerging disenchantment with retrospective meta-analysis www.nature.com/articles/s41... Retrospective meta analysis lacks prior unobservability of data, hence all problems known from empirical research apply. Multilab preregistered studies are seen as best alternative.

New publication by Jasmijn Krujit, @bnbakker.bsky.social et al. in @cogitatiomac.bsky.social: Only a minority of studies in Communication transparently share research materials or data. Way to go ... #openscience www.cogitatiopress.com/mediaandcomm...

AUTOMATING COMPUTATIONAL REPRODUCIBILITY My colleague @philipjakobbln.bsky.social and I are currently engaged in a research project where we reproduce scientific results en masse. To that end, we built rigor.me, a platform for automatically reproducing papers using agents 🧵

New findings from Giving in the Netherlands: in 2024, the Netherlands donated € 5.49 billion. That's $ 6.38 billion or £ 4.77 billion. An increase from 2022, but in terms of GDP a decline from 0.58% to 0.54%. 70% of households give, and 45% of the population volunteers. osf.io/8fvbc/files/...

Our metacheck package now has an online manual! In 29 short chapters we explain how to set up metacheck (running all modules 100% locally if you want), and explain the 18 available modules. www.scienceverse.org/metacheck_bo... We are still in beta, and would love to get feedback!

The Metacheck Manual

A guide to Metacheck, an R package providing a module based tool for checking scientific manuscripts and data repositories for best practices using text search, R code, and optional LLM queries.

scienceverse.org

📢 Call for Papers! We are partnering with the Journal of Economic Surveys on a special issue: Reproducibility and Meta-Science in Economics. Submission deadline: 1 March 2027. Details in thread 🧵

What happens when researchers stop competing for funding? Twenty Dutch metascientists replaced competition with collaboration for NWO grant applications, proving collective approaches can outperform individual ones. @sajedehra.bsky.social @lakens.bsky.social www.tue.nl/en/news-and-...

What happens when researchers stop competing for funding? - tue.nl/stories | funding | open science | Daniel Lakens | Sajedeh Rasti

Twenty Dutch metascientists explore whether collaboration can outperform competition.

tue.nl

Very pleased to have been on the leadership team for this paper! LLMs are already being used all over behavioural science. But it is often pretty hard to work out exactly what has been done, and therefore how much confidence to place in the results. www.nature.com/articles/s41...

A reporting checklist for large language models in behavioural science - Nature Human Behaviour

Large language models offer new opportunities for behavioural science, but their rapid evolution poses challenges for research rigour. We introduce a consensus-based reporting checklist to improve tra...

nature.com

My PhD Sajedeh Rasti has published a World View piece in Nature today: "Don’t compete, collaborate: why collective funding applications are the future" www.nature.com/articles/d41... She discusses how she successfully brought together a team of meta-scientists to collaboratively apply for funding.

Don’t compete, collaborate: why collective funding applications are the future

Scientists with disparate expertise writing grants together can identify knowledge gaps and drive progress — but systems must change to incentivize them.

nature.com

Only 9.9% of top Sociology articles from 2019-2024 provided replication packages. And ≈ half of those with replication packages could not be verified due to missing or incomplete materials. www.econstor.eu/bitstream/10...

1/ Can AI help researchers check whether published social science results actually reproduce? In our new PNAS paper, we tested this directly in the AI Replication Games: 288 researchers, 103 teams, and real replication packages from quantitative social science.

Should automated tools for peer review come with a "nutritional label" for validity?🥪 And if so, what should be on it? Join my workshop on June 5th @ the META/e Conference on Transparency, Technology and AI in Peer Review in Eindhoven! Results will inform our FAIR validation framework. Details >>

Excellent news! Nature is expanding Registered Reports to all the fields in which they publish! A great result by all those proposing this strong solution to publication bias and selective reporting (and getting expert feedback before data collection)! www.nature.com/articles/d41...

Are you in the process of preregistering your study or thinking about performing it in future? The final workshop of this academic year at PMGS is exactly focusing on this. @denolmo.bsky.social will cover core principles of preregistration and common mistakes, and gives a hands-on guide for it.

A master student, Mink Veltman, is helping us to develop a miscitation detection module for Metacheck. If you want to help us to reduce miscitations, it would be great if you could take 5 minutes to fill out this survey: forms.cloud.microsoft/e/J7ugQw8mic Thanks in advance!

Julia M. Rohrer@dingdingpeng.the100.ci · 3mo ago

Sometimes co-authors ask me “did you see what they cited us for???” and of course I didn’t, because if you want to keep faith in humanity, you don’t look up what people cite you for, just like you don’t read comment sections.

I also created a version of the 'guess the correlation' game. Students always enjoyed it, but the original website (with fun retro design) started to require a google account login. So I created my own version. Play it here: lakens.github.io/statistical_...

My article "Three-Sided Testing to Establish Practical Significance" with @jackfitzgerald.bsky.social is now published in AMPPS! journals.sagepub.com/doi/10.1177/... Three-sided testing is an improved version of TOST that lets you test for equivalence, superiority and inferiority simultaneously.

Sage Journals: Discover world-class research

Subscription and open access journals from Sage, the world's leading independent academic publisher.

journals.sagepub.com

I’m working on the revised docs for the upcoming beta release of metacheck and am looking for advice on what kind of questions you’d have and what level of detail you’d want to know about data privacy and AI use.

The tools in Metacheck will be able to help with menial tasks like this. Read in PDFs and automatically check for matching entries in CrossRef. www.scienceverse.org/metacheck/

Julia Lynch@jlynch13.bsky.social · 3mo ago

I have 3 reviews sitting on my desk right now and I’ve been avoiding them bc of how long it’s going to take to look up every reference to see if it’s hallucinated. This is not sustainable.

Reminder: If researchers find Cohen's d = 6, no they didn't. trustworthy.scientific.claims/posts/if-res...

Martin Plöderl@ploederl.bsky.social · 4mo ago

1. The paper with the implausibly large effects of Omega-3 fatty acids on mental health was now retracted. A little thread on the process where @ianhussey.mmmdata.io and I was involved. www.sciencedirect.com/science/arti...

Recently, I got quite frustrated listening to a large group of economists discussing preregistration and pre analysis plans. So, I've channelled that into an argument for why they should be preregistering their work whenever performing confirmatory research kdoroc.substack.com/p/arguing-wi...

Arguing with economists: the case for preregistration

At a recent economics conference there was a long discussion on preregistration where we heard questions and comments from ~20 different people at varying levels of seniority.

kdoroc.substack.com