Given how machine learning practices have evolved in the last decade, I was very curious if these findings would hold up. Many thanks to Florian for his nuanced and thorough review. Romantic desire: not predictable in 2017, not predictable in 2026.

ERROR@error.reviews · 5mo ago

New report: Joel, Eastwick, & Finkel (2017) “Is Romantic Desire Predictable? Machine Learning Applied to Initial Romantic Attraction”. Based on the review by @florianpargent.bsky.social, we find Minor Errors that do not affect the core conclusions of the manuscript. osf.io/hxnum

New report: Joel, Eastwick, & Finkel (2017) “Is Romantic Desire Predictable? Machine Learning Applied to Initial Romantic Attraction”. Based on the review by @florianpargent.bsky.social, we find Minor Errors that do not affect the core conclusions of the manuscript. osf.io/hxnum

News from scientific self-correction: Authors pushing to get errors in their papers corrected. Lukas continues to be a role model for how scientists should handle post-publication peer review.

Lukas Röseler@aufdroeseler.bsky.social · 7mo ago

I made a mistake: A researcher who re-used our dataset noticed an inconsistency. We corrected multiple errors and requested a correction at the journal. - Article: doi.org/10.5964/ejop... - Corrected version: osf.io/sz5b9/files/... Luckily, none of the conclusions were affected.

Improving scientific practice can seem daunting. In this fantastic talk (and thread below), Julia Rohrer shares practical ways to communicate methodological insights to a wider audience of researchers.

Julia M. Rohrer@dingdingpeng.the100.ci · 7mo ago

Some people bring up (1) the cost of criticism and (2) that a lot of criticism has already been voiced but ignored. Both points are valid, so here are some suggestion for (1) reducing backlash and (2) increasing impact (from this talk of mine: juliarohrer.com/wp-content/u...

At ERROR, we cannot compete with million-dollar bounties for whistleblowers. But it is great to see sleuthing work rewarded, and institutions admitting when their researchers engaged in misconduct.

Sholto David@sholtodavid.bsky.social · 8mo ago

Dana-Farber Cancer Institute Agrees to Pay $15 Million to Settle Fraud Allegations Related to Scientific Research Grants. The relator [Sholto David] will receive $2,625,000 under today’s settlement. www.justice.gov/usao-ma/pr/d...

Post-publication peer review is at it best when it's thoughtful, scrupulous, steeped in detail – and challenges key claims of the paper. @janhove.bsky.social's discussion of a recent paper on multilingualism exemplifies this.

Jan Vanhove@janhove.bsky.social · 8mo ago

A recent study purports to have found that multilingualism protects against accelerated ageing. I've taken a closer look at it, and it doesn't look good. New blog post: "Does multilingualism really protect against accelerated ageing? Some critical comments" janhove.github.io/posts/2025-1...

Metascientists step up as role models for a healthy error culture in science. Here is a great case where an author and a critical reader collaborated to set the record straight.

Lukas Röseler@aufdroeseler.bsky.social · 8mo ago

I made an error that I found thanks to @lukaswallrich.bsky.social. The code and preprint have been corrected/updated. Effects on false positive rates are smaller and only for alternative and correlation method but I think the overall argument still works.

Voluntary retraction remains a key way to put scientific self-correction into practice. Zhu and Holmes (2024) did the right thing when they realized that some of their results were based on a coding error. Original: psycnet.apa.org/fulltext/202... With retraction: psycnet.apa.org/fulltext/202...

Many error remain to be found in clinical trials. Patients deserve reliable results. Kudos to these authors for their persistent work to correct the record.

Ian Hussey@ianhussey.mmmdata.io · 9mo ago

"In 2019 we notified journals about serious integrity concerns in 172 clinical trials. Over five years later, only 22 have been retracted. The 135 unretracted trials have 1989 citations in systematic reviews, clinical guidelines, and consensus statements" [paraphrased] www.bmj.com/content/390/...

Congratulations to @simine.com for winning the Einstein Foundation Individual Award! 🎉 A well-deserved recognition for her seminal efforts to improve scientific rigor, which includes instituting detailed checks for errors and computational reproducibility at Psychological Science.

I think this is an overly pessimistic take from the @bmj.com. Sharing data does not inherently increase trust, rather it enables verification which allows for trust calibration. This example is a win. Serious issues were rapidly detected that would not have been without mandatory data sharing.

The BMJ@bmj.com · 9mo ago

"The goal is to act in the best interests of the public, to devise more robust processes and new solutions that indeed allow evidence and data to rebuild confidence." @kamranabbasi.bsky.social reflects on the potential and limits of scrutiny in medical research www.bmj.com/content/391/...

Synchronous Robustness Reports could explore implications of different analytical choices – but they could still suffer from bias. Hardwicke argues that preregistration is crucial to prevent it. @tomhardwicke.bsky.social

MetaArXivBot@metaarxivbot.bsky.social · 9mo ago

Risk of bias in robustness reports: https://osf.io/wj26e

Are methodological and causal inference errors creating a false impression that the gut microbiome causes autism? In this strong analysis, Mitchell, Dahly, and Bishop question the evidence. They show that triangulation in science requires multiple robust lines of research.

Cell Press@cellpress.bsky.social · 9mo ago

The link between the gut #microbiome and autism is not backed by science, researchers say. Read the full opinion piece in @cp-neuron.bsky.social: spkl.io/63322AbxpA @wiringthebrain.bsky.social, @statsepi.bsky.social, & @deevybee.bsky.social

a man with a beard and mustache is smiling and looking at the camera .

ALT: a man with a beard and mustache is smiling and looking at the camera .

media.tenor.com

Annie Whamond@anniewham.bsky.social · 11mo ago

Excited to finally have my first paper officially published! "How Do Psychology Journals Handle Postpublication Critique?" is now online with AMPPS. Huge thanks to my supervisors @tomhardwicke.bsky.social and @simine.com and co-authors @bethclarke.bsky.social, N Moodie, S Schiavone, and R Thibault ☺️

New Nature podcast episode about ERROR and the Perspectives on Scientific Error workshop!

Nick Petrić Howe@nickpetrichowe.bsky.social · last yr.

Researchers are trialling all sorts of inventive ways to root out errors. Scientists are human. So, sometimes mistakes slip through the cracks. To make science as robust as possible, though, they've created all sorts of methods to seal those cracks. I explore these in a new @nature.com podcast.

“We pay experts to examine important and influential scientific publications for errors ... We expect most published research to contain some errors ... our reward system pays bonuses to both authors and reviewers even when minor errors are found ..." statmodeling.stat.columbia.edu/2025/07/13/e...

✨ ERROR (@error.reviews) is a bug-bounty program for science that seeks to estimate the prevalence and nature of errors. error.reviews

Home

ERROR is a bug bounty program for science to systematically detect and report errors in academic publications

error.reviews

EU legislation requiring clothes be reused and recycled may be based on a numerical error in a 2017 NGO report where $460 billion was added instead of subtracted. www.frontiersin.org/journals/sus...

Frontiers | The Emperor’s old clothes: a critical review of circular fashion in gray literature

frontiersin.org

Thank you for sharing this! Reposting to note a @nature.com feature from Aug 2024 by @plantnerd.bsky.social on the ERROR project. 🧪 www.nature.com/articles/d41...

Cash for errors: project offers bounty for spotting mistakes in published papers

ERROR project borrows a strategy from the software industry.

nature.com

Eric Hehman@erichehman.bsky.social · 2y ago

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

My first time acting as a recommender for the ERROR project. Huge props to the authors @erichehman.bsky.social et al. for exposing themselves to this detailed post-pub peer review and to Will Lowe @conjugateprior.org for his detailed review and reproduction.

ERROR@error.reviews · 2y ago

New report! Hehman et al. (2018) "Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases". Based on the review by @conjugateprior.org, we find a Major Error that affects a core conclusion. We recommend the authors seek a correction. error.reviews/reviews/hehm...

Really amazingly heartening to see that academic discourse can also be like this

Eric Hehman@erichehman.bsky.social · 2y ago

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

One of our main goals with @error.reviews is to shift scientific culture to identify, acknowledge, and correct errors in the literature. Eric’s response to the recent report on his work is a perfect example of embodying these values. Kudos to the authorship team and reviewer Will Lowe.

Eric Hehman@erichehman.bsky.social · 2y ago

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

It is an honor for anyone to care enough about your work to check it, so thanks dudes. A mistake I suggest others not make is failing to ensure that all the data, from download, to merge, to analysis is available and each step, not just the final analysis, is reproducible.

ERROR@error.reviews · 2y ago

We are grateful to Will Lowe for his diligent examination of the article and its materials, and to @erichehman.bsky.social, @jkflake.bsky.social, & @jimmycalanchini.bsky.social for their willingness to have their impactful work examined for errors.

This sounds like the same error I blogged about a few years ago, the common error of trying to control for population (or body size or many etc) by dividing the outcome variable by it. Props to the authors for seeking review and taking the issue seriously. Role models for us all.

There Are No Magic Outcome Variables

I'm very sorry but I am going to write about statistics and causal inference again. I'd much rather be doing science. I'll make it brief. I was reading a preprint about the relationship between kinsh...

elevanth.org

ERROR@error.reviews · 2y ago

New report! Hehman et al. (2018) "Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases". Based on the review by @conjugateprior.org, we find a Major Error that affects a core conclusion. We recommend the authors seek a correction. error.reviews/reviews/hehm...

A ⭐️⭐️⭐️⭐️⭐️ model response to finding problems in your published research. If you’ve published more than a few papers, you too have almost certainly made at least one mistake, so we should normalise addressing them like this! Great to see @error.reviews approaching error detection so sensitively

Eric Hehman@erichehman.bsky.social · 2y ago

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

THIS is how science works. A great and honorable reaction - especially in these times of resistance to facts and smug self-righteousness. Thanks to @erichehman.bsky.social & @error.reviews!

Eric Hehman@erichehman.bsky.social · 2y ago

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

ERROR@error.reviews · 2y ago

New report! Hehman et al. (2018) "Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases". Based on the review by @conjugateprior.org, we find a Major Error that affects a core conclusion. We recommend the authors seek a correction. error.reviews/reviews/hehm...