I think this is kinda sorta how we did it so I can RT it.

Jamie Cummins@jamiecummins.bsky.social · 2d ago

Have you built an LLM-based research tool and had to stumble your way through figuring out how to validate it? Me too! So I wrote a guide on how to systematically approach this. Preprint here: osf.io/preprints/ps...

Anyway, this is just another sign that the world needs a @dingdingpeng.the100.ci blogpost on immortal time bias 🙇‍♀️

Also, a reminder that if you notice any factual errors I make in my published work, please tell me - I'll pay you a reward! I've found most of them myself so far, so I just donate extra instead. But I can't tell if this is because others haven't noticed them or are unwilling to tell me - please do!

About - Scientific Discovery

Tracing the steps towards scientific progress. Click to read Scientific Discovery, by Saloni Dattani, a Substack publication with tens of thousands of subscribers.

scientificdiscovery.dev

Psychological Science is running a special issue on "Advancing Psychological Science using Large Language Models", co-edited by me and @ruben.the100.ci. Check out the calls for submissions below! www.psychologicalscience.org/publications...

my kid likes making stop motion movies. the apps I found cost money, had ads or were not very child-friendly, so I had claude fable one-shot a cross-platform PWA rubenarslan.github.io/trickfilm/ works a charm :-) and completely local

The first official data on the effectiveness of Australia's social media ban is out. Nothing unexpected, but it's worth reading. www.esafety.gov.au/research/soc...

New blog post! Odds ratios are...a bit weird. Here I explain two problems (interpretation, non-collapsibility) and provide some simple recommendations for improved reporting. www.the100.ci/2026/07/30/r...

Reviewer notes: Odds ratios are really odd

When psychological researchers investigate binary outcomes, they routinely report odds ratios to quantify the strength of an effect or an association. This is quite understandable given that psycholog...

the100.ci

New preprint w/ Leo Tiokhin & @lakens.bsky.social! Registered Reports (RRs) are great for science b/c publication is guaranteed before results are known, reducing publication bias & QRPs. This feature supposedly also makes them great for *scientists*. But is that really true? 1/ tinyurl.com/4sy993rr

Career incentives make Registered Reports unattractive under realistic academic conditions

Registered Reports are a publication format designed to reduce publication bias by guaranteeing publication before results are known. This guarantee is considered attractive in the prevailing ‘publish...

tinyurl.com

Roses are red violets are lame

Orcas seen ramming prey so hard it explodes may be playing game

Scientists believe sunfish-smashing behaviour could be form of entertainment or to help younger whales feed

theguardian.com

New paper out now 🥳 When psychologists discuss generalisability, they often refer to vague notions of representativeness. We provide an accessible intro to the total survey error framework as a tool to reason about this more rigorously. w @taymalsalti.bsky.social @ruben.the100.ci >

Today in schmeasurement: Critical methods review of literature on 'double empathy' in autism concludes that, of 20 studies, > the experimental measures in these studies are largely not making measurements that would detect cognitive or affective empathy. link.springer.com/article/10.1...

Cognitive Empathy, Affective Empathy and Empirical Research on the Double Empathy Problem: A Critical Methodology Review - Journal of Autism and Developmental Disorders

Purpose The double empathy problem posits that autistic people and non-autistic people both lack empathy with the other side rather than the lack of empathy solely residing in one side. There have bee...

link.springer.com

great thread

Quentin André@quentinandre.bsky.social · 3w ago

I wish journalists reported more carefully on science. I really do. A short thread. The study cited at in the intro of this article is a mice study. Researchers randomize male to have access to a low/med/high amount of ethanol... 1/N www.theatlantic.com/family/2026/...

Yet the effects on morphological differences are HUGE. I'm talking p = 2.1 × 10⁻⁵⁴. This alone should raise a giant red flag: How can the downstream effects of alcohol availability be several dozens of magnitudes larger than the upstream effects? 4/N

Our article introducing the #openESM database for experience samling data, led by @bsiepe.bsky.social, has now been published 🎉 Paper: link.springer.com/article/10.3... openESM interactive website: openesmdata.org GitHub repository: github.com/openesm-proj...

7/🧵 Here's what a full RegCheck workflow looks like from scratch - including linking a registration directly from OSF, and navigating the RegCheck report. Completed end-to-end in about 5 minutes in real time.

6/🧵 But our goal with RegCheck has always been to facilitate human comparisons. To this end, we now also include the 'Evidence' viewer: where you can view documents side-by-side, with the specific quotes from the documents highlighted in-context.

If you were planning on secretly deviating from your preregistration, you might wanna change plans.

Jamie Cummins@jamiecummins.bsky.social · 3w ago

Today we launch the first stable release of RegCheck: v1.0.0. RegCheck makes it easier and quicker to compare study registrations to published papers for consistency - something we know is important in principle, but rarely done in practice. A 🧵 on what's new: regcheck.app

But why? In a brilliant new video, Justin Portela goes much deeper into the history than I'd ever known. I highly recommend watching it. www.youtube.com/watch?v=BJYf...

The Deadliest Decision You've Never Heard of

YouTube video by Justin Portela

youtube.com

Once again can confirm that reading Dubief’s “Precious little sleep” is the single best parenting hack that I know ✨

Looks like a cool analysis, but note that it excludes preprints that were never published. So this last sentence in the abstract amounts to ‘conditioning on preprints accepted by the scientific community, preprints are good enough to be accepted by the scientific community’.

Hao Yin@haoyin.bsky.social · 3w ago

Nature News covered our preprint on LLM analysis of Preprint-to-Publication transition 😆👇 Think preprints are unreliable? Analysis of 70,000 studies might change your mind www.nature.com/articles/d41... Kudos to @rrust.bsky.social for leading this project👍 www.biorxiv.org/content/10.6...

not being able to use subs with a third party harness is why i stopped going to german night clubs

Scoiattolo@scoiattolo.mountainherder.xyz · 4w ago

Honestly, the real reason to cancel is that Claude Code is an awful harness that hinders the true potential of the models, and one of the main reasons I'm canceling is that they don't let you use subs with third party harnesses.

Is it time to tell the biological aging clocks people about the age-period-cohort problem or are they still busy overfitting breakpoint regressions in tiny cross-sectional convenience samples?

I don't know how people can look at this and think to themselves "that seems plausible" (rather than e.g., maybe this is an artefact of participation into the samples) Demographers, have you ever seen a health trend over age that looks like this?

WE ARE HIRING! Come work with us (me + @malte.the100.ci) on developing and validating RegCheck. PhD or postdoc applications welcome. Folk with backgrounds in meta-science, psychology, (pre)clinical trials, and/or economics are especially invited to apply. jobs.unibe.ch/job-vacancie...

PhD Student/ Postdoc Position in Meta-Science - Universität Bern

Universität Bern is looking for PhD Student/ Postdoc Position in Meta-Science

jobs.unibe.ch

Nice replication with Norwegian twin data of @pamherd.bsky.social et al.'s polygenic-score-based result that heritability of education increased for women over time with increasing gender equality. www.nature.com/articles/s41... by @olerogeberg.bsky.social @kph3k.bsky.social @torkildl.bsky.social

Our department just announced a new Associate Professor position in quantitative research methodology 🇳🇴 One of the favorable qualifications is experience with open science practices + R/Python experience www.jobbnorge.no/en/available...

Associate Professor in Quantitative Research Methodology (302697) | University of Oslo

Job title: Associate Professor in Quantitative Research Methodology (302697), Employer: University of Oslo, Deadline: Tuesday, September 1, 2026

jobbnorge.no

Interested in doing a postdoc with me in Auckland, New Zealand? 🇳🇿 See info below on a competitive award opportunity, and please spread the word!

Julia Stern is hiring! Strongly recommend this opportunity. I know her from grad student days and am confident she's a good one!

Julia Stern@juliastern.bsky.social · last mo.

I'm hiring! I am currently establishing my new Research Group for Personality Psychology at Ulm University and am looking for a motivated researcher to join the team starting October 1, 2026 (or by arrangement). 🧵👇 (1/4) 🔗 Full description and details: stellenangebote.uni-ulm.de/jobposting/a...

The good news is that someone else now gets to join this amazing team! If you're keen to help develop RegCheck, I'd strongly encourage you to apply. Working with @jamiecummins.bsky.social and @malte.the100.ci is reason enough to apply! jobs.unibe.ch/job-vacancie...

PhD Student/ Postdoc Position in Meta-Science - Universität Bern

Universität Bern is looking for PhD Student/ Postdoc Position in Meta-Science

jobs.unibe.ch