In this preprint we evaluate how maladaptive personality traits matter for Statement Validity Assessment (SVA). Traits and facets matter differently when assessing motives for a lie, statement quality or suggestibility. with @jzpsych.bsky.social Feedback warmly welcome! osf.io/preprints/ps...

New preprint: population-representative norms for the PID-5 and its short forms (PID-5-SF, PID-5-BF, PID5BF+M), conditional on age and gender, for the German population. With @andrekerber.bsky.social, Christoph Kemper & Katharina Weweck. osf.io/preprints/ps...

Out now, with @jamiecummins.bsky.social & @uschroeders.bsky.social: why ML research benefits from Registered Reports, and where the format reaches its limits. Table 1 goes through 7 scenarios, from confirmatory prediction to climbing a benchmark leaderboard. Free view-only link: rdcu.be/MCamygrKHKxa

Nature Human Behaviour@nathumbehav.nature.com · 3w ago

How Registered Reports can improve trust in machine learning research

When you answer "I often don't know who I really am" – which time frame are you thinking of? PD diagnoses require symptoms lasting 2+ years, but common questionnaires specify no time frame at all. We tested whether that matters: 🧵 osf.io/zdbmc with @jzpsych.bsky.social and @uschroeders.bsky.social

Psychological Science is running a special issue on "Advancing Psychological Science using Large Language Models", co-edited by me and @ruben.the100.ci. Check out the calls for submissions below! www.psychologicalscience.org/publications...

Item-level prediction models often outperform scale scores in personality research — usually credited to "nuances" (stable item-specific variance). Our new simulation study shows: measurement non-invariance can produce similar predictive gains.

PsyArXivBot@psyarxivbot.bsky.social · 2mo ago

Beyond Nuances: Measurement Non-Invariance as a Source of Item-Level Predictive Gains: https://osf.io/9wtay

Revisiting the Blind Implantation Paradigm: Robustness of False Memory Induction Under Controlled Conditions: https://osf.io/2587c

Imagine my shock. Surely nobody could have predicted this. www.newscientist.com/article/2531...

People training new AI models admit they just get chatbots to do it

The next generation of AI models are meant to be trained by people paid to have conversations with them, but several of these workers have admitted to New Scientist that they simply get chatbots to do...

newscientist.com

I built a website that tracks open source tools that provide automated checks for scientific manuscripts: lakens.github.io/automated_re... It provides a summary through RSS feed that you can subscribe to. It tracks 13 repositories - if you want your repo to be included, let me know!

The paper "The Bots Ruining Social Science Are Not Bots at All" by Jaffe et al. got a lot of attention when it came out in Perspectives on Psychological Science. We read it in our journal club and have a few comments that we now posted to PubPeer pubpeer.com/publications...

1/ Can AI help researchers check whether published social science results actually reproduce? In our new PNAS paper, we tested this directly in the AI Replication Games: 288 researchers, 103 teams, and real replication packages from quantitative social science.

A Framework for Reproducible AI-Assisted Research in the Social and Behavioral Sciences: https://osf.io/xdtqh

Don't be shy to take on a little two-week side project. These five months will be the most precious three years of your academic journey.

Three recent papers examined reproducibility of a large sample of findings. Join this webinar to discuss them and explore where the findings converge and differ. 1. SCORE: www.nature.com/articles/s41... 2. I4R: www.nature.com/articles/s41... 3. Meta-Rep: royalsocietypublishing.org/rsos/article...

Center for Open Science@cos.io · 6mo ago

What can large-scale studies tell us about reproducibility? In our webinar on April 16, researchers from COS, I4R, and META-REP will discuss findings from three papers—one from the recently published SCORE effort—and insights on reproducibility, transparency, and credibility cos-io.zoom.us/webin...

same :)

mareike@mareike.bsky.social · 6mo ago

proud to have contributed a tiny bit by being one of the 865 💪

Today, the SCORE program releases its primary results! 865 researchers examined research credibility across the social and behavioral sciences, publishing three papers in Nature + five preprints. 📑 Explore the papers: www.nature.com/colle... ℹ️ Read more about SCORE: www.cos.io/score

Reliable research in the social and behavioural and sciences

Sweeping new investigations probe the replication, robustness and reproducibility of results across the behavioural and social sciences.

nature.com

-- New Open Research Game -- I reached the rank of Asst. Prof in #TenureRun! 🎓 I published 98 papers and paid $115.000 in APCs to publishers. My grant money is gone and I am exhausted. 💸 How far can you make it? Play now at forrt.org/TenureRun/

Interview and self-report measures of personality disorder criteria are not structurally equivalent. psycnet.apa.org/doi/10.1037/...

delighted to see this published 🙌 using EMA, we (with @roryoc.bsky.social) investigated suicidal ideation in daily life. we found that it arises from an interaction of within-person increases in loneliness and an individual’s level of personality functioning psycnet.apa.org/fulltext/202...

Construct validity is a cornerstone of psychological science. But too often, construct validation practices fall short, leaving uncertainty about what our measures really capture. Why is that? We (@stmllr.bsky.social) tackle this question in our 𝗻𝗲𝘄 𝗽𝗿𝗲𝗽𝗿𝗶𝗻𝘁: 🧵…

The package formerly known as papercheck has changed its name to metacheck! We're checking more than just papers, with functions to assess OSF projects, github repos, and AsPredicted pre-registrations, with more being developed all the time. scienceverse.github.io/metacheck/

Check Research Outputs for Best Practices

A modular, extendable system for automatically checking research outputs for best practices using text search, R code, and/or (optional) LLM queries.

scienceverse.github.io

The Big Three-Mini: Development of a Cross-Culturally Invariant Indicator of Fundamental Character Constructs: https://osf.io/nketr

On Prolific, "we estimate that about 34% of online study participants use LLMs to answer open-ended questions atleast some of the time..." Seems like a very timely paper for behavioural scientists using online samples: osf.io/preprints/so... ; We really need more papers on this issue

Thanks to everybody who chimed in! I arrived at the conclusion that (1) there's a lot of interesting stuff about interactions and (2) the figure I was looking for does not exist. So, I made it myself! Here's a simple illustration of how to control for confounding in interactions:>

Julia M. Rohrer@dingdingpeng.the100.ci · last yr.

Does anybody have a good visualization to explain how interactions can be confounded, and why interactions require interaction controls? @urisohn.bsky.social maybe? (Asking because I have an idea, but want to check out what exists already before investing the effort)

New preprint with @uschroeders.bsky.social. In a simulation study, we explored the predictive performance of ML models across various data conditions typical for psych research. Spoiler: Data quality limits the performance of ML models in a similar way to more "traditional" regression analyses.

PsyArXivBot@psyarxivbot.bsky.social · 2y ago

Beyond the Hype: A Simulation Study Evaluating the Predictive Performance of Machine Learning Models in Psychology: http://osf.io/4vjtx/