Daniel Heck

@danielheck.bsky.social

Professor of Psychological Methods @Phillips-Universtät Marburg Mathematical psychology | Cognitive modeling | Psychometrics | Bayesian statistics Personal: www.dwheck.de Team: https://www.uni-marburg.de/de/fb04/team-heck

At the end of the term I asked my college creative wriing students to submit anonymous thoughts on AI. No real surprises: Mood ranges from resignation to despair, capitulation from embittered erosion of standards to total, feelings of betrayal from deep to furious. 1/

New preprint together with @timangelike.bsky.social: Evaluating Large Language Models for Feature Extraction from Verbal Stimuli: A Simulation-Based Workflow We draw on generalizability theory and the ADEMP-template for simulation studies to assess the multiverse of prompt decisions. osf.io/xphm9

Tim Angelike@timangelike.bsky.social · 2mo ago

In a new preprint with @danielheck.bsky.social, we adapted the ADEMP framework to provide a structured workflow for evaluating LLM feature extraction from verbal stimuli. Link: doi.org/10.31234/osf.... Figure 1 from the paper shows the proposed workflow:

Email: "anbei senden wir Ihnen Ihre Biographie für den Europäischen Gelehrtenkalender. Wir wären Ihnen dankbar, wenn Sie uns etwaige Korrekturen mitteilen könnten." Kleingedrucktes im PDF: "Eintragungspreis Euro 369,-. Durch Rücksendung dieser bestätigten Textvorlage kommt ein Vertrag zustande."😮

Bild

"What follows are the five horsemen of the significance apocalypse [...] To the truly free-spirited researcher, they are a coordinated assault on productivity, publishability, and the ancient scholarly right to discover one’s hypotheses only after the data have had a proper chance to speak." 😅

Robert Böhm@robertboehm.bsky.social · 3mo ago

🚨NEW PREPRINT🚨 together with Jürgen Huber & Michael Kirchler: “Massaging Significance in the Age of Open Science: A Satirical Tutorial,” which is a public service for anyone unwilling to let inconvenient data get in the way of a compelling narrative. 😉 Link to preprint: osf.io/preprints/ps...

Great quote by Cronbach, Gleser & Nanda (1972): "The tidy theory of error laid down for psychology at the start of the century by Spearman and Brown has always seemed just a little too tidy to describe the perverse behavior of real data." 😅

Does it make sense to preregister simulation studies? This question has sparked a lot of debate. ▶️We* work through the why, when, and how ▶️We discuss different phases of methodological research to clarify where preregistration might (or might not) add value 📝 Preprint: doi.org/10.31234/osf...

Diagram showing four phases of methodological research (Theory, Exploration, Systematic Comparison, Evidence Synthesis) with an arrow indicating that preregistration usefulness increases from early to late phases. Each phase lists its aim, elements, outcome, and an example from factor retention research.

New preprint by @semihaktepe.bsky.social 🎉 We compare ANOVA/SDT/GLMM for binary judgments in 20 datasets of the truth effect. #lme4 Main conclusion: "GLMMs are a theoretically sound and practically robust method and thus superior for analyzing binary judgments in social and cognitive psychology.”

Bild
PsyArXivBot@psyarxivbot.bsky.social · 6mo ago

Analyzing Binary Judgments: A Comparison of ANOVA, Signal Detection Theory, and Generalized Linear Mixed Models in the Context of the Illusory Truth Effect: https://osf.io/xn397

New paper by @matzekloft.bsky.social in Psychometrika🎉 We developed a model for aggregating response intervals which are obtained if a participant judges that the word "most" covers the range [86% - 97%]. The model aggregates such intervals while considering people's proficiency & item difficulty.

Matthias Kloft@matzekloft.bsky.social · 9mo ago

The final paper of my dissertation has just been published in Psychometrika (@pmetricsoc.bsky.social). Many thanks to @bsiepe.bsky.social and @danielheck.bsky.social, who co-authored it with me. www.cambridge.org/core/journal...

There still seems to be a lot of confusion about significance testing in psych. No, p-values *don’t* become useless at large N. This flawed point also used to be framed as "too much power". But power isn't the problem – it's 1) unbalanced error rates and 2) the (lack of a) SESOI. 1/ >

Post nicht verfügbar.