Tessera
Folgen

petersuber

@petersuber.fediscience.org.ap.brid.gy

I work for #OpenAccess to research. Formerly directed the Harvard Office for Scholarly Communication. Now semi-retired but still working for the cause. Also an […] 🌉 bridged from ⁂ https://fediscience.org/@petersuber, follow @ap.brid.gy to interact

1873 Posts2278 Follower37 Folgt
James Smith 💾@floppy.org.uk · last wk.

RE: https://mastodon.me.uk/@Floppy/116940067151749214 This is actually happening; we're building a community to execute a bigotry-free #Rails fork (29 people and increasing at time of writing). Come join us, we need your help and expertise. https://matrix.to/#/#amiko:matrix.org

James Smith 💾@floppy.org.uk · 3w ago

I am ashamed of the fact that I’m stuck on #Rails because of the sunk costs of what I’ve already built, as are *so many* other project like Mastodon. We cannot “just move”, the cost is enormous. The Rails core team seem utterly compromised and under his spell. Nobody will speak out. We need a […]

petersuber@petersuber.fediscience.org.ap.brid.gy · 3w ago

New study: "Here we show, in a large post-publication peer review database, that research assessment is driven more by differences between evaluators than by difference in the evaluated research." https://arxiv.org/abs/2607.09783 #Academia #Assessment #PeerReview #ScholComm #Universities

Judges matter more than papers in post-publication research assessment

Research assessment relies on expert evaluations, yet human judgement is noisy, and it is unclear whether differences in assessment arise primarily from differences in genuine research quality or from unwanted differences between evaluators. While numerous studies highlight disagreement and biases in research assessment, they have not quantified judge-related noise relative to variation in the evaluated works. Here we show, in a large post-publication peer review database, that research assessment is driven more by differences between evaluators than by difference in the evaluated research. We partition variance in 239,521 research quality ratings assigned by 12,649 judges to 193,128 papers from the H1 Connect post-publication peer review platform. Using multilevel models, we decomposed judge-related variation into differences in overall severity and differences in the weighting of scientific attributes. We found that judge-related effects accounted for substantially more variance in ratings than the evaluated papers. In our most detailed model, judge-level effects and judge-specific slopes explained 61% of the total variance, whereas combined paper and journal-level effects accounted for only 7%. By contrast, examined measures of directional bias, such as author gender and global affiliation, explained less than 1% of the variance. We conclude that assessment outcomes were shaped more by the judges than by the papers themselves. Our results demonstrate the necessity of noise audits in high-stakes scientific evaluation.

arxiv.org

Dan Q@dan.m.danq.me.ap.brid.gy · 4w ago

Fucking "app culture" drives me crazy. I got asked to install a mobile app just to look at the itinerary and accommodation information for a trip I'm going on. Why isn't it just a web page? So obviously I said "fuck that noise, hold my beer", reverse-engineered the app, and re-implemented it […]

Original post on danq.me

m.danq.me