Research on Research Institute (RoRI)
@rorinstitute.bsky.social
RoRI’s mission is to accelerate transformational research on research systems, cultures & decision-making.
Churn, not crisis, for UK science @profgeraintrees.bsky.social & I have written the editorial in this week’s @science.org, reflecting on early moves by the Burnham government + scope for US-UK cooperation on scale-up finance & AI for science www.science.org/doi/10.1126/... @rorinstitute.bsky.social
Churn, not crisis, for UK science | Science www.science.org/doi/10.1126/... Editorial just out by @profgeraintrees.bsky.social & RoRI’s @jameswilsdon.bsky.social
Churn, not crisis, for UK science
When Andy Burnham became the United Kingdom’s prime minister last week, among his first acts was to abolish the Department for Science, Innovation and Technology (DSIT), folding it into a new supermin...
science.org
As publishers crack down on unapproved AI use in peer review, @jameswilsdon.bsky.social advocates for a more realistic approach in @resprofnews.bsky.social (£): www.researchprofessionalnews.com/rr-news-uk-o...
Warning system cracks down on peer reviewers’ AI use - Research Professional News
Experts debate whether peer reviewers should be allowed to use generative AI as supportive tool
researchprofessionalnews.com
Utilising language models to examine citation timelines, a new study co-authored by RoRI Metascience AI Fellow Basil Mahfouz explores the "evidence lag" in emergency policymaking, revealing how COVID-19 education policies frequently overlooked newly available research: www.cell.com/star-protoco...
COVID-19 education policy often relied on older research despite available new evidence
Policymakers are increasingly relying on scientific research, but it remains unclear whether policy decisions reflect the latest available evidence. Building on a previously developed semantic matchin...
cell.com
LLMs are better at expressing novelty than understanding it. Carlos Arleo shares his work on the Disruption & Novelty Indicator: an experimental tool exploring how structured, constrained AI judgment might evaluate scientific novelty: open.substack.com/pub/research...
Can AI tell if your research is actually new?
What I learned from building an AI novelty detector
open.substack.com
📌 Save the date: Metascience 2027 is heading to Montreal, June 14–16, 2027! The Tanenbaum Open Science Institute (@theneuro.bsky.social) and @cos.io are bringing the global metascience community together in one of the world's top research cities. Updates: metascience.info #Metascience2027
DSIT is dead, but UK science will be fine - writing in @theconversation.com, RoRI’s @jameswilsdon.bsky.social breaks down the UK government's latest Whitehall reorganisation and why the research community shouldn't be alarmed by closer proximity to business: theconversation.com/what-the-sud...
What the sudden demise of the UK’s department of science means for the future of research
If we want to be reminded what real damage to a science system looks like, we only need to glance across the Atlantic.
theconversation.com
DSIT is dead, but UK science will be fine. Keep your eyes on the AI prize. A few thoughts by me @theconversation.com on the reorganisation of Whitehall science theconversation.com/what-the-sud... @ersatzben.com @hetanshah.bsky.social @rorinstitute.bsky.social @profgeraintrees.bsky.social
What the sudden demise of the UK’s department of science means for the future of research
If we want to be reminded what real damage to a science system looks like, we only need to glance across the Atlantic.
theconversation.com
We have expanded RoRI's Atlas of Research Assessment to include expert-curated data on national research assessment systems in Finland 🇫🇮 and South Korea 🇰🇷. Browse the Atlas here: researchonresearch.org/atlas/#atlas...
Writing in @financialtimes.com (£) Anjana Ahuja warns of new US proposals to allow political appointees to veto research grants; RoRI’s @jameswilsdon.bsky.social argues that restricting international partnerships on climate and health is a major blow to global cooperation: www.ft.com/content/e003...
Trump is driving another nail into the coffin of US science
Under proposed reforms, political appointees will review grants to ensure they “advance the President’s priorities”
ft.com
"Evidence and analysis are necessary for good policy making... but they are not sufficient" - Andy Stirling argues that metascience must embrace plural values and uncertainty to truly strengthen both science and democracy: researchonresearchinstitute.substack.com/p/illuminati...
Illuminating neglected uncertainties
How metascience can help rather than hinder democracy
researchonresearchinstitute.substack.com
In a new piece for @wonkhe.bsky.social , RoRI’s André Brasil and Peter Kolarz draw on our latest report to argue that a healthy research system doesn’t strictly need peer review as we know it: wonkhe.com/blogs/peer-r...
How to reduce review burden, improve research culture and strengthen public trust in science
André Brasil and Peter Kolarz set out why we really don't need peer review
wonkhe.com
Outstanding and important *NEW* post on the @rorinstitute.bsky.social Substack from Andy Stirling on the democratic challenges for metascience and its role in informing science policy. open.substack.com/pub/research...
Illuminating neglected uncertainties
How metascience can help rather than hinder democracy
open.substack.com
‘…political appointees patrolling the borders of knowledge.’ @anjahuja.bsky.social on the US OMB’s attempt to rewrite the rules of federal grantmaking giftarticle.ft.com/giftarticle/... t
Trump is driving another nail into the coffin of US science
Under proposed reforms, political appointees will review grants to ensure they “advance the President’s priorities”
giftarticle.ft.com
What happens when research funders treat failure not as a flaw, but as part of the model? Anete Vingre @technopolis-group.com shares data from FWF's 1000 Ideas programme to show how partial randomisation and lotteries change what gets funded: researchonresearchinstitute.substack.com/p/funding-hi...
Funding high-risk research
What changes when the funding rules change?
researchonresearchinstitute.substack.com
If scientific discovery relies on a whole ecosystem, why do we only reward the individual? Similo Ngwenya & Nelius Boshoff view metascience through the African philosophy of Ubuntu ("I am because we are"): researchonresearchinstitute.substack.com/p/i-am-becau...
I am because we are
Ubuntu and what it means for metascience
researchonresearchinstitute.substack.com
Fantastic perspective on peer review by @rorinstitute.bsky.social, with some great ideas for future improvements to the current limping system: doi.org/10.6084/m9.f...
The Future of Peer Review: A system-wide perspective
This report presents the main outcomes of the PEER REVIEW project conducted by the Research on Research Institute (RoRI) from 2023 to 2026. It draws on the substantial literature on peer review as well as 39 in-depth interviews with experts from across the research system, including policymakers, funders, publishers, editors, researchers and others. Peer review is often seen as one of the cornerstones of the research system. At the same time, many researchers and other actors in the system have expressed deep concerns about dysfunctions of peer review. In response to this, much work has been done to assess and improve peer review in individual domains, notably within research funding, scholarly publishing, career advancement, and assessment of research units.However, by considering peer review in each domain in isolation, knowledge about peer review has remained fragmented, offering a weak basis for strengthening the organisation of peer review.In this report, we take a systemic perspective on peer review across domains. Rather than focusing on incremental process-level innovations in individual domains, we conclude there is a need for system-wide reform in peer review. We outline future pathways for such reform.
doi.org
@rorinstitute.bsky.social also called for this in a recent paper, amid fears of a break down in peer review, saying it would require political "bravery" (See www.researchprofessionalnews.com/rr-news-uk-r...) Call me cynical but that level of reform feels like a very distant prospect
‘Bravery’ needed to ditch grants for individual researchers - Research Professional News
Research on Research Institute urges shift to institution-level funding and move beyond “laissez-faire” research system
researchprofessionalnews.com
Great to be among friends at the UK Metascience Unit’s annual cohort meeting, where Ben Steyn is kicking us off with a review of progress. @ukri.org
In a recent piece, @vtraag.bsky.social writes about how finding equal research funding rates between men and women doesn't necessarily mean the academic system is free of gender bias: researchonresearchinstitute.substack.com/p/we-found-n...
A new RoRI study finds no gender differences in funding rates – but warns equal numbers can mask bias, with women representing only 36% of all applicants. Read the paper here: rori.figshare.com/articles/pre...
A great start yesterday to RoRI’s 3-day summit on ‘augmented assessment’ in Cambridge, where we’ve been joined by the architects of research assessment systems from 11 countries, and a fantastic team of metascience experts & facilitators.
RoRI’s @jameswilsdon.bsky.social and @profgeraintrees.bsky.social warn that autonomous AI agents pose an existential threat to research funding by driving grant application costs to zero. They argue assessment must urgently shift away from AI-optimized text to what technology cannot simulate.
LERU Talks - AI and research grants: is the system doomed to collapse?
YouTube video by The League of European Research Universities (LERU)
youtube.com
Speaking to @resprofnews.bsky.social (£), RoRI's Peter Kolarz explains why plummeting success rates and AI pressures mean the current funding treadmill is unsustainable, drawing on solutions from our latest report, The Future of Peer Review: www.researchprofessionalnews.com/rr-news-uk-r...
‘Bravery’ needed to ditch grants for individual researchers - Research Professional News
Research on Research Institute urges shift to institution-level funding and move beyond “laissez-faire” research system
researchprofessionalnews.com
‘Bravery’ needed to ditch grants for individual researchers. Research on Research Institute urges shift to institution-level funding and move beyond “laissez-faire” research system. www.researchprofessionalnews.com/rr-news-uk-r...
‘Bravery’ needed to ditch grants for individual researchers - Research Professional News
Research on Research Institute urges shift to institution-level funding and move beyond “laissez-faire” research system
researchprofessionalnews.com
Is equity in science a beautiful lie? At a high-level summit in Germany, RoRI & British Academy Fellow Similo Ngwenya discussed the structural inequalities embedded in global research systems: researchonresearch.org/equity-in-sc...
Driven by AI surges and tight budgets, global science has become an unsustainable competition machine. @ludowaltman.bsky.social & co-authors argue we must move away from micro-managing individual survival towards strategic leadership: researchonresearchinstitute.substack.com/p/from-hyper...
From hyper-competition to strategic leadership
The role of funders and research institutions in 21st-century science
researchonresearchinstitute.substack.com
Latest episode of my Under Review podcast is out and this time we look at how generative AI has changed the research funding landscape beyond recognition with @jameswilsdon.bsky.social @rorinstitute.bsky.social Ep 7: The death of the funding bid as we know it. www.youtube.com/watch?v=xWbp... 1/2
Under Review with Professor James Wilsdon - The death of the funding bid as we know it
YouTube video by Andy Tattersall
youtube.com
In this panel I will discuss some of the ideas developed in the RoRI Future of Peer Review report, to be published early next week! @rorinstitute.bsky.social @cwts.nl
Charleston Hub@chashub.bsky.social · 2mo agoWhat if the journal article is no longer the center of scholarly communication? 👉 Join us virtually to hear more and be part of the discussion! bit.ly/4or88JG Panelists: @tobyabgreen.bsky.social @johanrooryck.bsky.social @ludowaltman.bsky.social @stockerm.bsky.social
📢 New RoRI Report: The Future of Peer Review Peer review is at a breaking point. Strained by hyper-competition & AI, we need system-wide reform. Read here: rori.figshare.com/articles/report/The_Future_of_Peer_Review_A_system-wide_perspective/32685351?file=65684898
