Jessica Hullman
@jessicahullman.bsky.social
Ginni Rometty Prof @NorthwesternCS | Fellow @NU_IPR | AI, decisions metascience | Blog @statmodeling substack.com/@jessicahullman | Direct hullmanlab.northwestern.edu
A few recent analyses paint a more skeptical picture of digital twins. It's good to see researchers preregistering new studies, comparing to simpler baselines, & paying more attention to how well they recover individual variation, instead of just how well they predict averages.
Digital twins version 2.0: The skeptics speak up statmodeling.stat.columbia.edu/2026/10/09/d...
AI impact evaluation is still often very rudimentary in practice. I was glad to be part of this preprint led by @rbly.bsky.social on how to think about the right evidence standards for evaluating models in clinical settings.
Robin Blythe@rbly.bsky.social · 3w agoNew #healtheconomics #preprint in which we propose a proportionality argument for evaluation of clinical AI: that if the model leads to changes in the way patients are treated, then it must meet the evidentiary standards of that treatment.
Nihar Shah did a heroic experiment for TMLR: he spent 20-25 hours over two weeks interviewing authors of seemingly low-quality submissions about their own papers. He confirmed what we all suspected: people submitting these papers have *no idea* what is going on in them.
Transactions on Machine Learning Research@tmlrorg.bsky.social · 4w agoTMLR has faced a deluge of submissions, necessitating stricter desk rejection policies due to limited reviewer capacity Co-EiC Nihar Shah reached out to authors of 10 papers slated for desk reject. Could they answer questions about their *own* submission? medium.com/@TmlrOrg/ask...
I'm getting asked to review a LOT of heavily AI-written sloppy papers. Very often from top general science journals. Most have policies to ensure author accountability. They don't seem to be working. Reviewers can incentivize better policy by temporarily creating friction. 1/
My new go-to review request response, sadly: Hi, I declined your request, as the abstract is flagged 100% AI-generated by Pangram. Deciphering what claims authors intended vs originated from AI is not a good use of my time. Jessica Also have one to reply to prospective students
If you feel the moral stakes of this moment are “universities must be defended,” this may not be a reassuring post. It backs up to ask “why,” and finds the answer not self-evident. But strangely, honest wrestling with fundamentals reassures me more in the end than polemic.
Jessica Hullman@jessicahullman.bsky.social · 2mo agoThe uncertain future of academia got me interested in the history of US science policy. It's surprising how fragile the basic vs applied research distinction behind the postwar “social contract for science” is. There are takeaways for what it means to defend universities now🧵
It feels like a time for new stories about how science works. personally really needed to hear someone else articulate how unstable things feel as old arguments about the role of science are disrupted
Jessica Hullman@jessicahullman.bsky.social · 2mo agoThe uncertain future of academia got me interested in the history of US science policy. It's surprising how fragile the basic vs applied research distinction behind the postwar “social contract for science” is. There are takeaways for what it means to defend universities now🧵
🎉 New paper (and data) on editorial & peer review at Science and Science Advances, two elite general science journals. With a wonderful team! @samzhang.bsky.social, @nicklaberge.bsky.social, Quinten and @aaronclauset.bsky.social. 1/ www.science.org/doi/full/10....
The uncertain future of academia got me interested in the history of US science policy. It's surprising how fragile the basic vs applied research distinction behind the postwar “social contract for science” is. There are takeaways for what it means to defend universities now🧵
What stories should we tell about scientific progress now?
In search of the mysterious fruits of basic science
jessicahullman.substack.com
This is an incomplete understanding of academic freedom, and partly confuses academic freedom with freedom of speech. (🧵 adapting some things from this paper, esp. §5: philsci-archive.pitt.edu/28499/. Highly recommend the papers by Dea and Kronfeldner that I cite there!)
philsci-archive.pitt.edu
Sasha Gusev@sashagusevposts.bsky.social · 2mo agoAcademia is a unique space where people can pursue controversial questions wherever they lead. If the work is pseudoscience (and, in my view, Cofnas' race science clearly is), academia has effective channels for response: public criticism and peer-review.
A personal essay about how I’ve been feeling and thinking about this new technology that I’m contributing to and what it might to do to us all. togelius.blogspot.com/2026/08/losi...
Losing my religion
In spring 2025 I had a crisis of faith. I thought about what the technology I'm helping to create might do to our future, and got scared. M...
togelius.blogspot.com
Nice post from @eytan.adar.prof on the cargo cult science of "AI-native" PhD students, who come in super productive because they perform all major parts of research with genAI. No one needs all these quick papers, and they can really hurt students down the road. eytanadar.medium.com/ai-native-ph...
AI-Native PhD Students
If you’re a student and just want the tl;dr, feel free to jump to the end.
eytanadar.medium.com
Eytan Adar@eytan.adar.prof · 2mo agoI wrote a post about the emergence of AI-native students who do their research using GenAI tools. I think that there are some real risks to job prospects and the likely changes in their fields.
This is an excellent conference. I attended a few years ago to present a digital experiment I ran and have been itching for an excuse to return. Call for abstracts now open!
CODE@MIT@codemit.bsky.social · 3mo ago📢 Call for abstracts for the Conference on Digital Experimentation (CODE@MIT), which takes place Nov 13-14. 📨 Submit your best work by *Sept 13th*! ide.mit.edu/events/2026-...
I couldn't make it to #IC2S2 but Huaman Sun will be presenting our work on validation of silicon samples shortly -- if you're attending check it out! Paper: arxiv.org/pdf/2602.15785
ICLR 2027 has authorship quotas: - No more than 20 submissions per author - No more than 1 submission where no author has a paper previously accepted to a major ML conference From iclr.cc/Conferences/...
About a year ago, I wrote skeptically about LLMs in peer review -- not because of skepticism about their inherent capabilities, but because I don't want the research community to optimize for the taste of any one person/system. What's changed since then?
What’s next for machine learning peer review?
A bit over a year ago, I wrote about the dangers of using LLMs for peer review. The most serious concern I had was algorithmic monoculture: the research community would collectively end up optimizing ...
bryanwilder.substack.com
Lol, it's true that Fable-style speak is the closest LLMs have come to poetry
Juan Diego Rodriguez@juand-r.bsky.social · 3mo agoFable: The stall sits at the deep-zone base of each arc slab; before rewriting the scheduler I make it explain itself at the stall, print each pending witness’s candidate decodes and their status, then add the post-pass that fixes one encoding per key against the recorded join order.
1. The new Trump / Kratsios plan for US science appears explicitly designed to destroy US universities. Diverting money to tech donors is probably just a bonus. Gift link.
Trump’s Plan for Science: More Money for A.I., Less for Universities (Gift Article)
Michael Kratsios, President Trump’s science adviser, proposed overhauling how the government funds research. Democrats said Mr. Trump’s actions had weakened science.
nytimes.com
Multiverse Analyses “[Only 6/152 (3.9%)] studies discussed whether their competing specifications were defensible or principled (distinguishing between equivalent, non-equivalent, or uncertain specifications) in the sense of Del Giudice & Gangestad (2021) [screenshot below]” doi.org/10.1177/2515...
Alejandro Sandoval-Lentisco@asandovall.bsky.social · 3mo agoHow often are multiverse analyses implemented, not just cited/discussed? How are they implemented? 🧵 New preprint on the uptake and implementation of multiverse-style analyses 👇 www.biorxiv.org/content/10.6...
I enjoyed this. But the prior that competence and efficiency can't be inspiring or breathtaking is silly. I got hooked on watching Spain after first seeing them play because it felt like they were doing something on par with great poetry or art.
Brian Phillips@brianphillips.lol · 3mo agoI wrote about the game
Out of curiosity I had Fable write a response to the article about the weaknesses of LLMs: www.nytimes.com/2026/06/30/o... It's uh, pretty good. claude.ai/code/artifac...
Opinion | The Unstoppable Force of A.I. Hype Is Meeting One Immovable Fact
Congratulations. You’re irreplaceable.
nytimes.com
Julia's contribution to the special issue (with @jessicahullman.bsky.social and Andrew Gelman) is definitely worth a read if you want a more complete picture of the multiverse and not just my rant: juliarohrer.com/wp-content/u... 4/4
Multiverse analysis, abdication of responsibility and manufacturing of doubt: I have written on some downsides I see with multiverse analysis (which I like in principle): arxiv.org/abs/2607.14623 I was inspired/provoked to write it by @dingdingpeng.the100.ci (thanks!)
Newly proposed rules from the federal government will irreparably damage US science: please express your dissent and post a comment before the OMB public comment period closes this Monday, July 13. Read about the endgame here: www.theguardian.com/commentisfre...
Is the US trying to make scientists’ work so difficult that they simply give up? | Daniel Malinsky
New Trump administration rules would undermine longstanding research practices. It’s death by a thousand cuts
theguardian.com
An essay on sincerity and embarassment and cliché, inspired by a not-very-good documentary about the Counting Crows watched on a flight. jessicahullman.substack.com/p/embarrass-...
Embarrass the sky
What’s the value of sincerity?
jessicahullman.substack.com
An essay on sincerity and embarassment and cliché, inspired by a not-very-good documentary about the Counting Crows watched on a flight. jessicahullman.substack.com/p/embarrass-...
Embarrass the sky
What’s the value of sincerity?
jessicahullman.substack.com
