Nari Johnson
@narijohnson.bsky.social
researching AI’s impacts on society 🔍
Pleased to support Mayor Mamdani's PIT Crew initiative. Done right, digital government saves people time and money. Done wrong, it can be costly, useless, or discriminatory. How a city interacts with its people shouldn't be outsourced. It should be local and accountable. www.nyc.gov/mayors-offic...
Mayor Mamdani Launches "Public Interest Technology (PIT) Crew" to Rapidly Build Digital Solutions to Public Problems
nyc.gov
Most Multilingual benchmarks measure what models know, not what they can reliably do. Our #ACL2026 paper introduces functional benchmarks in six languages from English to Yoruba to test whether models can actually execute tasks across languages, not just answer fixed questions about them. 🧵1/n
Attending the @eval-eval.bsky.social workshop at #ACL2026 to present this paper! Happy to chat about all things LLM-as-a-judge + the role of human feedback/participation in designing evals ✨
Nari Johnson@narijohnson.bsky.social · last mo.All AI evaluations embed assumptions about what "good" behavior looks like. Our #FAccT2026 paper explores how we can center the perspectives of impacted communities - specifically, subjects of AI-generated media - in designing LLM-as-a-judge evaluation rubrics.
it's the last day of #FAccT2026 in Montreal (bonjour hiii), and I'm presenting my paper with Diag Davenport on the promise of #PublicInterestTech clinics for training the next gen of critical sociotechnical thinkers. 🏆 plus we got an honorable mention!? come thru @ 10:45! 📄 tiny.cc/pit-clinics-26
Super excited to share a new #FAccT2026 paper with Nel Escher and Nikola Banovic! We show that algorithm auditing policies in the US are wildly insufficient, as they don't account for how public sector algorithms actually function in practice. dl.acm.org/doi/abs/10.1...
Algorithm Auditing Policies Rest on Flawed Assumptions About Public Sector Systems | Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency
You will be notified whenever a record that you have chosen has been cited.
dl.acm.org
RLHF is, instead, a form of survey, which is sensitive to the framing of the solicitation questions and other survey design practices. It's also a form of inverted content moderation - sometimes described as "content moderation via vibes". This moderation operates over generation, not screening.
We want human preferences to be an objective measure of human values, but this view is not grounded in RLHF practice. Instead, the raters who provide preferences are screened to express certain values and are then trained to provide preferences that adhere to some specific criteria. #FAccT2026
All AI evaluations embed assumptions about what "good" behavior looks like. Our #FAccT2026 paper explores how we can center the perspectives of impacted communities - specifically, subjects of AI-generated media - in designing LLM-as-a-judge evaluation rubrics.
yessss! People often don't realize that policy change is often a 2-3 year game of getting stuff drafted and socialized and "waiting in the wings" for the right moment. NOW IS THE TIME
Policymakers by and large don't read academic papers! If you're looking for policy impact, you need to show up. First you need to learn _how_ to show up. Excited to cohost this #FAccT2026 craft session with @rone.bsky.social, @narijohnson.bsky.social, Cynthia Bailey, and Danaé Metaxa.
Nari Johnson@narijohnson.bsky.social · last mo.🔬Science can play a critical role in guiding policy interventions. How can we make our research legible to the policymakers who work on AI? Our tutorial "From FAccT to Policy Impact" explores how we can translate our research in a specific format: the policy brief! #FAccT2026
🔬Science can play a critical role in guiding policy interventions. How can we make our research legible to the policymakers who work on AI? Our tutorial "From FAccT to Policy Impact" explores how we can translate our research in a specific format: the policy brief! #FAccT2026
AI companies are paying screenwriters, lawyers, and other white-collar professionals to produce the training data needed to automate their jobs. I spoke with more than 30 workers about conditions inside this fast-growing and extremely secretive new gig economy.
The laid-off lawyers and PhDs training AI to steal their careers
Experienced white-collar workers are now part of a miserable gig economy.
theverge.com
🎨💻 What is a “high-quality” or “aesthetic” image according to generative AI developers? Happy to share that our investigation of the LAION-Aesthetics Predictor has been accepted at #FAccT2026! 🧵 (1/5) Take a look at a preprint here: arxiv.org/abs/2601.09896
🚨The 2025 AI Agent Index is out! 🚨 Amidst recent buzz over 🦀 and NIST's new agent initiative, we find: - Selective reporting – esp. on safety - Almost all agents backend just 3 model families - Many agents don’t ID themselves as bots online - Big US/China gaps - And more…
We opened up applications for the Brown AI Policy Summer School! Please share with any computing or computational social science students who want to engage substantively with policymaking in the United States: cntr.brown.edu/summer-school. Deadline March 27th!!! Funding available!
CNTR Tech & Policy Summer School
cntr.brown.edu
The deadline for the 2026 FAccT DC is next Tuesday, February 24! If you are a student working on topics relevant to the FAccT's scope, this is an opportunity to interact with a diverse set of peers and mentors! #facct2026 #facct26 #facct Details here: facctconference.org/2026/callfor...
ACM FAccT 2026 Doctoral Colloquium Call for Applications
facctconference.org
Alexandra Olteanu@aolteanu.bsky.social · 7mo agoWe are hoping to see applications from and bring together a diverse group of students across multiple disciplinary areas! If you are a graduate student and interested in FAccT’s scope, the Doctoral Colloquium is for you! #facct2026 #facct26
PhD admissions visits/open houses are starting to happen, and I got a comment on an old Reddit post where I was offering advice, and realized that it's actually really good advice. So here it is! (And this applies whether you've already been admitted to the program or not.) 🧵
Yep and it gets worse! Owner doesn't even care to remove hundreds of skills which directly instruct the model to install malware opensourcemalware.com/blog/clawdbo...
Our call for craft and tutorial sessions for #FAccT2026 is now live! ▶️ Craft CfP: facctconference.org/2026/cfpcraf... ▶️ Tutorials CfP: facctconference.org/2026/cft.html Both kinds of proposals are due March 25!
🎭 How do LLMs (mis)represent culture? 🧮 How often? 🧠 Misrepresentations = missing knowledge? spoiler: NO! At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India 📜 arxiv.org/abs/2511.21322 1/10
Microsoft Research NYC is hiring a researcher in the space of AI and society!
A new report by the Center for Tech Responsibility at Brown University and the ACLU uses computational tools to analyze legislative trends on AI across 1,804 state and federal bills, while offering recommendations for how to integrate the technology into policy analysis.
Making Sense of AI Policy Using Computational Tools | TechPolicy.Press
A new report examines how to use computational tools to evaluate policy, with AI policy as a case study.
techpolicy.press
We are studying the sentiments of visual artists towards generative AI in the workplace and their impacts on creative careers. If you're an artist, please consider filling out this recruitment form for access to our survey! cmu.ca1.qualtrics.com/jfe/form/SV_...
Most LLM evals use API calls or offline inference, testing models in a memory-less silo. Our new Patterns paper shows this misses how LLMs actually behave in real user interfaces, where personalization and interaction history shape responses: arxiv.org/abs/2509.19364
US CAISI is hiring -- the internal govt name for the role is "IT Specialist" but it is effectively a research scientist role! Salary is $120,579 to - $195,200 per year, and you get to work on AI evaluation within government agencies! Job posting (**closes EOD 12/28/2025**): lnkd.in/exJgkqr5
Also, our team is hiring an AI Research Scientist! www.usajobs.gov/job/851528400
USAJOBS connects job seekers with federal jobs across the United States and around the world as the official employment site for the federal government
<p>NIST works with industry and science to advance innovation and improve quality of life. We're looking for an IT Specialist (AI) to join our team!<br> <br> <a href="https://www.nist.gov/caisi">CAISI...
usajobs.gov
Ryan Steed@rbsteed.com · 8mo agoOur team at NIST's Center for AI Standards and Innovation (CAISI) just released a blog post with open questions for AI measurement science: www.nist.gov/blogs/caisi-...
Our team at NIST's Center for AI Standards and Innovation (CAISI) just released a blog post with open questions for AI measurement science: www.nist.gov/blogs/caisi-...
Accelerating AI Innovation Through Measurement Science
Building gold-standard AI systems requires gold-standard AI measurement science – the scientific study of methods used to assess AI systems’ properties and impacts. NIST works to improve measurements ...
nist.gov
Did you know that one base model is responsible for 94% of model-tagged NSFW AI videos on CivitAI? This new paper studies how a small number of models power the non-consensual AI video deepfake ecosystem and why their developers could have predicted and mitigated this.
I appreciate this sympathetic position people's feelings of emotional dependency on these "human-like" bots is real. ridiculing them doesn't help anyone
