Regular reminder that we have an alt-ARR slack workspace where ACs and SACs can support each other through the sometimes confusing process of the ARR cycle! Post or DM me a good email address for a Slack invitation and I will add you. #EMNLP2026
Marzena Karpinska
@markar.bsky.social
#nlp researcher interested in evaluation including: multilingual models, long-form input/output, processing/generation of creative texts previous: postdoc @ umass_nlp phd from utokyo https://marzenakrp.github.io/
this checkbox at #arr really seems like a get-out-of-jail-free card; if we really need to allow for lang edits, why not require the reviewers to submit their pre-GPTed draft along with the edited one? 😭😭😭
I've been getting tired of people arguing they 'only translated' or 'lightly edited' their reviews/papers #ARR so I decided to check whether that's really detected as 100% AI by @pangram.com ... Spoiler alert, it's not: marzenakarpinska.substack.com/p/no-ai-tran...
No, AI translations are NOT detected as AI generated text
And small language edits give low AI likelihood as well..
marzenakarpinska.substack.com
Text as Data is happening at Berkeley right before COLM. Please share!
Denis@dpeskoff.bsky.social · 3w agoDo you work across computational methods, social sciences, and the humanities? Submit to Text as Data 2026! 📄 One-page submissions 🔓 Non-archival ⏰ Due August 1 📍 October 5 @UCBerkeley tada2026.org
Peer review was one of the most-discussed topics at #ACL2026 . Many folks were concerned about the incredible growth in the number of ARR submissions (17K for the May26 ARR cycle 😱), and even more shocked that ~40% didn't have any authors qualified to review. What is going on?? I did some digging...
hot take: don’t even use it for polishing your reviews. as an AC and SAC, the last thing i care about is your grammar, spelling, or formatting. i get the presentation issues with papers, where you might face reviewers’ language biases, but you really don’t need to worry about this for reviews.
Marzena Karpinska@markar.bsky.social · 4w agoI think I will be posting this after each #ARR cycle: Please 🥺🙏 let's prohibit AI review writing. Otherwise, we will get lazy reviewers claiming they wrote bullet points and used AI only to put that in prose.
I think I will be posting this after each #ARR cycle: Please 🥺🙏 let's prohibit AI review writing. Otherwise, we will get lazy reviewers claiming they wrote bullet points and used AI only to put that in prose.
5 years later MTurk is on its way out. A lot has improved in open-ended gen eval, but it still suffers from underreporting, lack of statistical analysis, and sometimes sloppy design. Perhaps authors should always do their own tasks to understand the implications of how they designed evals.
Ethan Mollick@emollick.bsky.social · 4w agoUnsurprising but still big: MTurk is on its way out, killed by AI. Mechanical Turk was a mainstay of social & survey research through the 2010s, as it allowed you to quickly buy access to many representative humans. It was pretty good at it, until LLMs came along and everyone started using AI
A great lesson learned from #ACL2026 test of time award winner: sometimes it just takes time for people to appreciate your work
In the spirit of #paperoverflow looking for emergency reviewers: - aspect-based summarization - hallucination detection - evaluation invariance - evaluation, evaluation bias - fine-grained evaluation for some, the reviewers filled in delay but became unresponsive later HELP, this house is on fire 🔥
Only 17% of May #ARR authors are qualified for reviewing, and almost 40% of papers have no qualified people?!?!
Spotted by @jennarussell.bsky.social at #ACL2026 A sober reminder to check the output when using Claude et al to generate your presentation. Context, the user likely prompted for 8 min long presentation since that was the limit. The model, just added this silly "8 minutes" to all slides.
For the last few months, I've been working with my new colleague at SFU @markar.bsky.social, students @yvesfrtl.bsky.social, @adampodoxin.bsky.social, Ty Brassington, and Roman Grundkiewicz of Microsoft, trying to understand the quality of machine translation for literary texts. ...
yvesfrtl.bsky.social@yvesfrtl.bsky.social · last mo.📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book
Excited to share this. @neel2112.bsky.social, @mariaa.bsky.social, and I analyzed 500K anonymous ChatGPT convos (shared w/ consent from WildChat) to see if people were generating fiction. We found tons of stories, fanfiction & erotica. Many users iterated on the same stories for days and weeks.
It's invaluable in good literature to have someone who has actual, lived experience and a deep cultural and historical understanding of the literature performing the translation @jricole.bsky.social translation of the Rubaiyat is a great example. Omar's words ring out with truth and subtle humor.
Marzena Karpinska@markar.bsky.social · last mo.A lot of people focus on AI-written fiction, but what about AI literary translation? 📚 We find that AI translation can be readable.. ‼️BUT it also flattens characters' voices 🧙and is less immersive 🫣 than published human translations. Below is my favorite quote from a reader 📖 lait.cs.sfu.ca
A lot of people focus on AI-written fiction, but what about AI literary translation? 📚 We find that AI translation can be readable.. ‼️BUT it also flattens characters' voices 🧙and is less immersive 🫣 than published human translations. Below is my favorite quote from a reader 📖 lait.cs.sfu.ca
yvesfrtl.bsky.social@yvesfrtl.bsky.social · last mo.📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book
📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book
Slop-eds are on the rise and seem to remain undisclosed. Please check out update to our audit of AI in news: ainewsaudit.github.io This work will be presented next week at #ACL2026 (Sunday 4:00-5:30 oral session)
AI News Audit - Search Articles
ainewsaudit.github.io
Jenna Russell@jennarussell.bsky.social · last mo.Unfortunately for @WSJopinion, @nytopinion, and @PostOpinions, slop-eds continue to rise. In June, roughly 6.5% of opinions are flagged for AI-use.
AI NEWS UPDATE We've added 200k+ recent news articles and 5k op-eds from Oct 2025 - June 2026. Local news is up to ~10.81% AI-generated.
One more thing @tuhinchakr.bsky.social 's post reminded me of... people tend to rationalize and see things not there. We saw it already in GPT-2 stories - we *expect* things to *mean* something, so we tend to see things that are not there... (link to this old paper: aclanthology.org/2021.emnlp-m...)
Marzena Karpinska@markar.bsky.social · 2mo agothis is how massive illusion of 'creativity' gets crushed... please read it to understand why models may appear to produce coherent text but are in fact Frankenstein factory ...
this is how massive illusion of 'creativity' gets crushed... please read it to understand why models may appear to produce coherent text but are in fact Frankenstein factory ...
Tuhin Chakrabarty@tuhinchakr.bsky.social · 3mo agoRan some 🧪 to 🔬 why the Granta short story was certainly 🤖 generated. A lot of bad writing happens because AI hasn’t learned aesthetics. It has simply memorized the whole internet and called it a day. So sure, maybe you don't trust AI detectors. But you can trust your own eyes. #AISlop
"Load bearing," "I keep coming back to," "Not just X, but Y" A curse of using AI a lot is that you realize how much of the writing around you is just AI, now People who don't use AI have historically been unable to identify AI prose on sight, but those who use it a lot can spot the tells easily
We were happy to be part of the B.C. AI Research-to-Adoption Summit last week co-hosted by @sfu.ca School of Computing Science and @science.ubc.ca 🎊🎊🎊 It was great to see this event so well attended!
Happy to share our work accepted to @icmlconf.bsky.social 🥳🎉🎉🎉🎉 See you in Korean🇰🇷 Check out 🧵 for more details & links!
Thinking of relocating your lab to Canada? 🇨🇦 There is still time to apply for #impact+ chair position at #SFU (until April 24th) This comes with up to $1M/year research award for 8 years. Feel free to reach out if you have any questions! www.sfu.ca/research/imp...
Canada Impact+ Research Chairs
sfu.ca
Thinking of relocating your lab to Canada? 🇨🇦 There is still time to apply for #impact+ chair position at #SFU (until April 24th) This comes with up to $1M/year research award for 8 years. Feel free to reach out if you have any questions! www.sfu.ca/research/imp...
Canada Impact+ Research Chairs
sfu.ca
