In a new blog, Russell Power explains how the Marin team nearly doubled its sustained TPU usage by creating a custom global scheduler: Iris. Iris searches every region where Marin has compute, places each job wherever capacity appears, and moves data along as needed. 🔗 openathena.ai/blog/cluster...

To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error. Getting there took some work 🧵

Super interested to what degree this interaction can be fine-tuned into models in a non-reversible fashion! Voice cloning is unfortunately a capability which inherently shows up in pretrained audio models. It would be great to be able to largely limit the capability at the level of model weights!

Margaret Mitchell@mmitchell.bsky.social · 9mo ago

🤖 Did you know your voice might be cloned without your consent from just *one sentence* of audio? That's not great. So with @frimelle.bsky.social, we brainstormed a new idea for developers who want to curb malicious use: ✨The Voice Consent Gate.✨ Details, code, here: huggingface.co/blog/voice-c...

Now that school is starting for lots of folks, it's time for a new release of Speech and Language Processing! Jim and I added all sorts of material for the August 2025 release! With slides to match! Check it out here: web.stanford.edu/~jurafsky/sl...

"GPT-5 shows scaling laws are coming to an end"

We’ve discovered a literal miracle with almost unlimited potential and it’s being scrapped for *no reason whatsoever*. This isn’t even nihilism, it’s outright worship of death and human suffering.

Jen Bendery@jbendery.bsky.social · 12mo ago

"The U.S. Department of Health and Human Services (HHS) today announced the beginning of a coordinated wind-down of its mRNA vaccine development activities...." cc: Sen. Bill Cassidy

Really great pointer from Hao Zhang on the other site in relation to GPT OSS use of attention sinks. If I were to guess, the attention sink is what allows them to omit QK-Norm which has become otherwise standard. www.evanmiller.org/attention-is...

Attention Is Off By One

Let’s fix these pesky Transformer outliers using Softmax One and QuietAttention.

evanmiller.org

The SALT Lab is at #ACL2025 with our genius leader @diyiyang.bsky.social. Come see work from @yanzhe.bsky.social, @dorazhao.bsky.social @oshaikh.bsky.social, @michaelryan207.bsky.social, and myself at any of the talks and posters below!

I'm in Vienna for #ACL2025! My work is all presented tomorrow, but today you'll find me today at the poster session from 11-12:30 evangelizing my labmate Yanzhe Zhang's work on his behalf. If you're interested in the risks traditional pop-up attacks present for AI agents, come chat!

A while ago I mentioned that for marin.community project, this gradient increase led to problematic loss ascent which we patched with Z-loss. I was curious, does AdamC just work? So over the weekend, I ran 4 experiments—130M to 1.4B params—all at ~compute-optimal token counts...🧵

Sung Kim@sungkim.bsky.social · last yr.

Why Gradients Rapidly Increase Near the End of Training During long-duration Large Language Model (LLM) training runs the gradient norm increases rapidly near the end of training. In this short note, they show that this increase is due to an unintended interaction between weight decay,

kyutai.org/next/unmute has built in turn-detection on the ASR and full I/O streaming for the TTS. Solves the latency issues that I think are 90% of why people use end-to-end speech models in the first place! From the details, you can @kyutai-labs.bsky.social is focused on real-world utility.

Flattered and shocked for our paper to receive the #facct2025 best paper award.

ACM FAccT@facct.bsky.social · last yr.

🏆 Announcing the #FAccT2025 best paper awards! 🏆 Congratulations to all the authors of the three best papers and three honorable mention papers. Be sure to check out their presentations at the conference next week! facct-blog.github.io/2025-06-20/b...

I've only seen Veo 3 (or any other video generation model) used to produce viral videos. The fake videos seem to successfully trick the majority of commenters and have no visible watermark or disclosure of AI use.

What would you say if you saw it in another country? A senator from a coequal branch of government dragged away by security from asking a question of a Cabinet official

Justin Baragona@justinbaragona.bsky.social · last yr.

Kristi Noem: "We are not going away. We are staying here to liberate the city from the socialists and the burdensome leadership that this governor and that this mayor have placed on this country and what they have tried to insert into the city." Sen. Alex Padilla is then forcibly removed!

🚨 70 million US workers are about to face their biggest workplace transmission due to AI agents. But nobody’s asking them what they want. While AI R&D races to automate everything, we took a different approach: auditing what workers want vs. what AI can deliver across the US workforce.🧵

Really cool to see theory connect to practice! We observed this phenomenon when trying to do deeper WSD cooldowns of our 8B model in the marin.community project! We Z-Lossed our way through the pain, but cool to see some stronger theory: marin.readthedocs.io/en/latest/re...

Sung Kim@sungkim.bsky.social · last yr.

Why Gradients Rapidly Increase Near the End of Training During long-duration Large Language Model (LLM) training runs the gradient norm increases rapidly near the end of training. In this short note, they show that this increase is due to an unintended interaction between weight decay,

What foreign power could do as much damage to the United States as Trump is doing to it right now? www.whitehouse.gov/presidential...

Enhancing National Security by Addressing Risks at Harvard University

BY THE PRESIDENT OF THE UNITED STATES OF AMERICA A PROCLAMATION Admission into the United States to attend, conduct research, or teach at our

whitehouse.gov

Based on current administration policies, China is about to have an influx of returning talent and a accelerated advantage in research investments. You need to be both sinophobic and irrational to expect the US to continue as the global scientific powerhouse with these policy own-goals.

Joshua Weitz@joshuasweitz.bsky.social · last yr.

How bad will it be? Catastrophic. Proposed cuts to #NSF, #NIH, and #NASA will set the US R&D landscape back 25 yrs+, cause economic and job loss now, and undermine innovations to come. But, this is the WH's *proposed* budget. Speak up now before it is too late. (inflation adjusted $-s below)

"“From time-to-time instances will arise in which the society, or segments of it, threaten the very mission of the university & its values... In such a crisis, it becomes the obligation of the university as an institution to oppose such measures & actively to defend its interests and its values.”

DrDinD.bsky.social@drdind.bsky.social · last yr.

Bravo, to Stanford faculty, led by physics, to ask their administrators to stand up and fight Trump. stanforddaily.com/2025/05/22/f...

Super excited Marin is finally out! Come see what we've been building! Code/platform for training fully reproducible models end-to-end, from data to evals. Plus a new high quality 8B base model. Percy did a good job explaining it on the other place. marin.community x.com/percyliang/s...

Percy Liang on X: "What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision: https://t.co/racsvmhyA3" / X

What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision: https://t.co/racsvmhyA3

x.com

How much faster would the science of large-scale AI advance if we could open-source the *process* of building a frontier model? Not just the final models/code/data, but also negative results, toy experiments, and even spontaneous discussions. That's what we're trying @ marin.community

It feels worth conference organizers running a study to see if this significantly impacts reviewer scores. I hope things like this are placebos, but if not we need to seriously consider whether existing peer-review processes for big ML conferences are providing value.

Introducing CAVA: The Comprehensive Assessment for Voice Assistants A new benchmark for evaluating the capabilities required for speech-in-speech-out voice assistants! - Latency - Instruction following - Function calling - Tone awareness - Turn taking - Audio Safety TalkArena.org/cava

Comprehensive Assessment for Voice Assistants

CAVA is a new benchmark for assessing how well Large Audio Models support voice assistant capabilities.

talkarena.org

How does the public conceptualize AI? Rather than self-reported measures, we use metaphors to understand the nuance and complexity of people’s mental models. In our #FAccT2025 paper, we analyzed 12,000 metaphors collected over 12 months to track shifts in public perceptions.

I wrote something up for AI people who want to get into bluesky and either couldn't assemble an exciting feed or gave up doomscrolling when their Following feed switched to talking politics 24/7.

The AI Researcher's Guide to a Non-Boring Bluesky Feed | Naomi Saphra

How to migrate to bsky without a boring feed.

nsaphra.net

Worth noting that a number of universities have now sued over withheld and canceled grants, but no university has yet sued over the arrest, detention, and threatened deportation of its foreign students. www.nytimes.com/2025/04/19/o...

Opinion | Our Foreign Students Are Terrified, and They’re Right to Be

The immigration crackdown has come to America’s campuses.

nytimes.com

Mahmoud Khalil writes movingly about what his detention by ICE means for America: www.washingtonpost.com/opinions/202...

Opinion | Mahmoud Khalil: What does my detention by ICE say about America?

A democracy for some is no democracy at all.

washingtonpost.com

aside: a stunning comment from David Baker, UW professor who won the Nobel Prize in 2024. Now 15 lab members are looking for positions overseas. “There’s so many amazing people who want to come in, & we can’t take them. The Nobel Prize was just a little blip. But things have gotten quite bleak.”

Jeremy Berg@jeremymberg.bsky.social · last yr.

www.nbcnews.com/health/healt...

Financial Times: "Since 1990, America has lost over 5 million manufacturing jobs. In that time, it has gained 11.8 million roles in professional and business services, and 3.3 million in transportation and logistical activities, linked to multinational supply chains." #EconSky

The Model Context Protocol is cool because it gives external developers a way to add meaningful functionality on top of LLM platforms. To limit test this, I made a "Realtime Voice" MCP using free STT, VAD, and TTS systems. The result is a janky, but makes me me excited about the ecosystem to come!