It is a major policy failure that the US cannot accommodate top AI conferences due to visa issues. buff.ly/DRJOGrB

bring back 8 page neurips papers

m̶e̶n̶ Americans will literally l̶e̶a̶r̶n̶ ̶e̶v̶e̶r̶y̶t̶h̶i̶n̶g̶ ̶a̶b̶o̶u̶t̶ ̶a̶n̶c̶i̶e̶n̶t̶ ̶R̶o̶m̶e̶ invest billions into self driving cars instead of g̶o̶i̶n̶g̶ ̶t̶o̶ ̶t̶h̶e̶r̶a̶p̶y̶ building transit

bring back length limits for author responses

in llm-land, what is a tool, a function, an agent, and (most elusive of all): a "multi-agent system"? (This had been bothering me recently; are all these the same?) @yoavgo.bsky.social's blog is a clarifying read on the topic -- I plan to adopt his terminology :-) gist.github.com/yoavg/9142e5...

What makes multi-agent LLM systems multi-agent?

What makes multi-agent LLM systems multi-agent? GitHub Gist: instantly share code, notes, and snippets.

gist.github.com

If you're in WA and think imposing new taxes on things we want more of (e.g., bikes, transit) is a bad idea, consider contacting your reps using this simple form! <3

Transportation Choices Coalition@transpochoices.bsky.social · last yr.

The Senate transportation budgets contain things we love ($450 million for safety on main street highways!) and others we strongly oppose, like new taxes on buses and e-bikes. Join us in asking legislators to say "no" to these taxes. actionnetwork.org/letters/say-...

Should you delete softmax from your attention layers? check out Songling Yang's (sustcsonglin.github.io) tutorial, moderated by @srushnlp.bsky.social, for a beginner-friendly tutorial of the why/how/beauty of linear attention :-) www.youtube.com/watch?v=d0HJ...

I've spent the last two years trying to understand how LLMs might improve middle-school math education. I just published an article in the Journal of Educational Data Mining describing some of that work: "Designing Safe and Relevant Generative Chats for Math Learning in Intelligent Tutoring Systems"

Very good (technical) explainer answering "How has DeepSeek improved the Transformer architecture?". Aimed at readers already familiar with Transformers. epoch.ai/gradient-upd...

How has DeepSeek improved the Transformer architecture?

This Gradient Updates issue goes over the major changes that went into DeepSeek’s most recent model.

epoch.ai

I've posted the practice run of my LSA keynote. My core claim is that LLMs can be useful tools for doing close linguistic analysis. I illustrate with a detailed case study, drawing on corpus evidence, targeted syntactic evaluations, and causal intervention-based analyses: youtu.be/DBorepHuKDM

Finding linguistic structure in large language models

YouTube video by Chris Potts

youtu.be

Here's my end-of-year review of things we learned out about LLMs in 2024 - we learned a LOT of things simonwillison.net/2024/Dec/31/... Table of contents:

It's ready! 💫 A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them. maria-antoniak.github.io/2024/12/30/o... Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.

Maria Antoniak@mariaa.bsky.social · 2y ago

So far the blog post draft is winning the distraction battle. Prepare for a very long and opinionated update about all the new tools and apps I’ve been using for work.

Some of my thoughts on OpenAI's o3 and the ARC-AGI benchmark aiguide.substack.com/p/did-openai...

Did OpenAI Just Solve Abstract Reasoning?

OpenAI’s o3 model aces the "Abstraction and Reasoning Corpus" — but what does it mean?

aiguide.substack.com

Sample and verify go brr

Mark Riedl@markriedl.bsky.social · 2y ago

Six months ago someone put a for-loop around GPT-4o and got 50% on the ARC-AGI test set and 72% on a held-out training set redwoodresearch.substack.com/p/getting-50... Just sample 8000 times with beam search. o3 is probably a more principled search technique...

Check out our new encoder model, ModernBERT! 🤖 Super grateful to have been part of such an awesome team effort and very excited about the gains for retrieval/RAG! 🚀

Jeremy Howard @howard.fm · 2y ago

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

I'm not an """ AGI """ person or anything, but, I do think process reward model RL/scaling inference compute is quite promising for problems with easily verified solutions like (some) math/coding/ARC problems.

Announcement #1: our call for papers is up! 🎉 colmweb.org/cfp.html And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social

Meanwhile in my neighborhood in Seattle we've been fighting 5 years for (1) bus lane and 30 years for a (1) mile bike path

excited to come to #neurips2024 workshops this weekend --- I'll be around sat/sun to say hi to folks :-)

🚨 I’m on the academic job market! j-min.io I work on ✨Multimodal AI✨, advancing reasoning in understanding & generation by: 1⃣ Making it scalable 2⃣ Making it faithful 3⃣ Evaluating + refining it Completing my PhD at UNC (w/ @mohitbansal.bsky.social). Happy to connect (will be at #NeurIPS2024)! 👇🧵

“They said it could not be done”. We’re releasing Pleias 1.0, the first suite of models trained on open data (either permissibly licensed or uncopyrighted): Pleias-3b, Pleias-1b and Pleias-350m, all based on the two trillion tokens set from Common Corpus.

Blue skies 🦋 , hot (?) takes 🔥 Constrained output for LLMs, e.g., outlines library for vllm which forces models to output json/pydantic schemas, is cool! But, because output tokens cost much more latency than input tokens, if speed matters: bespoke, low-token output formats are often better.

Information retrieval systems usually operate as a model "cascade" -- fast vector search over billions of documents followed by a more expressive LLM "re-ranking" the resulting top-K. But beware 👻 ! Despite expressivity, top-K re-rankers generalize poorly as K increases. arxiv.org/pdf/2411.11767

OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social

LLMs generate novel word sequences not contained in their pretraining data. However, compared to humans, models generate significantly fewer novel n-grams. RLHF = 30% *more* copying than base! Awesome work from the awesome Ximing Lu (gloriaximinglu.github.io) et al. 🤩 arxiv.org/pdf/2410.04265

I'm recruiting 1-2 PhD students to work with me at the University of Colorado Boulder! Looking for creative students with interests in #NLP and #CulturalAnalytics. Boulder is a lovely college town 30 minutes from Denver and 1 hour from Rocky Mountain National Park 😎 Apply by December 15th!

I'm recruiting a PhD student to join my group in 2025-2026. If you like the mountains and interdisciplinary research that blends data and culture, this could be a good fit! UW iSchool PhD apps due Dec 2nd: ischool.uw.edu/programs/phd... More info about my group: melaniewalsh.org/mentorship

📣 I am recruiting 1-2 PhD students for Fall 2025 at the University of Maryland College of Information. Consider applying if you're interested in language, society/politics, and computers! Deadline Dec 3: ischool.umd.edu/academics/ph... And pls share with anyone who may be interested!

Doctor of Philosophy in Information Studies (PhD) - College of Information (INFO)

This doctoral program prepares students to address the hardest social and technical problems of today and tomorrow.

ischool.umd.edu

Hello Bluesky!! 🦋 Exciting to see many friends here! I'll be cross posting my nlp/ml/transit vibes here 💙