NEW: Luke Zettlemoyer (@lukezettlemoyer.bsky.social) of the University of Washington and Meta AI walks through different approaches to building multimodal foundation models. Watch the video: youtu.be/vTI4cziw84Q #NeuroAI2025 #AI #ML #LLMs #NeuroAI

Mixed-modal Language Modeling with Luke Zettlemoyer

YouTube video by Kempner Institute at Harvard University

youtu.be

What if LLMs knew when to stop? 🚧 HALT finetuning teaches LLMs to only generate content they’re confident is correct. 🔍 Insight: Post-training must be adjusted to the model’s capabilities. ⚖️ Tunable trade-off: Higher correctness 🔒 vs. More completeness 📝 🧵

Excited to continue my research adventure as a postdoc at @uwnlp.bsky.social and Meta! I’ve joined @lukezettlemoyer.bsky.social’s fantastic lab. Together, we plan to rethink how LLMs perceive data to unlock their capabilities to uncharted language and, further, beyond text!

Excited to announce the COLM 2025 keynote speakers: Shirley Ho, Nicholas Carlini, @lukezettlemoyer.bsky.social, and Tom Griffiths! See you in October in Montreal!

Chatbots don't *want* anything and don't *recognize* anything.

Chatbots, Like the Rest of Us, Just Want to Be Loved

A study reveals that large language models recognize when they are being studied and change their behavior to seem more likable.

wired.com

LLMs can't take responsibility for their mistakes. When a human journalist puts their name on AI-written text, they take on that responsibility. Increasingly I see inaccurate and badly written news stories authored by AI, many of which have actual humans listed as authors or editors.

#AI hallucinations are a problem; #UWAllen Ph.D. student @akariasai.bsky.social may have the answer. She was named a @techreviewjp.bsky.social Innovator Under 35 for her work to make #LLMs more transparent and useful—without making stuff up. #IU35 #AIforGood news.cs.washington.edu/2025/01/07/w...

‘Working to solve global problems’: Allen School Ph.D. student Akari Asai named one of MIT Technology Review’s Innovators Under 35 Japan - Allen School News

Despite their growing potential and increasing popularity, large language models (LLMs) often produce responses that are factually inaccurate or nonsensical, also known as hallucinations. Allen School...

news.cs.washington.edu

My Keynote Talk entitled “Dungeons and DQNs: The Serious Quest for Open Ended Role Playing Game Playing Agents” is now online. youtu.be/EiurL9eyUNc In which I might or might not have said “I’m working to take the ‘ick’ out of ‘agentic’”

Dungeons and DQNs: The Serious Quest for Open Ended Role Playing Game Playing Agents

YouTube video by AIIDE Conference

youtu.be

kicking off 2025 with our OLMo 2 tech report while payin homage to the sequelest of sequels 🫡 🚗 2 OLMo 2 Furious 🔥 is everythin we learned since OLMo 1, with deep dives into: 🚖 stable pretrain recipe 🚔 lr anneal 🤝 data curricula 🤝 soups 🚘 tulu post-train recipe 🚜 compute infra setup 👇🧵

📚 How good are language models at utilising contexts in RAG scenarios? We release 🧙🏽‍♀️DRUID to facilitate studies of context usage in real-world scenarios. arxiv.org/abs/2412.17031 w/ @saravera.bsky.social, H.Yu, @rnv.bsky.social, C.Lioma, M.Maistro, @apepa.bsky.social and @iaugenstein.bsky.social ⭐️

A Reality Check on Context Utilisation for Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) helps address the limitations of the parametric knowledge embedded within a language model (LM). However, investigations of how LMs utilise retrieved information o...

arxiv.org

🚀 Introducing the Byte Latent Transformer (BLT) – A LLM architecture that scales better than Llama 3 using patches instead of tokens 🤯 Paper 📄 dl.fbaipublicfiles.com/blt/BLT__Pat... Code 🛠️ github.com/facebookrese...

Remember Molmo? The full recipe is finally out! Training code, data, and everything you need to reproduce our models. Oh, and we have updated our tech report too! Links in thread 👇

Check out my new piece on AI terms of use restrictions w/ Mark Lemley ( @marklemley.bsky.social ). There's been a recent stir about terms of use restrictions on AI outputs & models. We dig into the legal analysis, questioning their enforceability. Link: papers.ssrn.com/sol3/papers....

I’m on the academic job market this year! I’m completing my @uwcse.bsky.social @uwnlp.bsky.social Ph.D. (2025), focusing on overcoming LLM limitations like hallucinations, by building new LMs. My Ph.D. work focuses on Retrieval-Augmented LMs to create more reliable AI systems 🧵

I'm on the academic job market! I develop autonomous systems for: programming, research-level question answering, finding sec vulnerabilities & other useful+challenging tasks. I do this by building frontier-pushing benchmarks and agents that do well on them. See you at NeurIPS!

I'm excited about scaling up robot learning! We’ve been scaling up data gen with RL in realistic sims generated from crowdsourced videos. Enables data collection far more cheaply than real world teleop. Importantly, data becomes *cheaper* with more environments and transfers to real robots! 🧵 (1/N)

We just updated the OLMo repo at github.com/allenai/OLMo! There are now several training configs that together reproduce the training runs that lead to the final OLMo 2 models. In particular, all the training data is available, tokenized and shuffled exactly as we trained on it!

GitHub - allenai/OLMo: Modeling, training, eval, and inference code for OLMo

Modeling, training, eval, and inference code for OLMo - allenai/OLMo

github.com

Hi everyone, I am excited to share our large-scale survey study with 800+ researchers, which reveals researchers’ usage and perceptions of LLMs as research tools, and how the usage and perceptions differ based on demographics. See results in comments! 🔗 Arxiv link: arxiv.org/abs/2411.05025

LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions

The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipe...

arxiv.org

How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️

We should make sure that only really big companies can afford to pay really big copyright holders to access the data needed to do stuff with AI, and keep everyone else out. Wouldn’t that be just super?

I am seeking multiple PhD students passionate about Generative Intelligence and its applications in empowering AI agents to interact with the physical world to join us at UPenn CIS for the 2024-2025 academic cycle. You can find more information at www.cis.upenn.edu/graduate/pro...

OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social

Excited to share OLMo 2! 🐟 7B and 13B weights, trained up to 4-5T tokens, fully open data, code, etc 🐠 better architecture and recipe for training stability 🐡 staged training, with new data mix Dolmino🍕 added during annealing 🦈 state-of-the-art OLMo 2 Instruct models #nlp #mlsky links below👇

very interesting work and it reminds me a bit of this paper. Tokenizers and ROPE must die. after samplers, i am on to those next ... arxiv.org/abs/2407.036...

Improving Self Consistency in LLMs through Probabilistic Tokenization

Prior research has demonstrated noticeable performance gains through the use of probabilistic tokenizations, an approach that involves employing multiple tokenizations of the same input string during ...

arxiv.org

Thomas Wolf@thomwolf.bsky.social · 2y ago

It's Sunday morning so taking a minute for a nerdy thread (on math, tokenizers and LLMs) of the work of our intern Garreth By adding a few lines of code to the base Llama 3 tokenizer, he got a free boost in arithmetic performance 😮 [thread]