Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.

🌱Very proud of our team's latest release 😊 meet Tiny Aya, a massively multilingual model with 3.35B parameters. Tech report here: github.com/Cohere-Labs/...

Cohere Labs@cohereforai.bsky.social · 6mo ago

Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are. Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.

What if the way we verify synthetic code is limiting model performance? In our latest work we uncover the Verification Ceiling Problem: strict “all tests must pass” rules throw away useful data, while weak tests let errors through.

We’re not your average lab. We’re a hybrid research environment dedicated to revolutionizing the ML space. And we’re hiring a Senior Research Scientist to co-create with us. If you believe in research as a shared, global effort — this is your chance.

I'm excited to share that I'll be stepping into the role of Head of @cohereforai.bsky.social. It's an honor and a responsibility to lead such an extraordinary group of researchers pushing the boundaries of AI research.

While effective for chess♟️, Elo ratings struggle with LLM evaluation due to volatility and transitivity issues. New post in collaboration with AI Singapore explores why Elo falls short for AI leaderboards and how we can do better.

Breaking into AI research is harder than ever, and early-career researchers face fewer chances to get started. Entry points matter. We started the Scholars Program 3 years ago to give new researchers a real shot — excited to open applications for year 4✨

Cohere Labs@cohereforai.bsky.social · 12mo ago

Applications are now open for the next cohort of the Cohere Labs Scholars Program! 🌟 This is your chance to collaborate with some of the brightest minds in AI & chart new courses in ML research. Let's change the spaces breakthroughs happen. Apply by Aug 29.

🌍 Language shapes how we think and connect—but most AI models still struggle beyond English. @microsoft.com's July seminar discussed how we can bridge the gap and build #AIforEveryone with @mziizm.bsky.social of @cohere.com. 📽️ www.microsoft.com/en-us/resear...

Building Better Language Models Through Global Understanding - Microsoft Research

Modern language models have achieved remarkable capabilities in English, but human knowledge and experience span thousands of languages, each encoding unique perspectives and problem-solving approache...

microsoft.com

ACL day 2 ✨

🖼️ Most text-to-image models only really work in English. This limits who can use them and whose imagination they reflect. We asked: can we build a small, efficient model that understands prompts in multiple languages natively?

Everyone talks about GEB (I agree, it's a gem) but Hofstadter's Analogy book is criminally underrated. If you're working on learning intelligence through language understanding, it’s a must-read.

🍋 Squeezing the most of few samples - check out our LLMonade recipe for few-sample test-time scaling in multitask environments. Turns out that standard methods miss out on gains on non-English languages. We propose more robust alternatives. Very proud of this work that our scholar Ammar led! 🚀

Cohere Labs@cohereforai.bsky.social · last yr.

Can we improve the performance of LLMs during inference without the need for extensive sampling OR special reward models? 🤔 Our latest work introduces a new inference time scaling recipe that is sample-efficient, multilingual, and suitable for multi-task requirements. 🍋

London has me under its spell. every. single. visit.

🚨LLM safety research needs to be at least as multilingual as our models. What's the current stage and how to progress from here? This work led by @yongzx.bsky.social has answers! 👇

Cohere Labs@cohereforai.bsky.social · last yr.

It’s been two years since cross-lingual jailbreaks were first discovered. How far has the multilingual LLM safety research field advanced? 🤔 📏 Our comprehensive survey reveals that there is still a long way to go.

Over 7000 languages are spoken worldwide 🌐, but AI safety efforts focus on only a fraction of them. Our latest paper draws on our multi-year efforts with the wider research community to explore why this matters and how we can bridge the AI language gap.

📢 The Copenhagen NLP Symposium on June 20th! - Invited talks by @loubnabnl.hf.co (HF) @mziizm.bsky.social (Cohere) @najoung.bsky.social (BU) @kylelo.bsky.social (AI2) Yohei Oseki (UTokyo) - Exciting posters by other participants Register to attend and/or present your poster at cphnlp.github.io /1

It is critical for scientific integrity that we trust our measure of progress. The @lmarena.bsky.social has become the go-to evaluation for AI progress. Our release today demonstrates the difficulty in maintaining fair evaluations on the Arena, despite best intentions.

Goodhart's law rules everything around me.

Sung Kim@sungkim.bsky.social · last yr.

You know how Meta was allegedly manipulating results on Chatbot Arena for LLama 4? Well, Cohere and its cohorts wrote a paper about it. Paper: The Leaderboard Illusion - arxiv.org/abs/2504.20879

1/ Science is only as strong as the benchmarks it relies on. So how fair—and scientifically rigorous—is today’s most widely used evaluation benchmark? We took a deep dive into Chatbot Arena to find out. 🧵

Not in Singapore for #ICLR2025 but our lab’s work is! In particular, I am very proud of these collaborations: ✨INCLUDE (spotlight) — models fail to grasp regional nuances across languages 💎To Code or Not to Code (poster) — code is key for generalizing beyond coding tasks

🚨 Excited to share our latest paper! Multilingual LLMs are getting really good. But the way we evaluate them? Not the best sometimes. 🌟 We show how decades of lessons from Machine Translation can help us fix it

Julia Kreutzer@juliakreutzer.bsky.social · last yr.

📖New preprint with Eleftheria Briakou @swetaagrawal.bsky.social @mziizm.bsky.social @kocmitom.bsky.social! arxiv.org/abs/2504.11829 🌍It reflects experiences from my personal research journey: coming from MT into multilingual LLM research I missed reliable evaluations and evaluation research…

Very excited to release Kaleidoscope—a multilingual, multimodal evaluation set for VLMs, built as part of our open-science initiative! 🌍 18 languages (high-, mid-, low-) 📚 21k questions (55% require image understanding) 🧪 STEM, social science, reasoning, and practical skills

Big news from WMT! 🎉 We are expanding beyond MT and launching a new multilingual instruction shared task. Our goal is to foster truly multilingual LLM evaluation and best practices in automatic and human evaluation. Join us and build the winning multilingual system! www2.statmt.org/wmt25/multil...

☀️ Summer internship at Cohere! Are you excited about multilingual evaluation, human judgment, or meta-eval? Come help us explore how a rigorous eval really looks like while questioning the status quo in LLM evaluation. I’m looking for an intern (EU timezone preferred), are you interested? Ping me!

Command🅰️ technical report is out. Information-dense. Detailed. Pretty. Simply A+! 💎: cohere.com/research/pap...

Command A: An Enterprise-Ready Family of Large Language Models

In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command

cohere.com

Max Bartolo@maxbartolo.bsky.social · last yr.

I'm excited to share the tech report for our @cohere.com @cohereforai.bsky.social Command A and Command R7B models. We highlight our novel approach to model training including self-refinement algorithms and model merging techniques at scale. Read more below! ⬇️

Good morning Paris

✨👓 Aya Vision is here 👓✨ A multilingual, multimodal model designed to understand across languages and modalities (text, images, etc) to bridge the language gap and empower global users!

Excited to share insights from our new paper on evaluating LLMs in multi-session coding interactions! 📚📚📚 We introduce MEMORYCODE, a novel dataset to assess how well LLMs track & execute coding instructions across multiple sessions, mimicking real-world collaboration.

INCLUDE is going to ICLR! Huge congrats to all my co-authors!

Marzieh Fadaee@mziizm.bsky.social · 2y ago

Good performance shouldn’t mean 'just in English' anymore 🪩 We provide a robust way to assess models with a new benchmark that captures in-language nuances and cultural contexts.

This #Neurips2024 was the perfect way to end this year. So long Vancouver!