Tessera
Folgen

Aryo Pradipta Gema

@aryopg.bsky.social

PhD student @BioMedAI_CDT @EdinburghNLP @EdiClinicalNLP | ex: AI Safety Fellow @Anthropic | Opinions are my own. Personal page: https://aryopg.com

9 Posts973 Follower679 Folgt
Emile van Krieken@emilevankrieken.com · last yr.

We propose Neurosymbolic Diffusion Models! We find diffusion is especially compelling for neurosymbolic approaches, combining powerful multimodal understanding with symbolic reasoning 🚀 Read more 👇

Aryo Pradipta Gema@aryopg.bsky.social · last yr.

MMLU-Redux just touched down at #NAACL2025! 🎉 Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅 If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋

MMLU-Redux Poster at NAACL 2025
Benjamin Minixhofer@bminixhofer.bsky.social · 2y ago

We created Approximate Likelihood Matching, a principled (and very effective) method for *cross-tokenizer distillation*! With ALM, you can create ensembles of models from different families, convert existing subword-level models to byte-level and a bunch more🧵

Image illustrating that ALM can enable Ensembling, Transfer to Bytes, and general Cross-Tokenizer Distillation.
Pasquale Minervini@neuralnoise.com · 2y ago

Garbage in, garbage out -- nice gem for the Italian-speaking folks on this platform 😅 TLDR, in arxiv.org/abs/2406.04127 we found that MMLU contains TONS of errors, and looks like all these seamlessly propagated to this new "Global MMLU" dataset

Bild
Sara Hooker@sarahooker.bsky.social · 2y ago

Is MMLU Western-centric? 🤔 As part of a massive cross-institutional collaboration: 🗽Find MMLU is heavily overfit to western culture 🔍 Professional annotation of cultural sensitivity data 🌍 Release improved Global-MMLU 42 languages 📜 Paper: arxiv.org/pdf/2412.03304 📂 Data: hf.co/datasets/Coh...

Aryo Pradipta Gema@aryopg.bsky.social · 2y ago

Super Cool work from Cohere for AI! 🎉 However, this highlights a concern raised by our MMLU-Redux team (arxiv.org/abs/2406.04127): **error propagation to many languages**. Issues in MMLU (e.g., "rapid intervention to solve ebola") seem to persist in many languages. Let's solve the root cause first?

Sara Hooker@sarahooker.bsky.social · 2y ago

Is MMLU Western-centric? 🤔 As part of a massive cross-institutional collaboration: 🗽Find MMLU is heavily overfit to western culture 🔍 Professional annotation of cultural sensitivity data 🌍 Release improved Global-MMLU 42 languages 📜 Paper: arxiv.org/pdf/2412.03304 📂 Data: hf.co/datasets/Coh...

Pasquale Minervini@neuralnoise.com · 2y ago

For clarity -- great project, but most of the MMLU errors we found (and fixed) in our MMLU Redux paper (arxiv.org/abs/2406.04127) are also present in this dataset. We also provide a curated version of MMLU, so it's easy to fix 😊

Daniel Vila@dvilasuero.hf.co · 2y ago

Announcing Global-MMLU - an improved MMLU Open dataset with evaluation coverage across 42 languages. The result of months of work with the goal of advancing Multilingual LLM evaluation. Built together with the community and amazing collaborators at Cohere4AI, MILA, MIT, and many more.

Cyril Zakka, MD@cyrilzakka.bsky.social · 2y ago

Super excited to introduce Halo: A beginner's guide to DIY health tracking with wearables! 🤗✨ Using an $11 smart ring, I'll show you how to build your own private health monitoring app. From basic metrics to advanced features like: - Activity tracking - HR monitoring - Sleep analysis and more!

A picture showing Halo's features which include heart rate, sleep cycle and SPO2 monitoring, using on-device ML.
Giwon Hong@giwonhong.bsky.social · 2y ago

🤔How to achieve efficient ICL without storing a huge dataset in one prompt? 💡Mixtures of In-Context Learners (𝗠𝗼𝗜𝗖𝗟): we treat LLMs prompted with subsets of demonstrations as experts and learn a weighting function to optimise the distribution over the continuation (🧵1/n)

Bild