FBK - NLP Research Group
@fbk-nlp.bsky.social
#NLP Research Unit at @ Fondazione Bruno Kessler Site: https://nlplab.fbk.eu #NLProc
🚀 Big news from #CLiCit2025! Our PhD student Davide Testa presented MAIA 🎞️🧠🇮🇹 — the first Italian benchmark to test Vision-Language Models on multimodal reasoning & robustness. 📄 You can check the Paper in the CLiC-it pre-processing: clic2025.unica.it/wp-content/u... #AI #NLProc #VLMs
🎉Congrats to our PhD student Sofia Brenna for presenting her poster at #SigDIAL2025 in Avignon 🇫🇷 about "Investigating Proactivity in Task-Oriented Dialogues", recently published in Dialogue & Discours journal. Paper here: journals.uic.edu/ojs/index.ph... #NLProc
🎉 Thrilled to share that our paper “All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark” has been accepted at the EMNLP 2025 conference! 🔥 📄 Preprint here: arxiv.org/abs/2502.16989 See you in Suzhou next November!!! 🇨🇳🚀 #EMNLP2025 #NLP #Multimodality
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark
We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other...
arxiv.org
🚀 Can Vision-Language Models plan effectively? We introduce ViPlan, a benchmark comparing: 🔹 VLM-as-planner 🔹 VLM-as-grounder 🏠 Home robotics 🧱 Visual Blocksworld Spoiler alert:❌ Visual-Reasoning 📍 Pre-print here: arxiv.org/abs/2505.13180 #VLM #AI #NLProc
🚀 Exciting News! The Evalita-LLM Leaderboard is now live on Hugging Face! Explore the performance of over 40 Large Language Models on native Italian tasks. Dive in here: huggingface.co/spaces/evali... @FondazioneBrunoKessler @igenius @diunito.bsky.social #NLProc #LLM #AI #Italian #Benchmarking
Evalita Llm Leaderboard - a Hugging Face Space by evalitahf
Duplicate this leaderboard to initialize your own!
huggingface.co
Don't miss out ‼️ Join us at Wired Health 2025 #WH25💭 Our Head Unit, Bernardo Magnini will take the stage to discuss "Artificial Intelligence and Clinical Data: The Future of Emergency Medicine." 🔥 Full program here: lnkd.in/eKi44biT @wired.com #AI #healthcare #Innovation
Welcome to MAIA! 🚀 Our new benchmark for evaluating multimodal reasoning in Vision-LMs on videos fully in Italian! 🇮🇹 MAIA tests understanding & generation with fine-grained reasoning categories and a brand-new evaluation metric! 🔎🔥Discover MAIA here: arxiv.org/abs/2502.16989 #NLProc #evaluation #AI
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark
We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other...
arxiv.org
🚀 **Exciting News!** 🎉 Evalita-LLM is here! 🇮🇹 A new benchmark for evaluating LLMs—offering native Italian tasks, generative challenges, and fair multi-prompt evaluations. Now also available in lm-evaluation harness by @eleutherai.bsky.social ! ArXiv: arxiv.org/abs/2502.02289 #NLProc #LLM #Evaluation
Evalita-LLM: Benchmarking Large Language Models on Italian
We describe Evalita-LLM, a new benchmark designed to evaluate Large Language Models (LLMs) on Italian tasks. The distinguishing and innovative features of Evalita-LLM are the following: (i) all tasks ...
arxiv.org
Our group leader took the stage at the FBK plenary session to showcase our research interests, ongoing projects, challenges and future plans. An exciting moment to share our vision and push the boundaries of NLP even further! Here’s a glimpse of the event! 📸✨ #NLProc #AI #Research #FBK #Innovation
A big congratulations to Sofia Lugli, student of our Carlo Strapparava, for receiving a special mention for her thesis at CLiC-it conference 2 weeks ago! 🎉👏 We are incredibly proud of her achievement. Can’t wait to see what she accomplishes next. Well done Sofia!🌟 #NLP #AI @ailc-nlp.bsky.social
🚀 #CALAMITA has officially kicked off - and we’re on board! Fondazione Bruno Kessler proudly participated in this 1st #evaluation #campaign for #LLMs with our Andrea Zaninello together with @fbk-mt.bsky.social group!🤖 Here some pictures of our calamitici!🧲 See u next year 4 other challenges! 🔥🔍
Exciting news! 🎉 If you’re curious about the opening #tutorial at CLiC-it 2024 conference made by our group on processing #data for #training and #evaluating #LLMs , here’s your chance! 📊 Explore the slides and get inspired!!! 🚀🔍✨ docs.google.com/presentation... @ailc-nlp.bsky.social
Data-LLM-Tutorial
You Are what You Eat Processing Data for Training and Evaluating LLMs Giovanni Bonetta and Bernardo Magnini Fondazione Bruno Kessler, Trento, Italy {gbonetta|magnini}@fbk.eu Tutorial at CLiC-it 2024, ...
docs.google.com
#NLP Research group in action! Our group leader Bernardo Magnini, alongside @tizaino.bsky.social, Sofia Brenna, and Giovanni Bonetta, presenting their #paper "Are you a Good Assistant? Assessing LLM Trustability in Task-oriented Dialogues" at CLiC-it '24 #Pisa 🔍✨ @ailc-nlp.bsky.social #NLProc #AI
What a conference! 🎉 CLiC-it in #Pisa was truly special! Returning to where it all began a decade ago! ❤️ Our group was honored to participate, share ideas and connect with such an inspiring community. Here are some highlights from the conference!!!📸✨ #CLiCit2024 #NLP #AI @ailc-nlp.bsky.social
🎄✨ Happy Holidays from our Research Center! ✨🎄 Our director @ferruccioresta shares a warm holiday message with all @FBK_research: gratitude for a year of outstanding achievements and best wishes for a joyful festive season and a 2025 filled with groundbreaking discoveries!
