Tessera
Folgen

Zhihang Xie

@zhihangxie.bsky.social

8 Posts21 Follower15 Folgt
Zhihang Xie@zhihangxie.bsky.social · 2mo ago

🚀 New paper: Speech-XL for long-form SpeechLLMs 📄 arxiv.org/abs/2602.05373 🧩 Uses Speech Summarization Tokens to compress local speech intervals into compact KV states efficiently. ✨ Improves long-form speech understanding while reducing memory and FLOPs on 10-minute audio.

Speech-XL: Towards Long-Form Speech Understanding in Large Speech Language Models

Despite the growing success of Large Speech Language Models (LSLMs) in processing short-term acoustic signals, their extension to long-form audio understanding is severely bottlenecked. This limitatio...

arxiv.org

MT Group at FBK@fbk-mt.bsky.social · 3mo ago

1️⃣ "Do What I Say: A Spoken Prompt Dataset for Instruction-Following" 👥 @maikezufle.bsky.social, @sarapapi.bsky.social, Fabian Retkowski, Szymon Mazurek, Marek Kasztelnik, Alexander Waibel, @luisabentivogli.bsky.social, @jan-niehues.bsky.social 🇪🇺 Meetween EU project 📄 arxiv.org/abs/2603.09881

Do What I Say: A Spoken Prompt Dataset for Instruction-Following

Speech Large Language Models (SLLMs) have rapidly expanded, supporting a wide range of tasks. These models are typically evaluated using text prompts, which may not reflect real-world scenarios where ...

arxiv.org

Zhihang Xie@zhihangxie.bsky.social · 4mo ago

🚀 New paper: Detecting Hallucinations in SpeechLLMs at Inference Time Using Attention Maps 📄 arxiv.org/abs/2604.19565 🧩 Lightweight inference-time detection for SpeechLLM hallucinations using audio-focused attention features. ✨ Attention classifiers outperform uncertainty baselines on ASR and S2TT.

Detecting Hallucinations in SpeechLLMs at Inference Time Using Attention Maps

Hallucinations in Speech Large Language Models (SpeechLLMs) pose significant risks, yet existing detection methods typically rely on gold-standard outputs that are costly or impractical to obtain. Mor...

arxiv.org

Zhihang Xie@zhihangxie.bsky.social · last yr.

🚀 AdvST: Adversarial training aligns speech and text distributions without parallel data! Combines adversarial learning + hidden-state swapping to fix length mismatch & boost low-resource speech translation. ieeexplore.ieee.org/document/108...

Adversarial Speech-Text Pre-Training for Speech Translation

Large-scale pre-training has been shown to benefit speech translation tasks. However, existing multimodal pre-training efforts rely on parallel corpora for semantic alignment, potentially limiting per...

ieeexplore.ieee.org

Zhihang Xie@zhihangxie.bsky.social · last yr.

🚀 Boost rare-phrase translation in speech! Uses **bilingual dictionaries** (e.g., "climate change"→"Klimawandel") to dynamically bias outputs. ✅ **+21%** recall in streaming ST ✅ **+85%** in multimodal LLMs 🔗: arxiv.org/abs/2506.09175

PHRASED: Phrase Dictionary Biasing for Speech Translation

Phrases are essential to understand the core concepts in conversations. However, due to their rare occurrence in training data, correct translation of phrases is challenging in speech translation task...

arxiv.org

Beatrice Savoldi@bsavoldi.bsky.social · last yr.

🔍 Stiamo studiando come l'AI viene usata in Italia e per farlo abbiamo costruito un sondaggio! 👉 bit.ly/sondaggio_ai... (è anonimo, richiede ~10 minuti, e se partecipi o lo fai girare ci aiuti un sacco🙏) Ci interessa anche raggiungere persone che non si occupano e non sono esperte di AI!

Qualtrics Survey | Qualtrics Experience Management

The most powerful, simple and trusted way to gather experience data. Start your journey to experience management and try a free account today.

bit.ly