Tessera
Folgen

Tomasz Limisiewicz

@tomlim.bsky.social

Postdoc at Meta and university of Washington in NLP. Before: PhD from Charles University (Prague 🏰). Interested in going into the inner workings of neural networks 🔍, multilinguality 🌍, tokenization 🔡 and fairer NLP ⚖️ (he/him)

32 Posts3561 Follower728 Folgt
Tomasz Limisiewicz@tomlim.bsky.social · 2mo ago

Happy to share that the unprocessed results and code for fitting scaling laws and plotting are now available at: github.com/facebookrese...

GitHub - facebookresearch/compute-optimal-tokenization: The repository contains raw data results and code for scaling laws fitting and visualization used in "Compute Optimal Tokenization" paper.

The repository contains raw data results and code for scaling laws fitting and visualization used in "Compute Optimal Tokenization" paper. - facebookresearch/compute-optimal-tokenization

github.com

Tomasz Limisiewicz@tomlim.bsky.social · 3mo ago

We present Compute Optimal Tokenization! 🔠 Common in LLM scaling works stick to one tokenizer, sweeping data/model size. But what happens when we control the tokenizer’s compression rate (bytes/token)? Here we sweep tokenizers, params, and data across compute budgets: [1/N]

Tomasz Limisiewicz@tomlim.bsky.social · 3mo ago

We present Compute Optimal Tokenization! 🔠 Common in LLM scaling works stick to one tokenizer, sweeping data/model size. But what happens when we control the tokenizer’s compression rate (bytes/token)? Here we sweep tokenizers, params, and data across compute budgets: [1/N]

Institute of Formal and Applied Linguistics@ufal.mff.cuni.cz · last yr.

Paper 👉Beyond Literal Token Overlap: Token Alignability for Multilinguality👈 by @kathaem.bsky.social, @tomlim.bsky.social, @jlibovicky.bsky.social and Alex Fraser will appear at #NAACL2025! arxiv.org/abs/2502.06468 Congratulations to all authors! 🥳

Beyond Literal Token Overlap: Token Alignability for Multilinguality

Previous work has considered token overlap, or even similarity of token distributions, as predictors for multilinguality and cross-lingual knowledge transfer in language models. However, these very li...

arxiv.org

Jindřich Libovický@jlibovicky.bsky.social · last yr.

Our paper 'Beyond Literal Token Overlap: Token Alignability for Multilinguality' will be at #NAACL2025! We show that token alignability is a stronger predictor of cross-lingual transfer than literal token overlap. Read it here: arxiv.org/abs/2502.06468

Kathy@kathaem.bsky.social · last yr.

Happy to say that our paper "Beyond Literal Token Overlap: Token Alignability for Multilinguality" will be presented at #NAACL2025! This is work with @tomlim.bsky.social, @jlibovicky.bsky.social, and Alex Fraser. arxiv.org/abs/2502.06468 #newpaper #NLP #NLProc

Beyond Literal Token Overlap: Token Alignability for Multilinguality

Previous work has considered token overlap, or even similarity of token distributions, as predictors for multilinguality and cross-lingual knowledge transfer in language models. However, these very li...

arxiv.org