Tessera
Folgen

@sebnagel.bsky.social

0 Posts69 Follower32 Folgt
Common Crawl Foundation@commoncrawl.bsky.social · 3mo ago

RSVP and join speakers @very-laurie.bsky.social and @pjox.bsky.social from the Common Crawl Foundation and Kostis Saitas Zarkias and Robert Pugh from Mozilla Data Collective for a truly hands-on session. Thursday, June 4th 6 PM CEST | 12 PM ET | 9 AM PDT Register via Zoom: zoom.us/meeting/regi...

Welcome! You are invited to join a meeting: Text Language Identification (LID) with CommonCrawl and Mozilla Data Collective. After registering, you will receive a confirmation email about joining the ...

Welcome! You are invited to join a meeting: Text Language Identification (LID) with CommonCrawl and Mozilla Data Collective. After registering, you will receive a confirmation email about joining the ...

zoom.us

Common Crawl Foundation@commoncrawl.bsky.social · 5mo ago

Common Crawl is switching to reporting dataset sizes in nibbles. As an organisation dedicated to data preservation, we feel it would be remiss to allow this underrepresented unit to fall out of use. Our latest crawl now exceeds 689 tebibbles. commoncrawl.org/blog/announc...

Common Crawl - Blog - Announcing a Change to Common Crawl Dataset Size Reporting

Common Crawl is switching to reporting dataset sizes in nibbles. As an organisation dedicated to data preservation, we feel it would be remiss to allow this underrepresented unit to fall out of use. O...

commoncrawl.org

@eleutherai.bsky.social · 6mo ago

Announcing our latest paper: CommonLID In collaboration with @commoncrawl.bsky.social @mlcommons.org @jhu.edu we built a LID benchmark on actual Common Crawl text covering 109 languages. Existing evaluations overestimate how well LangID works on web data. arxiv.org/abs/2601.18026

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data

Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heterogeneous web data of...

arxiv.org

Common Crawl Foundation@commoncrawl.bsky.social · 11mo ago

Common Crawl’s Web Languages initiative has had many contributions since its introduction. We’re calling for native speakers of certain languages to review language contributions, to ensure that links we’re adding to our seed crawl are of good quality. commoncrawl.org/blog/web-lan...

Common Crawl - Blog - Web Languages Needing Review by Native Speakers

Common Crawl’s Web Languages initiative has had many contributions since its introduction. We’re calling for native speakers of certain languages to review language contributions, to ensure that links...

commoncrawl.org

Common Crawl Foundation@commoncrawl.bsky.social · last yr.

"MOIC will also partner with Common Crawl, one of the largest free and open repositories of web crawled data. MOIC will fund work at Common Crawl, leveraging native speakers to annotate and seed European language data in the publicly available Common Crawl data set."

Unlocking data to advance European commerce and culture - Microsoft On the Issues

Microsoft launches 2 initiatives to open Europe’s languages and culture, building on AI, cloud, and digital sovereignty commitments.

blogs.microsoft.com

Common Crawl Foundation@commoncrawl.bsky.social · last yr.

The Common Crawl Foundation, MLCommons, EleutherAI, and John Hopkins' Center for Language and Speech Processing have the pleasure of inviting you to register for the 1st shared task on Language Identification for web data. commoncrawl.org/blog/wmdqs-s...

Common Crawl - Blog - WMDQS Shared Task on Language Identification

The Common Crawl Foundation, MLCommons, EleutherAI, and John Hopkins' Center for Language and Speech Processing have the pleasure of inviting you to register for the 1st shared task on Language Identi...

commoncrawl.org