Matt Spike

@matspike.bsky.social

> Language + CogSci + Evolution + NLP/ML/AI COMPLEXITY, FUNCTION & FORM in - language, culture, cognition - evo dynamics - info & computation - explanation homeostatic property cluster at large LangEvo is Hard Reading List https://t.ly/gfGj

jesus FUCKING chrrrrriissttt THIS WAS CYBERNETICS ONCE OBSERVATION THAT INDEPENDENT PARAMETER ACHIEVES CHANNEL CAPACITY UPON REACHING 400 MILLION BUCKS ==> **ICML 2026 Outstanding Paper Honorable Mention** FFS

Jack Morris

@jxmnop

new paper from our work at Meta!

**GPT-style language models memorize 3.6 bits per param**

we compute capacity by measuring total bits memorized, using some theory from Shannon (1953)

shockingly, the memorization-datasize curves look like this:

Extracting structural/linguistic properties for large text datasets can be annoying. Existing tools either are not maintained, do not scale, or do not cover extensive sets of linguistic features. For this reason, I implemented 🧝elfen, a Python package for efficient linguistic feature extraction

Code examples for the Python package elfen:

# initializing extractor
extractor = elfen.Extractor(
data = df,
language = "en",
text_column = "text")
# extracting a single feature: ttr
extractor.extract("ttr")
# extracting a feature area/group: readability
extractor.extract_feature_group("readability")
# extracting all available features
extractor.extract_features()

MY GRAMMAR OPINIONS? FROM A MACHINE LEARNING ENGINEER? WITH A SIDELINE IN POLITICAL PUNDITRY? ON THEIR BLOG OR FCKIN PODCAST OR SOME SHIT LIKE THAT? INDEED THEY ARE SIR, INDEED THEY ARE

"Knight Rider" (1984) s02e17, Garthe Knight (Michael Knight's evil twin) breaks out of prison. Now in glorious HD.

I've found it kind of a pain to work with resources like VerbNet, FrameNet, PropBank (frame files), and WordNet using existing tools. Maybe you have too. Here's a little package that handles data management, loading, and cross-referencing via either a CLI or a python API.

GitHub - aaronstevenwhite/glazing: Unified data models and interfaces for syntactic and semantic frame ontologies.

Unified data models and interfaces for syntactic and semantic frame ontologies. - aaronstevenwhite/glazing

github.com