Sam Harsimony

@harsimony.bsky.social

I write about opportunities in science, space, and policy here: https://splittinginfinity.substack.com/

Google DeepMind's DiffusionGemma Technical Report They feel text diffusion models open up a radically different part of the latency–quality Pareto frontier and hope the report makes it easier for researchers and engineers to understand the model, build on it, and create things we haven’t thought of

Bild

Qwen just announced Qwen3.8-27B along with Qwen3.8-Max! 🔥 Qwen3.8-27B will run locally on 17GB RAM/VRAM setups and is expected to be the best performing model for its size.

Bild

The fact that the price is pretty reasonable and that this model is ready soon after Sol release suggests that it's not much bigger, but rather, has lots of RL to act as a good agent orchestrator and team player. As we leave the CoT s-curve, agent orchestration becomes the next one?

mr. TIM@timkellogg.me · 4d ago

this is their first time referring to it officially, and they called it “Astra”, which implies that it’s a fully new series like o1 & o3 when addressing cost, they quantified in terms of Sol, implying that it’s the same size, but maybe not a general purpose model bsky.app/profile/timk...

DwarfStar branchk "ds4f-mxfp4" now can run the lossless MXFP4 DeepSeek v4 Flash GGUF I published on my Hugging Face account. It rocks even with SSD streaming in 128GB systems at > 20 t/s in case you want to try the *actual* DS4F weights released without any quantization.

The pace of progress on models from so many organizations at once is genuinely incredible. Building LLMs isn't driven by rare secrets, but consistent effort, mass capital, and effective organization design. It is great that know-how of such a powerful technology is diffused.

This is a robot failing to grasp a ball. Almost every robot lab produces clips like this daily… and almost all of them get thrown away. This is the most abundant but underused resource in robot learning. We’re collecting all of it now as “OopsieData”, please join us at oopsie-data.com! (1/13)

I have this issue too. Resolvable with follow up questions but annoying. I think part of this is how the model achieves higher performance. The CoT needs to navigate to sparse parts of latent space to have more wiggle room for reasoning. As side effect, weird terminology in the final output.

ver 🗿@ver.ooo · 6d ago

I am getting SO FRUSTRATED with the way Opus 5 speaks. It starts throwing around vagaries like "the T4 fork" and "the §8 spike" like these terms should make any sense to me. continually making up its own jargon in the course of a conversation

A few things to take away from Aschenbrenner's hedge fund collapsing: 1. Predicting the future is very different from making money off of it. 2. You often want to place bets on worlds where you're wrong rather than those where you're right.

It’s baffling to me that quantizing embedding vectors is not a common task: it works wonderfully well, far better than Matryoshka in fact. I ran remex and 1-bit remax on a 5K SciDoc corpus: 32x storage (and memory) savings, with marginal loss in accuracy

Two side-by-side line charts comparing ways to shrink the search vectors from
LFM2.5-Embedding-350M, tested on BEIR SciFact. Both x-axes are bytes per vector
on a log scale: 8 bytes at left (tiny index) to 4096 bytes at right (uncompressed).

LEFT — "which codec wins at which byte budget". Y-axis is nDCG@10, retrieval
quality. A grey dashed line at 0.7122 marks uncompressed float32. Purple
triangles (remex Lloyd-Max scalar quant, 1 to 8 bit) sit essentially on that
dashed line all the way down to 128 bytes: 32x smaller, almost no quality lost.
Red circles (remax centered SimHash) track just underneath. A green triangle
(int8, 1024 bytes) lands exactly on the line. Grey diamonds are float32 with
dimensions truncated instead of quantized, and they form a separate, much lower
curve — at 1024 bytes truncated float32 scores 0.657, while 1-bit over all
dimensions at 128 bytes scores 0.702. Eight times less storage and better
quality. The shape is the finding: the coloured curves stay flat along the
baseline from 4096 bytes down to 128, then fall off a cliff.

RIGHT — "and which one preserves the fp32 ranking". Same x-axis; y-axis is
agree@10, the overlap with the uncompressed top 10. Nothing is flat here. Every
curve declines steadily as the index shrinks; at 128 bytes remex agrees with
only about 80% of uncompressed results. Both panels are true at once:
compression holds quality while quietly reshuffling near-equivalent neighbours.
Fine for RAG, less fine if you promised reproducible ordering.

An orange X marks an ablation — 1-bit without centering — which landed
marginally above the centered version, contradicting the prediction.

Harness, raw numbers and the geometry diagnostic that called the winner before
any search ran: github.com/oaustegard/remax/pull/53

Experiment and write-up by Oskar Austegard with Claude.
mr. TIM@timkellogg.me · last wk.

Liquid LFM2.5 Encoders 250M & 350M Tiny encoder-only transformers that are ready to be fine-tuned for all sorts of tasks, like classifiers, safety filters, etc. This one is made to run fast on CPU! It’s basically a replacement for ModernBERT that runs 3.7x faster www.liquid.ai/blog/lfm2-5-...