Limits of vector search a new GDM paper shows that embeddings can’t represent combinations of concepts well e.g. Dave likes blue trucks AND Ford trucks even k=2 sub-predicates make SOTA embedding models fall apart www.alphaxiv.org/pdf/2508.21038

On the Theoretical Limitations of Embedding-Based Retrieval | alphaXiv

View recent discussion. Abstract: Vector embeddings have been tasked with an ever-increasing set of retrieval tasks over the years, with a nascent rise in using them for reasoning, instruction-followi...

alphaxiv.org

Longcat-Flash-Chat (560B) uh, holy shit this one is intriguing. bare minimum they compare themselves to all the (actual) top models and do okay but inside.. damn this one has some cool ideas huggingface.co/meituan-long...

In 2012 when I had to clean data it seemed natural to look for rules I could use to clean it. Now it seems natural to model the noise, find new clean data it can destroy, and then train a model to reverse the process. Machine learning makes you a sicko.

Three things to note about this: 1) AI has obvious utility to many, this is a tremendous amount of use already 2) There is room for multiple frontier model providers, at least for now 3) Any losses from subsidizing cost of AI use (and it is not clear this is happening) are now relatively small

"The Serial Scaling Hypothesis" (arxiv.org/abs/2507.125..., Liu et al) is interesting I think, not as new as it completely looks (autoregressive models are used serially, models have depth,..) but feels like a good formalization and intuition as of where current GPT based LLMs will typically fail

1/ Can open-data models beat DINOv2? Today we release Franca, a fully open-sourced vision foundation model. Franca with ViT-G backbone matches (and often beats) proprietary models like SigLIPv2, CLIP, DINOv2 on various benchmarks setting a new standard for open-source research.

In the coming age of agents, I think vibe coding will die out, same lasting power as prompt engineering. For things LLMs excell at, you might as well stick to higher level directives and let it own the work, Claude Code is a good example. 1/2

this is probably why Meta was able to poach OpenAI ppl aside from the absolute piles of cash, Sama is very SV-minded and can’t imagine building apart from a product a lot of accelerationists see things differently, more broadly, and ids dissatisfying to be forced into a product box

mr. TIM@timkellogg.me · last yr.

explaining why they open sourced — to ensure that it’s broadly useful OpenAI self-admits that they optimize their models for ChatGPT, o3 was made for DeepResearch Moonshot was dissatisfied with that

Still not a lot of ML talk on bsky (at least in my feed), hence paper Sunday: my two most interesting recent reads - H Nets arxiv.org/abs/2507.07955 - Energy Based Transformers arxiv.org/abs/2507.02092

Dynamic Chunking for End-to-End Hierarchical Sequence Modeling

Despite incredible progress in language models (LMs) in recent years, largely resulting from moving away from specialized models designed for specific tasks to general models based on powerful archite...

arxiv.org

Little bit of personal news, shared in other circles already: I'm moving to Mistral in August, after three years at Photoroom. I'm really proud of what we built in the ML team with relatively limited means, lasting SOTA on the existing foundations (saliency segmentation) while growing a lot on genAI

𝗗𝗲𝗽𝘁𝗵 𝗔𝗻𝘆𝘁𝗵𝗶𝗻𝗴 𝗮𝘁 𝗔𝗻𝘆 𝗖𝗼𝗻𝗱𝗶𝘁𝗶𝗼𝗻 Boyuan Sun, Modi Jin, Bowen Yin, Qibin Hou arxiv.org/abs/2507.01634 Trending on www.scholar-inbox.com

kyutai open sources its TTS model as well as Unmute, a framework for building audio AI apps notable: - high accuracy - actually streaming (can use streaming text input) - serves 32 simultaneous users on a single GPU - voice cloning - supports all 24 official EU languages kyutai.org/next/tts

Alex Nichol is one the rare many-hits researchers of the field, with on top of that a track record of practical models which affect the public/ship. That Meta wouldn't target him is pretty rich

Alex Nichol@unixpickle.bsky.social · last yr.

Kinda offended that meta didn't try to recruit me 😂

Automatically generate a fused megakernel in triton.. diving in, but if it works half as well as it reads it would already be quite something. Aligns with torch.compile of course github.com/mirage-proje...

GitHub - mirage-project/mirage: Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA

Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA - mirage-project/mirage

github.com

Sharing that Photoroom open sourced _Dataroom_, as promised some time ago. Accompanying blog post and mini thread github.com/photoroom/da... www.photoroom.com/inside-photo... 1/N

Photoroom Visual Ads Automation & GenerateBanners Acquisition

Photoroom launches Visual Ads Automation, a GenAI API turning product catalogs into branded ad creatives; GenerateBanners acquisition adds text automation.

photoroom.com

Still haven't tried Cursor, but I recently moved from Github Copilot to Continue with Codestral (free API), and it's absurd how much better Continue with Codestral is (vs. Copilot with expensive and slow models). Made me realize that there is zero moat in this field, at least for Copilot.

In the last 2 weeks: - Slack locked down its messages data. - X locked down its post data. - Anthropic cut off OpenAI's Windsurf. - Google will stop using Scale. The dream of unfettered MCP interconnects is a mirage. www.dbreunig.com/2025/06/16/d...

The Drawbridges Go Up

The AI era is speedrunning the Web 2.0 story. Open and accessible MCPs are not our future. Integrations will be tightly governed.

dbreunig.com

While framed as a critique of Apple’s recent paper, I found this article mostly interesting because it made me think about reasoning in general: mikecaulfield.substack.com/p/the-apple-...

The Apple "Reasoning Collapse" Paper Is Even Dumber Than You Think

We're this far into reasoners and neither hypesters nor skeptics really understand their significance. Also: Read Toulmin.

mikecaulfield.substack.com

Self adapting language models, still early but fascinating prospects. There's a dimensionality curse of course: since the dimensions the LLM can touch per generated token are very small as such, needs a massive lever / dimension reduction to be able to self improve. arxiv.org/pdf/2506.10943

Great write up of AMD new offerings, catching the nvidia train on the software side it seems. 3x speedup on MI300X since release, was required but still great to grab morethanmoore.substack.com/p/amds-ai-fu...

AMD's AI Future is Rack Scale 'Helios'

Key Announcements from AMD Advancing AI 2025

morethanmoore.substack.com

Got nerdsniped into printing this a little while ago.

datago now available with webdataset compatibility (streaming tarballs, so you get the data as it arrives). Just pip install datago and give it a whirl if you'd like ? Speed without the dataloader processes, and typical ViT/DiT pre-processing baked in. example code here github.com/Photoroom/da...

datago/python/benchmark_webdataset.py at main · Photoroom/datago

A Rust-based data loader which can be used from Python. Processing data per sample at GB/s speeds, covering various use cases eventually. - Photoroom/datago

github.com

Great link with bsky.app/profile/dbre...

Eugene Vinitsky 🍒@eugenevinitsky.bsky.social · last yr.

Some interesting work suggesting that recent shocking RL+LLM results are due to incorrect baselines and most of the gains are from better format following: safe-lip-9a8.notion.site/Incorrect-Ba...

SageAttention3 paper reads great, and looks like B200s just got a good value boost. QAT or PTQ-free use of FP4, I expected this to be much more complicated or come later to be honest. Only at the attention level and LLMs are most often MLP bottlenecked but stil arxiv.org/abs/2505.11594

SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training

The efficiency of attention is important due to its quadratic time complexity. We enhance the efficiency of attention through two key contributions: First, we leverage the new FP4 Tensor Cores in Blac...

arxiv.org

Big Marigold update! Last year, we showed how to turn Stable Diffusion 2 into a SOTA depth estimator with a few synthetic samples and 2–3 days on just 1 GPU. Today's release features: 🏎️ 1-step inference 🔢 New modalities 🫣 High resolution 🧨 Diffusers support 🕹️ New demos 🧶👇

Striking in retrospect how some leaders position at the time of the first atomic bomb (“other countries won’t get it”), Truman for instance, then space age, then now AI age, rhyme. Same as before, I think a bunch of places are bound to be SOTA AI, ideas and progress cannot be pinned to a wall

Apple readying in house server grade AI processors feels quite bizarre to me: Apple Intelligence has been underwhelming so far, so current status is probably far from end game. But lowering the code now into hardware limits future flexibility, bad timing ? www.tomshardware.com/pc-component...

Apple reportedly readies Baltra processors for AI servers

Apple's working with Broadcom on Baltra.

tomshardware.com

[1/2] We've released the code for LegoGPT. Our autoregressive model generates physically stable and buildable designs from text prompts by integrating physics laws and assembly constraints into LLM training and inference. Code: github.com/AvaLovelace1... Website: avalovelace1.github.io/LegoGPT/

Some men just want to watch the world burn.