I wrote something about building systems people actually want as an academic in ML. It's pretty much an open letter to 6-months-ago me. magnusross.github.io/posts/moms-m...

❗️ We just expanded our capacity of B200 SXM6 180GB servers – available in the DataCrunch Cloud Platform. The best thing is… You can deploy the Blackwell platform without approvals. Just sign in, select the instance type, and start your deployment: cloud.datacrunch.io?utm_source=b...

A new paper just dropped from Tri Dao(🐐)'s lab! arxiv.org/abs/2505.21487 Here is my hot take!

Hardware-Efficient Attention for Fast Decoding

LLM decoding is bottlenecked for large batches and long contexts by loading the key-value (KV) cache from high-bandwidth memory, which inflates per-token latency, while the sequential nature of decodi...

arxiv.org

🆕 Inference API for FLUX.1 Kontext [max] & [pro] are now available on DataCrunch! We are an infrastructure partner of Black Forest Labs for Kontext, a suite of generative flow matching models for text-to-image and image-to-image editing. Learn more: datacrunch.io/managed-endp...

🚨 Summer Inference by Symposium AI is happening next Wednesday, June 4, at 16:00-22:00. 🇫🇮 This event will bring together 250 AI engineers, researchers, and founders under one roof in Helsinki. 🔗 You can still grab one of the last remaining seats: lu.ma/x5hhj79x

Symposium AI - Summer Inference · Luma

Join 250 leading AI builders for an epic night in Helsinki! Symposium AI events bring together top AI talent, researchers, and engineers who are actively…

lu.ma

Algorithm hardware co-design was a big reason the whale 🐋(DeepSeek) made such a splash 💦 with its V3 and R1 releases.

"Cost-aware simulation-based inference" is accepted at AISTATS 2025. Check out our poster #205 on Sunday May 4th in Hall A-E if you are in Phuket. Finland's rising star @huangdaolang.bsky.social will be there to assist you :D arxiv.org/abs/2410.07930 @fxbriol.bsky.social @samikaski.bsky.social

Cost-aware simulation-based inference

Simulation-based inference (SBI) is the preferred framework for estimating parameters of intractable models in science and engineering. A significant challenge in this context is the large computation...

arxiv.org

I don’t mean to be a broken record but AI development could stop at the o3/Gemini 2.5 level and we would have a decade of major changes across entire professions & industries (medicine, law, education, coding…) as we figure out how to actually use it & adapt our systems. AI disruption is baked in.

1/ If you are at ICLR / AABI / AISTATS, check out work from our lab and collaborators on *inference everywhere anytime all at once*! Go talk to my incredible PhD students @huangdaolang.bsky.social & @chengkunli.bsky.social + amazing collaborator Severi Rissanen. @univhelsinkics.bsky.social FCAI

I wrote something up for AI people who want to get into bluesky and either couldn't assemble an exciting feed or gave up doomscrolling when their Following feed switched to talking politics 24/7.

The AI Researcher's Guide to a Non-Boring Bluesky Feed | Naomi Saphra

How to migrate to bsky without a boring feed.

nsaphra.net

1/10🔥 New paper alert in #AABI2025 Proceedings! Normalizing Flow Regression (NFR) — an offline Bayesian inference method. What if you could get a full posterior using *only* the evaluations you *already* have, maybe from optimization runs?

Tired of your open-source ML work not getting the academic recognition it deserves? 🤔 Submit to the first-ever CodeML workshop at #ICML2025! It focuses on new libraries, improvements to established ones, best practices, retrospectives, and more. codeml-workshop.github.io/codeml2025/

This is a great list, things that “the best engineers I know” do, stuff like: - understanding things deeply, reading the actual source - being willing to help other people - status doesn’t matter, good ideas come from anywhere endler.dev/2025/best-pr...

The Best Programmers I Know | Matthias Endler

I have met a lot of developers in my life. Late…

endler.dev

B200 go brrrr! It seems by doubling the TFLOPs you get double the speed. Cool stuff by Antonio and WavespeedAI team to get FLUX-dev inference (SOTA diffusion) in under a second on a B200. datacrunch.io/blog/flux-on...

FLUX on B200: Real-Time Image Inference with WaveSpeedAI + DataCrunch Collaboration

How WaveSpeedAI and DataCrunch achieved an up to 6x faster image inference by optimizing FLUX-dev's latency and efficiency: NVIDIA B200 vs. H100 benchmark.

datacrunch.io

1/ We asked GPT-4.5 -- allegedly the model with the best sense of humor, according to the site we do not talk about here -- to write a comic about our recent AISTATS paper on the Amortized Conditioning Engine (ACE). Then gpt-4o drew it. You judge the result... (text continues 👇)

The Llama 4 model that won in LM Arena is different than the released version. I have been comparing the answers from Arena to the released model. They aren't close. The data is worth a look also as it shows how LM Arena results can be manipulated to be more pleasing to humans. t.co/rqAey9SMwh

I wanted to change the color scheme on a blog for some plots so I decided to test Claude code. github.com/datacrunch-r... I didnt relaize it inserts "Co-Authored-By: Claude <noreply@anthropic.com>" I would have got away with it if it wasn't for that pesky Claude code.

Update plot-script.py to use 2025 brand colors · datacrunch-research/blogs@6955de0

- Added brand colors 2025 palette - Updated all plots to use the new color scheme - Regenerated all plot images with the new colors - Set consistent style across all plots 🤖 Generated with [Claude...

github.com

This is a new blog looking at the individual optimizations that went into serving the DeepSeek model class in SGLang. I have been observing the SGLang repo for a few months now, and it's crazy how quickly they integrate new optimized features. It's a very cool open-source project!

@datacrunch.io · last yr.

New blog post: Optimization techniques applied by the SGLang team for DeepSeek-V3 inference. You'll find a comprehensive overview of the techniques, their benefits and implications, and our benchmarks. datacrunch.io/blog/deepsee...

Llama 4 uses both interleaved chunked attention and global (NoPE) attention mechanisms, similar to a recent Cohere paper. It's cool to see innovation in attention layer architectures for the large models, and it showed to the world. arxiv.org/abs/2501.18795.

Rope to Nope and Back Again: A New Hybrid Attention Strategy

Long-context large language models (LLMs) have achieved remarkable advancements, driven by techniques like Rotary Position Embedding (RoPE) (Su et al., 2023) and its extensions (Chen et al., 2023; Liu...

arxiv.org

The Johnson–Lindenstrauss Lemma states that a set of high-dimensional points can be mapped into a much lower-dimensional space while approximately preserving pairwise distances. This is useful for dimensionality reduction, clustering, etc. stanford.edu/class/cs114/...

Free Ghibli images through the WaveSpeed API pretty cool! wavespeed.ai/blog/posts/2...

🚨 NVIDIA HGX B200: available NOW on DataCrunch! Be among the first to gain instant access to 1x, 2x, 4x, and 8x B200 GPUs with our high-performance VMs. Sign up and enjoy expert support with secure service where performance meets sustainability. 🔗 cloud.datacrunch.io

I just came across this paper from ICLR 2024 which proposes an intricate combination of transformers and diffusion models to generate forecasts with these uncertainty bounds (red box), which are clearly inappropriate and could likely be outperformed by modelling the data as a random walk...

🥉 SemiAnalysis awarded DataCrunch with bronze on the GPU Cloud ClusterMAX™ Rating! We thank their team for this independent evaluation, validating our approach to pushing the boundary of resource-efficient AI infrastructure ⬇️

Pretty exciting idea about enforcing sparsity by changing the activation function used. arxiv.org/abs/2503.16672

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity

In this paper, we demonstrate how to leverage 2:4 sparsity, a popular hardware-accelerated GPU sparsity pattern, to activations to accelerate large language model training and inference. Crucially we ...

arxiv.org

Training with more data = better LLMs, right? 🚨 False! Scaling language models by adding more pre-training data can decrease your performance after post-training! Introducing "catastrophic overtraining." 🥁🧵👇 arxiv.org/abs/2503.19206 1/10

This looks a cool workshop and great lineup! The structured foundation models, feels like part 2 of time series foundation models lots of cool problems to solve.

Nick Erickson@nickerickson.bsky.social · last yr.

We are excited to announce #FMSD: "1st Workshop on Foundation Models for Structured Data" has been accepted to #ICML 2025! Call for Papers: icml-structured-fm-workshop.github.io

We are excited to announce #FMSD: "1st Workshop on Foundation Models for Structured Data" has been accepted to #ICML 2025! Call for Papers: icml-structured-fm-workshop.github.io

I've been diving into the "black magic" world of CUDA recently. More posts may follow, but I think we're at an interesting point. Perhaps the "CUDA moat" is under pressure and perhaps changing how we interact with GPU programming. 🧵

Even though I know this is who I am talking to, I still can't help saying thank you when they help me out 😆.