Made a thing: Pollard is a MCP server for analyzing Firefox performance profiles. crates.io/crates/pollard

crates.io: Rust Package Registry

crates.io serves as a central registry for sharing crates, which are packages or libraries written in Rust that you can use to enhance your projects

crates.io

Highlighting some of my team's recent work: We've changed Materialize to use swap instead of memory-mapped files, with nice performance and efficiency improvements.

Materialize@materialize.com · 11mo ago

We’ve released a major improvement to our memory spilling infrastructure: Materialize now uses swap to scale SQL workloads beyond RAM. ✅ Faster hydration ✅ Efficient memory utilization ✅ Bigger workloads supported Full post from antiguru.bsky.social → bit.ly/46EF2iJ

Based on Timely 0.24, we released Differential Dataflow 0.17. Compatibility changes plus removing the Filter and Freeze wrappers because they weren't used and difficult to maintain. github.com/TimelyDatafl...

Release differential-dataflow-v0.17.0 · TimelyDataflow/differential-dataflow

Other Adjust Differential to recent Timely changes (#643) Remove Filter and Freeze wrappers (#644)

github.com

We just released Timely Dataflow 0.24! Here are some exciting changes from @frankmcsherry.bsky.social and myself. The container abstractions got a complete rework, and we introduce a new pattern to distribute data. Details below. github.com/TimelyDatafl...

We just released Timely Dataflow 0.24! Here are some exciting changes from @frankmcsherry.bsky.social and myself. The container abstractions got a complete rework, and we introduce a new pattern to distribute data. Details below. github.com/TimelyDatafl...

Release timely-v0.24.0 · TimelyDataflow/timely-dataflow

This version of Timely has some exciting new features. The Distributor trait offers a generalization of the Exchange type. It allows users to define custom distribution strategies for routing data...

github.com

A lot of performance improvements went into this one!

Materialize@materialize.com · last yr.

Materialize 25.2 is here! New features include live freshness reports for all your views, 2.5x faster data product deployment times, and native SQL Server support. See how these updates can help streamline your operations: bit.ly/44i2hg2

The Materialize engineering team uncovered a rare concurrency bug 🪲in Rust’s 🦀 unbounded channels that could lead to double-free memory errors. After thorough debugging and working with the Rust and crossbeam communities, the fix is now part of @rust-lang.org 1.87.0. 🔗 bit.ly/3Fan1Om

Agents generate more data and place more demand on systems than ever before—and standards like MCP will only accelerate this trend. Learn how Delphi is rethinking how they build data-intensive applications—from the db to the UI: bit.ly/42BZdf9

Scaling queries on agent-produced data: How Delphi transformed its data infrastructure

Join our webinar with Delphi to discover they evolved their data infrastructure to handle the rapid increase in AI-driven interactions with Materialize.

bit.ly

I'm very excited to share that @materialize.com now has a Self-Managed option, as well as a Community Edition license that lets you play around with the exact same bits but for free. materialize.com/blog/materia...

Materialize For Everyone: Introducing Self-Managed and our Free Community Edition

Unlock real-time data transformation with Self-Managed Materialize. Now, deploy Materialize in your own cloud for security, compliance, and performance control—powered by the same battle-tested, incre...

materialize.com

For interested folks, the RustNYC presentation I gave about the `columnar` crate is now available in the repo: github.com/frankmcsherr... It's not as good without the talk track, but it's got the benchmark results, you can admire the stylistic watercolors (so therapeutic to make; strong recommend).

We just landed the first use of the columnar crate in Materialize! Stay tuned for more. github.com/MaterializeI...

Columnar in logging dataflows by antiguru · Pull Request #30883 · MaterializeInc/materialize

Convert logging dataflows to columnar The aim of this PR is to convert some of the logging dataflows to use columnar data on dataflow edges, wherever it makes sense to do so. It introduces building...

github.com

Frank McSherry@frankmcsherry.bsky.social · 2y ago

For interested folks, the RustNYC presentation I gave about the `columnar` crate is now available in the repo: github.com/frankmcsherr... It's not as good without the talk track, but it's got the benchmark results, you can admire the stylistic watercolors (so therapeutic to make; strong recommend).

I wrote a bit about efficient maintenance of temporal filters in @materialize.com. By periodically restarting replicas, we can limit resource requirements (RAM and CPU), all without any other user-visible impact. materialize.com/blog/replica...

Replica expiration: Limiting temporal filters' resource requirements

Replica expiration is a new feature in Materialize that limits the resource requirements of temporal filters.

materialize.com

An ocelot with an odd shape at the Museum für Naturkunde in Berlin.

I was recently coveted a 10 year journal on jetpens, because that would be cool to do, and realized that I already had one! The blog turns 10 this month, and I thought it might be worth taking a tour through each of the years and seeing what I can jot down about each. github.com/frankmcsherr...

Origami tree at the @natural-history.bsky.social

Sonne additional details by @frankmcsherry.bsky.social bsky.app/profile/fran...

Sometimes it's easier to mitigate a problem than solve it: a feature I recently implemented reduces the memory and CPU @materialize.com requires to maintain future updates. Some details why solving the problem is hard and the mitigation easy-ish. materialize.com/changelog/20...

Sometimes it's easier to mitigate a problem than solve it: a feature I recently implemented reduces the memory and CPU @materialize.com requires to maintain future updates. Some details why solving the problem is hard and the mitigation easy-ish. materialize.com/changelog/20...

Improved performance for temporal filters

Optimization time! We've significantly reduced the cost of using temporal filters in Materialize.

materialize.com

Enjoying the call of the coquí in Puerto Rico

Just published lgalloc 0.4.0. The only change is that stats don't print errors anymore but forward to the caller. Docker on MacOS doesn't present numa information! crates.io/crates/lgalloc

crates.io: Rust Package Registry

crates.io

Interesting article on the future of aircraft design: hackaday.com/2024/11/21/b...

Blended Wing Body Passenger Airplanes And The End Of Winged Tubes

Ask someone to picture an airplane and they’re likely to think of what is essentially a tube with wings and a stabilizing tail tacked onto one end of said tube. Yet it is also no secret that …

hackaday.com