We're proud to announce the first release of our latest data compression project, Zstandard in Rust! Read why we did this, the current state of the project, and where we're hoping to take it: buff.ly/JoruN7e Thanks to Chainguard, Astral, NLnet Foundation, and @sovereign.tech. #rustlang #opensource

Announcing Zstandard in Rust - Trifecta Tech Foundation

Over the past year, we've been silently working on our third compression project. After zlib and bzip2 we're now taking on zstd with libzstd-rs-sys, and are proud to announce its first release.

trifectatech.org

fjall is at 1989 stars 😵 - The road to 2K is on!!

Oh yeah bluesky exists too. I just released 3.1 - which features compaction filter support, meaning you can custom logic during compaction (e.g. TTL rules, or soft deletions)! fjall-rs.github.io/post/fjall-3...

Fjall 3.1: Compaction filters

Run custom logic during compactions

fjall-rs.github.io

the sheer speed from initial bug report to @fjallrs.bsky.social getting a fix fully released is just wild.

Release 3.0.1 · fjall-rs/fjall

[fix] Edge case that could result in the second memtable being dropped before being flushed [fix] Custom filter policy not being applied

github.com

A strange moment, but - I just released 3.0.0. fjall-rs.github.io/post/fjall-3/ What has started as a rewrite of the block format has become almost a full rewrite, interrupted by a bachelor's thesis, faulty hardware and a couple of weeks of sickness - but I think the end product is pretty good.

Releasing Fjall 3.0

A year in the making

fjall-rs.github.io

Love it when SQLite just stops working but allocates hundreds of GBs of disk space.

Check out my talk on physical replication and Graft that I delivered at the recent @syncconf.bsky.social in SF. youtu.be/QoKzDyH2MEA?...

Carl Sverre (SQLSync) - Why Physical Replication Still Matters

YouTube video by Local-First Conf

youtu.be

Well... looks like my Kingston disk just died...?!

This year's winter album... goes to Maria Somerville youtu.be/k9qEKyQ7jc4

Maria Somerville - Projections (Official Video)

YouTube video by Maria Somerville

youtu.be

RC 1 is out! I forgot to post about RC 0, but here we are. At this point API and disk format are unlikely to further change - now it's all about stabilization for the final release... crates.io/crates/fjall...

crates.io: Rust Package Registry

crates.io

v3 pre.6 is out - probably the penultimate prerelease before going into release candidate(s). At this point, all major architectural reworks are done, and just some APIs need some changes. Followed by a final cleanup and stabilization phase. crates.io/crates/fjall...

crates.io: Rust Package Registry

crates.io

3.0.0 pre.5 is out and brings journal compression support. Here's 16K JSON blobs being written:

v3 pre-release 2 is out! Still not file format -or API stable, but most importantly this release marks lsm-tree being *feature complete*. crates.io/crates/fjall...

crates.io: Rust Package Registry

crates.io

Well, it was a faulty RAM stick everybody

So I have now tried btrfs, xfs, ext4 on two diff. SSDs, updating to Ubuntu 24; and I'm still getting corruptions... at this point I don't really know what to do. This is not code specific (RocksDB is also affected), yet I don't understand how my OS and all apps just seem to continue to work fine...

95% Zipfian reads, 5% random updates on a Kingston PLP SSD with 16K JSON blobs - sled DNF! (OOM) fjall uses LZ4 compression

constellation runs on a bare metal server with 4 dedicated CPU cores, 16GiB memory, 1TiB fast NVMe SSD attached, and 500mbit unmetered network connection.

Why@why.bsky.world · 10mo ago

turns out raspberry pis are really fast these days

Blob GC working again (now fully automatic) - much less impact on foreground work (old GC was stop-the-world)

Fun coloured bars and lines

If you sponsor NOW (github.com/sponsors/fja...), you can get an exclusive preview of version 3!! jk I pushed the first pre-release to crates.io - it's unstable though! Breakages guaranteed, and some APIs are missing! bye

Sponsor @fjall-rs on GitHub Sponsors

Support fjall-rs's open source work

github.com

If you sponsor NOW (github.com/sponsors/fja...), you can get an exclusive preview of version 3!! jk I pushed the first pre-release to crates.io - it's unstable though! Breakages guaranteed, and some APIs are missing! bye

Sponsor @fjall-rs on GitHub Sponsors

Support fjall-rs's open source work

github.com

ngl v3 is a bit more involved than I originally planned or anticipated; I don't want breaking changes for a while, so it becomes this chase of making sure everything is really future-proof

I will never do a major release like this again

I really dread having to some day write a quick_cache replacement, tailored for lsm-tree

friendship with ext4 ended, xfs is my new best friend (whyyy is ext4 still giving me corruptions sometimes, even after updating my kernel??)

Yeah, that's a bit better

For weeks I've been regularly getting corruptions while running benchmarks... for RocksDB, too. It's infuriating because running a 15min+ benchmark is a coin flip whether it completes. Seems like it was an issue with my arguably outdated Linux kernel... there were ext4 corruption bugs a while ago.

For a framework renowned for its innovation, it is funny how little Solid has changed in the last 7 years. Yeah we've innovated on features, fine-grained concurrency/hydration, streaming, server functions, single flight mutations, but the core tech and structure are untouched.

History repeats itself

Lovely 3D plots to sweeten the day