v
@avi.im
breaking databases @tur.so W1 '21 @recursecenter.bsky.social excited about databases, storage engines and message queues
I often get emails asking for ideas (or materials) to learn about distributed systems. I always recommend getting started with Gossip Glomers. Only six challenges, some easy and some difficult, but an absolute top tier fun and learning experience https://fly.io/dist-sys
Here is an insane piece of lore inside SQLite's source code I am researching VACUUM and I was studying their code. In VACUUM, SQLite creates a temp file prefixed with `etilqs_` Here is why:
AI is going to radically change how people learn and upskill. Last week, my fren asked for resources to learn distributed systems. I asked him to try Gossip Glomers. Today he mentioned that he did them all: he prompted GPT to solve them and then verified the solutions 🔥
redditor explains why job hunting in tech is exactly like auditioning for acting roles
In the 90s, Linus Torvalds had a much superior language to write the Linux kernel. But, since he is Finnish, he couldn't Smalltalk
AEADs provide a verification tag after encryption. For each page, we need a nonce too. Both the nonce & the tag become metadata for an encrypted page So where do you store them? We could store them separately, but it's much better & neater to store them in the page itself (1/5)
v@avi.im · 9mo agoThe B Tree data structure fascinates me. Databases use B Trees to store data on disk, organizing everything into pages that typically range from 4kb to 8kb. All I/O operations happen in units of these pages. The page looks like this... (1/9)
The B Tree data structure fascinates me. Databases use B Trees to store data on disk, organizing everything into pages that typically range from 4kb to 8kb. All I/O operations happen in units of these pages. The page looks like this... (1/9)
Pro database tip: enable `SQL_SAFE_UPDATES` in MySQL to avoid accidental UPDATE/DELETE queries without a WHERE clause. It forces you to use a key or a LIMIT, instead of wiping whole database by mistake at 2:19am.
Sharding. Database sharding is one of the common techniques to scale a database horizontally. You split the db into small parts called shards and distribute them across machines. Shards are typically in the few hundreds or even thousands (for extremely large databases).
SQLite has a page where they explain why they use C. They specifically elaborate on why not Rust www.sqlite.org/whyc....
excited to share that we are following through on our earlier commitments and putting together an independent+neutral organization to house the DID PLC system, includes the directory service
Creating an Independent Public Ledger of Credentials (PLC) Directory Organization | Bluesky
The Bluesky Social app is built on an open network protocol that refers to each user by a unique Decentralized Identifier, or DID (a W3C standard). The most popular supported DID method was developed ...
docs.bsky.app
Next week is the start of @db.cs.cmu.edu's latest seminar series: Future Data Systems @samarchdb.bsky.social and I are hosting speakers from leading systems in the datalake / lakehouse space. Mondays @ 4:30pm ET via Zoom. Open to the public. Videos posted to YouTube: db.cs.cmu.edu/seminars/fal...
The correct answer is either. Transaction B gets a snapshot that may or may not include the changes from A. SI does not provide real time guarantees. If you need that, you need Strict Serializability, which guarantees that transactions are ordered in real time.
v@avi.im · 11mo agoDatabase systems question Assume the database is in snapshot isolation mode. If transaction A updates, and writes x, commits, *then* transaction B starts and reads x's value, then B will see (assume single node for simplcity): 1 - Value before A's write 2 - Value written by A 3 - Either 4 - 🍿
Database systems question Assume the database is in snapshot isolation mode. If transaction A updates, and writes x, commits, *then* transaction B starts and reads x's value, then B will see (assume single node for simplcity): 1 - Value before A's write 2 - Value written by A 3 - Either 4 - 🍿
Published a new blog post: Setsum - order agnostic, additive, subtractive checksum post - avi.im/blag/2025/setsum code - github.com/avinassh/...
The great lock in is here! For those wanting to get into systems programming and/or database internals, consider hacking on Turso DB, the SQLite rewrite in Rust. Here's why: 1. It's a database!
Hey we're hiring for in-person engineering roles in SF. I really enjoy my job and you might too. Come hang out and build developer tools!
Where can I learn about how AI companies use caching, KV stores, and databases differently for LLMs, agentic workloads? Someone also mentioned to me that old/traditional services aren't suited for these, so they also build databases internally. (e.g., OpenAI acquired Rockset)
For nearly a decade, MongoDB provided reliable persistence with one of the most robust storage engines. Yet somehow, the oldest jokes keep persisting, too, so here are some facts: dev.to/franckpachot...
Resilience of MongoDB's WiredTiger Storage Engine to Disk Failure Compared to PostgreSQL and Oracle
There have been jokes that have contributed to persistent myths about MongoDB's durability. The...
dev.to
Netflix had it all wrong, don’t waste engineering resources to build your own chaos monkey infrastructure, just put production on AWS us-east-1 and you get chaos monkey for free. Just kidding, just kidding…
This is the opening text of Transaction Processing: Concepts and Techniques by Jim Gray "Six thousand years ago, the Sumerians invented writing for transaction processing."
My ”workshop” docs.trychroma.com/docs/overvie... github.com/ollama/ollam... www.zotero.org Basically I toss documentation into Zotero, parse the snapshots to create embeddings. The query first retrieves documents via the embedding, then shoves those into the LLM prompt with the query to respond
github.com
Published a new post: Oldest recorded transaction. This totally could have been just a tweet (skeet?), but I wanted to publish something today. avi.im/blag/2025/old...
Which is the best local LLM to set up for asking questions about code? I have large codebases like the Linux kernel, Postgres, etc. I want to ask questions like "find methods that do XYZ" and also post large code snippets and ask for explanations.
If you're you curious about #LeanLang and want to understand the connection between #programming and #proofs, check out this great new video by Ank Yog. The analogy between Chess and true propositions is particularly compelling! www.youtube.com/watch?v=QXQN...
My extreme opinion is that anything other than serializable isolation is a scam. Database people haven't figured out how to make it fast, so we have ended up with other half baked isolation levels.
This is the oldest transaction database from 3100 BC - recording accounts of malt and barley groats. Considering this thing survived 5000 years (holy shit!) with zero downtime and has stronger durability guarantees than most databases today. I call it rock solid durability.
Nice one! Spoke about this kind of architecture a while ago too: speakerdeck.com/gunnarmorlin.... Back then, using Kafka Streams as an IVM engine and Infinispan as the serving layer for denormalized views, but Sqlite (or DuckDB) would work too, with better queryability, as you say m
Keep your cache always fresh with Debezium! (Current 22)
The saying goes that there are only two hard things in Computer Science: cache invalidation, and naming things. Well, turns out the first one is solved …
speakerdeck.com
