You can now use S2 as a backend for Mastra Durable Agents! The @ s2-dev/mastra-pubsub adapter maps each durable topic to an S2 stream, with live delivery, replay after refreshes or process restarts, and distributed leases for coordinating thread ownership across processes.
What if your brain could be a pet? @anthropic.com have shared the benefits of decoupling the brain from its hands. We have also written about using a stream per agent session as a WAL and what that unlocks. If it seems complicated, it is. And durable execution frameworks are far too intrusive.
S2 source and sink plugins are now available for Streamling. Read durable S2 streams into Arrow batches, or write Streamling output back to S2. ClickHouse ingestion, Postgres CDC, and a full Postgres → S2 → ClickHouse example: s2.dev/docs/integr...
Also considered EFS, inspired by S3 Files being EFS in a trench coat. Built an async Rust lib that lets you use NFS directly from application code. It performs better than a Linux mount, but writes are an order of magnitude slower than what's achievable by purpose-building. Reads look great though.
shikhar@schmizz.net · 4w agothinking about a single-digit ms storage class for @s2.dev it's a plunge from just being s3-based, but gotta do what you gotta do?
thinking about a single-digit ms storage class for @s2.dev it's a plunge from just being s3-based, but gotta do what you gotta do?
Seems like Local AI is getting more and more relevant, so I wrote about it! s2.dev/blog/local-ai
Serving Local AI on my Jetson through Durable Streams
Its time to self-host!
s2.dev
If you have wondered whether to use the s2.dev cloud service, or self-host s2-lite (github.com/s2-streamstore/s2) – s2.dev/docs/s2-lite...
Cargo.lock +40 -1,142 nbd just @sbalogh.bsky.social replacing polars with csv
Serverless gets clowned on sometimes (ha ha, there are servers) but is ultimately about pricing usage and not having to provision resources you don't use, or think about such things as "clusters". It is genuinely hard for stateful services, but I don't think there is a better DevX.
turns out "stream per session" is not just for agents. it works for browser sessions too. built RePlaya, a self-hosted session replay tool where each visit is just an @s2.dev stream you can play back or tail live. recording = append playback = read live replay = tail github.com/s2-streamsto...
Streams as a cloud storage primitive, now in more places. S2 is the serverless API for unlimited, durable, real-time streams. Today, we're adding US West and EU North as new basin locations. s2.dev/blog/locatio...
Two new locations for your streams: US West and EU North
S2 introduces location as a first-class construct, with two new ones to choose from: western US and northern Europe.
s2.dev
Maybe QUIC is an opening for a gRPC challenger. I would call it Spooky. For spooky action at a distance, the outmoded critique of RPC.
shikhar@schmizz.net · 3mo agoOn the retirement of S2's gRPC API, realized how to describe our current API: REST-first. REST undersells, it supports append & read sessions over HTTP/2 similar to gRPC. We'll do HTTP/3 too when the ecosystem is ready.
On the retirement of S2's gRPC API, realized how to describe our current API: REST-first. REST undersells, it supports append & read sessions over HTTP/2 similar to gRPC. We'll do HTTP/3 too when the ecosystem is ready.
maybe it's not too surprising plaintext can be competitive with vector dbs when you consider agents effectively do semantic grep by query expansion
@s2.dev now supports client-supplied encryption keys for durable streams. Bring your own per-request key, keep key material out of S2 storage and logs, and let S2 handle request-time encryption behind the stream API. s2.dev/blog/encrypt...
Your data, your keys
S2 now supports client-supplied encryption keys for durable streams, giving teams control over key material without changing record APIs.
s2.dev
Today I felt like a cigarette. I have to say "coding" agents are a champ at operational investigation.
new Dart SDK for @s2.dev from the community: pub.dev/packages/s2_...
s2_sdk_dart | Dart package
Dart SDK for S2 Stream Store.
pub.dev
This is video conferencing on S2, but also... video conferencing on S3. Really shows off the power of pipelining writes!
Mehul Arora@infiniteregrets.bsky.social · 5mo agoSorry folks, if you are going to video conference over a database, it better be a Stream Store. s2.dev/blog/video-c...
Pipelining is now default in latest s2-lite! 8x increase in throughput and 6x lower latency when running against an S3 standard bucket.
shikhar@schmizz.net · 5mo agoExcited to enable pipelining by default in s2-lite with this upgrade! Expect lower latency and higher throughput. github.com/s2-streamsto...
There's two new ways to manage S2 as code! The Terraform provider lets you manage resources with full lifecycle support. Available on the Terraform Registry at s2-streamstore/s2. Or you can use the CLI by `s2 apply` which takes a declarative JSON spec and creates/reconfigures specified resources.
Excited to enable pipelining by default in s2-lite with this upgrade! Expect lower latency and higher throughput. github.com/s2-streamsto...
default to pipelining appends and remove `S2LITE_PIPELINE` knob · Issue #48 · s2-streamstore/s2
Improves performance a lot against high-latency object storage Depends on slatedb/slatedb#1138
github.com
Chris@chris.blue · 5mo agoSlateDB 0.11.0 is out! We had 5 new RFCs, 10 new contributors, and 3 new language bindings. This is a big release with a bunch of improvements and new features. 🧵
The s2.dev dashboard now has a Data Plane studio to make it convenient to work with streams. You can: -Do one-shot reads, live tail from any stream, start from an offset, seq no., or timestamp -Append records in multiple modes -Run stream commands -Inspect records with command record aware rendering
