The New Wave of the Streaming Log Technologies www.streamingdata.tech/p/the-new-wa...

The New Wave of the Streaming Log Technologies

Streaming logs are entering a third wave: Kafka-free systems rethink performance, simplicity, routing, protocols, and object-native design.

streamingdata.tech

Playing with Redpanda SQL today

Stonebraker is not happy with existing Text2SQL tooling: cacm.acm.org/blogcacm/if-...

I wrote a tutorial on using Streamling to send rich Slack notifications based on Kafka events. The tutorial highlights how you can easily combine SQL and TypeScript transformations to filter data and get it precisely into the shape you need.

S2 source and sink plugins are now available for Streamling. Read durable S2 streams into Arrow batches, or write Streamling output back to S2. ClickHouse ingestion, Postgres CDC, and a full Postgres → S2 → ClickHouse example: s2.dev/docs/integr...

ClickHouse Cloud has the best onboarding experience. 2-3 clicks, done.

Pretty relevant these days

Earlier this week, I presented a talk at the Vancouver Systems Meetup: "Streamling: Building a Lightweight Data Streaming Runtime on Apache DataFusion".

I'm excited to announce Streamling: lightweight, performant and extensible data streaming runtime. Streamling is built using Rust, Apache Arrow and Apache DataFusion.

Analyzing a 1000+ line Flink SQL pipeline today 🫠

Found a love letter to @andypavlo.bsky.social at #PgConf in Vancouver

streamacademy.io/tutorial/fli...

Apache Flink: Postgres to Postgres Replication with Flink CDC

A practical guide to building a generic, multi-table Postgres-to-Postgres replication pipeline with Flink CDC - one replication slot for any number of tables, automatic schema discovery, and a custom ...

streamacademy.io

www.streamingdata.tech/p/can-kafka-...

Can Kafka Queues Make Consumers Faster? Part 2: Head-Of-Line Blocking

Kafka Queues can scale consumers beyond partitions by reducing head-of-line blocking when message processing has I/O delays.

streamingdata.tech

Just in the nick of time, it's my round-up of Interesting Links for April 2026! There's links about all sorts of stuff around Data Engineering, CDC, Iceberg, Data architectures, and even a little hint of AI (but only the useful kind, not the gross hype-y kind). rmoff.net/2026/04/30/i...

Interesting links - April 2026

rmoff.net

Last night 🤟🥁

www.streamingdata.tech/p/can-kafka-...

Can Kafka Queues Make Consumers Faster?

Kafka Queues promise consumer scaling beyond partitions, but benchmarks show share consumers still lag far behind standard Kafka consumers.

streamingdata.tech

www.independentengineer.co/p/how-to-gro...

How to Grow Your Network

Learn how software engineers can build a valuable network before going independent by helping others, nurturing work relationships, attending events, and joining communities.

independentengineer.co

Sharing some thoughts this morning about different paths you can take as a software engineer who's looking for more independence, and perhaps, eventually becoming a solopreneur. It's not just about building a product or becoming a consultant. www.independentengineer.co/p/software-e...

The Many Ways Software Engineers Can Go Independent

How software engineers can go independent: consulting, productized services, SaaS, courses, newsletters, and other realistic solopreneur paths.

independentengineer.co

Data Streaming Academy has tutorials now! The first tutorial covers reading and writing Kafka consumer offsets in Apache Flink using the State Processor API. streamacademy.io/tutorial/fli...

Recently, I spent a lot of time benchmarking various Iceberg sinks (Spark, Flink, Kafka Connect) and trying to beat @supermetal-inc.bsky.social for moving data from Postgres to Iceberg. Here's my report: thenewstack.io/postgres-ice...

Postgres to Iceberg in 13 minutes: How Supermetal compares to Flink, Kafka Connect, and Spark

Supermetal's new Iceberg sink snapshots Postgres data in 13 minutes versus 90+ for Flink, Kafka Connect, and Spark in this CDC benchmark.

thenewstack.io

Is this peak bubble? techcrunch.com/2026/04/15/a...

After sale of its shoe business, Allbirds pivots to AI | TechCrunch

Allbirds is ditching wool sneakers for AI servers, rebranding as NewBird AI after locking in a $50M convertible financing facility.

techcrunch.com

Never gets old

Second post in The Independent Engineer (my new newsletter!) focuses on a very important question: how to (re)discover your calling as a software engineer? I believe specialization is important, especially if you'd like to explore the idea of becoming a solopreneur in the future.

Why does it take AWS 40 (!) minutes to provision a 3-node MSK cluster? Do they manually bring a new server rack for this? I understand why a rolling update might be slow, but creating from scratch? Really?

I'm starting a new newsletter! For a while, I wanted to share my experience building an independent business. Specifically, building a business as a software engineer. A business where you keep building systems, solving problems, writing code - whatever you really enjoy doing.

It's beautiful github.com/nicolasmd87/...

GitHub - nicolasmd87/aether: A compiled actor-based programming language with type inference, designed for concurrent systems.

A compiled actor-based programming language with type inference, designed for concurrent systems. - nicolasmd87/aether

github.com

The Advanced Apache Flink training is now available as a self-paced, on-demand course! I ran the first cohort of the training in January and also had the chance to conduct a few corporate training sessions with the same program. I’ve received a lot of great feedback, iterated and polished content.

Recording and slides available from the DataFusion Stockholm Meetup: Recording: youtu.be/9u4cNmL14Xs Slides: github.com/apache/dataf...

Apache DataFusion Meetup Stockholm, March 2026

YouTube video by Andrew Lamb

youtu.be

I'm not asking much, can we open two files with the same name in 2026?

Practicing