Kafka message that refuses to die: The poison pill problem💊

Imagine a Kafka consumer processing millions of events every day. Everything runs smoothly until a single malformed message arrives…

adaksh5.medium.com

Kafka CommitFailedException: Causes and Prevention - Java Code Geeks

Java kafka commitfailedexception: Learn Java Kafka CommitFailedException causes, solutions, and best practices to avoid consumer failures.

javacodegeeks.com

What Running Kafka On Vms Taught Us About Systems Thinking | Moniepoint | Engineering

Most infrastructure stories get written after something breaks. This one is diff...

engineering.moniepoint.com

Schema Evolution in Streaming Systems - codewithfimi.com

Learn what schema evolution is, how it works in streaming systems, and the best practices for managing changing data structures without breaking data pipelines.

codewithfimi.com

What Really Matters When Building Applications with Kafka: The Production Survival Guide

When developers first adopt Apache Kafka, they often treat it as a glorified message queue — a faster, distributed RabbitMQ or SQS. But…

medium.com

Scaling StreamHub: Transitioning from Kinesis to Kafka for 145 Billion Daily Events - Inside Atlassian

How Atlassian scaled StreamHub from 22B to 150B daily events by migrating from Kinesis to Kafka, and the hard lessons learned along the way.

atlassian.com

How Kafka Actually Works: Why an Append-Only Log Was the Right Shape All Along

Part 6 of “How It Actually Works” — a series breaking down the systems every backend engineer depends on but few can explain end to end.

medium.com

Eligible Leader Replicas (ELR) in Kafka 4.1: What You Need to Know

Discover how Eligible Leader Replicas (ELR) and KIP966 in Apache Kafka 4.1 solve the Last Replica Standing issue to protect committed data during unclean shutdowns.

softwaremill.com

Your model is already outdated: online Machine Learning training with Apache Kafka and DJL - Paolo Patierno

paolopatierno.dev

Kafka Streams Dead Letter Queue in Spring Kafka 4.1

Introduction Spring Kafka 4.1 raises the Apache Kafka baseline to 4.2.0. With this release, it introduces Spring-friendly integration for Kafka Streams KIP-1033 (processing exception handling) and KI...

blogit.michelin.io

Streams Are Not Queues - Shahbaz Ahmed

My personal blog

shahbaz.cc

Broker-Visible vs Client-Local Parallelism — Jack Vanlightly

This post is a little side-quest from my “Kafka Share Groups and Parallelizing Consumption” series. My “Kafka Share Groups and Parallelizing Consumption” series ( part 1 , part 2 ) has been laser f...

jack-vanlightly.com

Getting Slack Notifications From Kafka Events — Streamling

Create a Streamling pipeline to consume data from Kafka, filter it with the SQL transform, prepare a Slack webhook payload with the TypeScript transform and deliver it to the Slack webhook endpoint wi...

streamling.dev

[EN] Why You Shouldn't Put Kafka Traffic in Your Service Mesh

In 2020, I gave a talk about Linkerd at DevOpsDays São Paulo. One of the recommendations in the...

dev.to

Complete Understanding Event-Driven Architecture With Apache Kafka

Interduction

medium.com

The Schema Proliferation Problem in Kafka and Flink Pipelines: How to Solve It

Schema proliferation builds slowly and gets expensive fast. One schema per event type feels right until there are ten tables, union queries spanning all of them, and a single field rename touching eve...

infoq.com

SenseHat with PI4J

YouTube video by Igor Souza

youtube.com

Day 65/100 - Implementing Exact-Once Semantics with ClickHouse® and Kafka

Introduction Real-time data pipelines power modern applications such as analytics...

dev.to

I Asked My AI Coding Assistant Why Kafka Was Lagging. It Diagnosed the Problem in 30 Seconds

How the Agent Toolkit for AWS turns natural language into expert-level MSK operations.

msayem.medium.com

Kafka in Production: 5 Problems Every Engineering Team Eventually Faces (And How to Solve Them)

Kafka is incredibly powerful — but production is where the real challenges begin.

medium.com

Running Pi4J on the NVIDIA Jetson Nano

Story Horse? This post was inspired by the excellent article by WebTechie, First Test of Java on...

dev.to

Kafka Queues for Quarkus

Learn how to use Queues for Apache Kafka (KIP-932) with the Quarkus framework, including share group configuration, explicit message acknowledgment, and cooperative consumption from a single partition...

developer.confluent.io

Where Kafka's Disk Is Going: Object Storage and the New Streaming Layer | Pletor Tech Blog

A practical look at WarpStream, AutoMQ, Apache Fluss, and Kafka Diskless Topics as the Kafka ecosystem moves durable storage away from broker-local disks.

tech.pletor.kr

Apache Kafka Essentials Every Java Developer Should Know for Interviews

If you’ve attended Java Backend or Microservices interviews, you’ve probably noticed one thing:

medium.com

Kafka Share Groups and Parallelizing Consumption - Part 2: Producer Batches and share.acquire.mode — Jack Vanlightly

In the last post we used simulated consumer processing time to reveal how important it is to set an appropriate value for max.poll.records. The rule of thumb was a value somewhat lower than: group....

jack-vanlightly.com

Kafka Share Groups and Parallelizing Consumption — Part 1: Tuning max.poll.records — Jack Vanlightly

All tests were executed against Kafka 4.2.0 using Dimster .  In the last post we measured the overhead that the mechanics of share groups adds, and saw that it is pretty small. Likewise we...

jack-vanlightly.com

Ditch Debezium + Kafka: How Apache Flink CDC Can Cut Your Streaming Infrastructure in Half

The Traditional CDC Stack and Its Hidden Costs

medium.com

The State of Streaming to Apache Iceberg in July 2026: Every Path, Its Latency, and What to Do When Seconds Are Not Fast Enough

The most common architecture question I get in 2026 is no longer "should we use Iceberg." That one is settled.

linkedin.com

Putting a Kafka Topic Naming Convention into Practice with Terraform

Putting a Kafka Topic Naming Convention into Practice with Terraform

jonasg.io