Seeing more and more people writing about swarm observability so thought I'd post some of the work I'd been doing to think this through. I built some tools to help AI engineers work with agent swarm traces and other signals. Specifically:
Alex Strick van Linschoten
@strickvl.bsky.social
ML Engineer (@ ZenML), researcher (& author of a few books).
When I started looking into agent swarm observability, I really wanted some real data to work with. I wasn't really able to find much, so I created my own and am releasing it on the @huggingface Hub for others to use.
4 real production LLM use cases (curated by yours truly). 2x per week. Under 3 minutes to read. Next email going out on Tuesday. Subscribe here: zenml.io/llmops-data...
As I learn more about agent swarm observability, I've been guessing at what people are recording about their agent swarms. I collated what I found, the evidence each one needs, and some ideas for capturing it. If you work on this, lmk where I'm wrong! alexstrick.com/posts/2026-...
Finally got round to adding in a way to subscribe to my blog / site via email. (Always supported RSS, but seems that's gone out of fashion a bit 🤷 ) Also added in a way to browse some of the blog series I've written over the years instead of just scrolling through a huge long list of posts.
Got interested in agent swarm observability after reading through some of the reports from labs who are working on this frontier capability. Seems like quite a bit of work still needing to be done. Some more posts to follow. alexstrick.com/posts/2026-...
Reading the Swarm: Why Today’s Agent Tracing Tools Fall Short – Alex Strick van Linschoten
Why agent swarms need observability tools built for them: a 2,000-agent example, the historian’s case for treating agent explanations as testimony rather than fact, and a look at what exists so far.
alexstrick.com
Jev from @typesafeai has obviously been on everyone's minds this past week. 'System One' models, decision models, smart classifiers, whatever you want to call them... they're very useful. We even built them into Kitaru as a whole new 'judge evaluator' package.
New: Kitaru now ships with a @typesafeai Jev evaluator that makes it really easy to run evals across your agent traces!
I love watching all the new videos on the AI Engineer YouTube channel. It's one of the main venues where people building agents really show their work. The problem is that it publishes more talks than I can honestly keep up with. So I made AIE Talks, a written archive of the channel!🧵
Kitaru's a product which very much rewards trying it out, but for those of you who just want to see it in action, I put together a video walkthrough that takes you through: - importing your traces (from @langfuse in this case) - registering your agent (@pydantic AI featured for the example)
New legal dataset up on the Huggingface Hub! Over the weekend I worked to finalise a snapshot of the official legal documents hosted by the Guantanamo Military Commissions. This is mostly court filings and transcripts from all the cases, around ~52GB of data.
When asking models to draft prompts (esp for things of more consequence like long-running processes / tasks), is it better to ask a model from the same family to write the prompt, or to use a different model family?
Published my first little environment on the PrimeIntellect Environments Hub yesterday evening. Very happy to finally have that out and complete! (links and more comments below, and a blog to follow I guess)
Latest ZenML release has a lovely new feature: an SSH orchestrator and stack component.
Have been working on reproducing my old ISAF press releases finetuning work that I originally did for the Hamel/Shreya course a couple of years back. I figured it's as good a way to get to grips with agentic RL as any, and so the first thing I did was to get to know my data a bit better.
Trying to understand the recent 'model routers are all you need' discourse. Obv frontier APIs / endpoints can be flaky and you probably want to have some kind of a (tested) fallback.
Sometimes you have to do a bit of data cleaning first before you get to the fun stuff. This week (aside from being sun-addled from the heatwave) I worked on that. It should unlock the real RL stuff that I am focusing on. (The temperature drop should also help with that) alexstrick.com/posts/2026-...
Is this data actually good, or does it just look good? – Alex Strick van Linschoten
Before trusting my ISAF press-release dataset to train and evaluate an RL model, I audited its gold labels and found misspelled provinces, ambiguous values where ‘unknown’ is the honest answer, and a subtle train/test leak. I cleaned it without overwriting the original.
alexstrick.com
Wrote my first RL environment this evening. A very simple on, mind, but 'verifiers' (by @PrimeIntellect and @willccbb) makes it very easy to slot in the pieces.
I'm now transitioning from the part of my agentic RL exploration where I learned the high-level concepts to seeing what people are doing in practice.
Whenever a frontier lab drops a new model you always see their employees posting things like "you’ll be surprised by how good we made our new model! throw your hardest problems at it".
Doing a bit of a self-study RL course at the moment and one of the really useful tweaks I always have my 'teacher' do is to revisit the early FastAI lessons from @howard.fm and to really live up to those invitations to make things interactive, to get a sense for how things work intuitively.
Published a new post on our Kitaru adapter for Claude Agent SDK. Claude owns the agent loop. Kitaru records the completed invocation as durable workflow state: result, artifacts, waits, and replay boundary. One completed invocation = one checkpoint.
OpenAI Agents SDK is a great harness. When you move your agent to production, you're probably going to need and want more. That's where Kitaru comes in... We build an adapter so you can keep your OpenAI Agents SDK code, but throw in some durability and other goodies on top.
Had fun chatting with Hamza for this CNCF webinar last week, all about Kitaru, durable agent harnesses + agent runtimes. The video is embedded in the link in the thread. If you have agents in production and are experiencing growing pains around the runtime layer of the stack, we'd love to talk!
Just made a bumper release this evening: 25 new format adapters covering the major cloud annotation platforms, autonomous-driving and aerial datasets, document layout, synthetic data, and the long tail of academic/community formats.
"Your Harness, Your Memory" by Harrison Chase argues that memory belongs inside your agent harness, not behind a third-party API. We've been building exactly that, and Kitaru 0.4.0 shipped it this morning. kitaru.ai/blog/kitaru...
Kitaru agents now have memory
Durable, versioned memory for agents is now built into Kitaru — across Python, the typed client, the CLI, and MCP.
kitaru.ai
We just shipped migration skills to allow you to migrate off 11 ML/data platforms to ZenML: Airflow, Argo, AzureML, Dagster, Databricks, Flyte, Kedro, Metaflow, Prefect, SageMaker, Vertex AI. Each has hand-curated concept maps baked in showing what maps 1:1 and what needs redesign.
I've been building panlabel — a fast Rust CLI that converts between dataset annotation formats — and I'm a few releases behind on sharing updates. v0.3.0: Hugging Face ImageFolder support v0.4.0: auto-detection UX overhaul + Docker
Last month I migrated our ZenML website from Webflow to Astro in a week during a Claude Code / Cerebras hackathon. 2,224 pages, 20 CMS collections, 2,397 images. The site you see now is the result. Didn't win the hackathon but got a production website out of it, so I'll take that trade.
panlabel 0.2 is out. It's a CLI tool (and Rust library) for converting between different dataset annotation formats. Now also available via Homebrew.