@HamelHusain keeps stopping eval reviews for the same reason: the model isn't broken, but the product is. In part 2 of our series Rise of the Agent Engineer, Hamel walks through why ambiguous inputs, generic metrics, and disconnected reviews make AI evaluations misleading, and how to fix them.

What if your agents got better every time they failed? Today, we’re launching Signal. It continuously reviews production traces, finds issues, and turns them into an investigation with evidence, root cause, and a proposed fix. Your engineers decide what ships. arize.com/blog/from-s...

Want to master the full workflow of shipping reliable AI agents? Laurie's workshop from AI Engineer World's Fair, "Evaluating and Shipping AI Agents That Work," is now a free, self-paced course on Arize University. Earn a certificate you can share on LinkedIn by completing 13 episodes that cover:

Two AI observability lessons from @bookingcom: - An agent latency spike came from a model running without the appropriate service tier. - Multi-turn eval scores fell because long URLs were being added back into the conversation history, causing the context to balloon.

Claude Opus 5 is available in Arize AX, supported across Anthropic API, AWS Bedrock, and Vertex AI! Opus 5 brings major improvements for long-running agents in coding and professional work. Instrument, evaluate, and improve your agents with day 0 support for Anthropic's newest model.

Token price tells you what a model costs to call. It does not tell you what it costs to finish the job. Arize and @FireworksAI_HQ benchmarked 10 models across 2,400 agent runs, including Kimi K3.

Gemini 3.6 Flash and Gemini 3.5 Flash-lite are now available in Arize AX! Both models are supported on direct Gemini and Vertex AI providers with configurable thinking levels, 65K max output tokens, and audio file input.

Coding agents become trustworthy when verification is designed into the development loop. At @cursor_ai, that loop helps roughly 30-40% of pull requests in its ecosystem merge without human review. Here is how it works: 🧵

Git can show you the code diff. But only traces and evals can show you the behavioral diff. That’s the difference between an agent change that looks right and one you can prove is better.

Your AI agent generated 40 PRs. Great. But how many were merged? How much rework did they create? And what did each successful PR actually cost? Tokens, prompts, and output volume measure motion. But measuring AI productivity requires connecting traces to outcomes.

There's a lot of talk about loops recently. But the term “loop” currently describes at least four different architectures: execution, task, product, and system (plus the human oversight loop governing them).

GPT-5.6 support just went live in Arize AX. 🚀 Now available: 🌞 gpt-5.6-sol 🌍 gpt-5.6-terra 🌙 gpt-5.6-luna Compare all three side-by-side in the Prompt Playground, plug them into LLM-as-a-judge evals, and watch them in production - all in one place. Try it 👇 app.arize.com/

An agent was told: “make the tests pass.” It deleted the tests. That story from WorkOS founder Michael Grinich is funny on its face. But it's also the exact reason agent engineering is getting harder. Full conversation below.

Most teams hear the same advice: “add evals.” But when you’re staring at a real LLM app, that advice gets vague fast. Should your first eval be an integration test? A golden dataset? A CI gate? A dashboard metric? An LLM judge?

Agent harnesses are becoming the durable layer of AI coding workflows, according to @aparnadhinak. The model answers once. The harness turns that answer into a loop: context, tools, permissions, edits, tests, failures, retries, recovery, and traces.

The difference between an agent that works and one that games you comes down to one habit: a good eval. ✅ Spell out the shortcuts you won't accept ✅ Check that the work actually happened ✅ Try to cheat it yourself first ✅ Test it on real traffic

“Which model is cheapest?” is the wrong question. The better question: which model is cheapest per successful task? A model that looks cheap at the token level can be expensive if it needs retries, tool calls, or human cleanup to finish the task correctly. /1

50 traces. That’s how much data @HamelHusain says you need to start building evals that actually work. Pull them. Label them with a PM. Cluster the failures. Pick the highest-impact one. Write a binary eval You’ll learn more in an hour by doing this than in a month of dashboard watching.

A year ago, 200 instructions was the ceiling. Today it's closer to 2,000 - and up to 5,000 on the strongest models. The capacity problem is largely solved, but the verification problem is wide open.

@SnorkelAI will be in the Evals track with us at AIE! Rustem Feyzkhanov will be talking about how agent evaluation is moving beyond reviewing static traces and into executable simulation environments that let you test agents repeatedly across realistic tasks.

Your LLM gateway can be more than a router. With TrueFoundry + Arize, every model call becomes an OpenInference trace: spans for auth, model resolution, provider calls, token usage, latency, cost, and more. Here's how: arize.com/blog/trace-...

Saturday in Dalston: a room full of people trying to make London even more lovable (and livable!), in eight hours. Will we see you there? luma.com/maxxing-london

Londonmaxxing 003: Maxxing London Hackathon · Luma

How can we make London better to build in, live in and fall in love with? London is on a generational run. Billions in investment is flowing into the city,…

luma.com

You can have production-quality evals running in minutes. Our Solutions Architect Ankur Duggal @Anky488 is leading a hands-on workshop at AI Engineer World's Fair, walking through how to stand up a production eval pipeline in minutes using Arize Agent Skills, no prior setup required.

Come see what we've been building at Arize. Our Fuad Ali is leading a live walk-through of the latest features in Arize on Day 1 at AI Engineer World's Fair.

Two workshops. Two chances to help you move from vibes-based development to production-ready AI agents.

Excited to have Uber on the Evals track with us at AIE. Soumya Gupta and Jai Chopra are presenting how @Uber used closed-loop evals for their food photography enhancement agent.

What does a failing agent look like when all your metrics say it's fine? Our Strategy lead Dat Ngo is unpacking one of the most common failure patterns in production AI: agents that report success without actually succeeding.

Long-running agents need more than logs. You need to know: 👉 which LLM/tool call failed? 👉 did quality regress? 👉 what did it cost? We teamed up with Restate to show durable agents + Arize Phoenix traces/evals in action. Read more: www.restate.dev/blog/arize-...

Resilient, observable agents with Restate and Arize Phoenix

Combine Restate and Arize Phoenix for agent reliability, observability, and evaluation in one stack.

restate.dev

Voice agents are one of the fastest-growing categories in AI and one of the hardest to debug.

Code review was designed for a world where humans wrote all the code. What happens when that world is gone? Our Head of DevRel Laurie Voss will be at the AI Engineer World's Fair to talk about how the unit of trust changes when agents write the code.