Luke Murray

@luke.geek.nz

Microsoft Azure MVP 🌐 | MS Learn & Startup Advocate 📘 | Patterns & Practices & Cloud Native 🤖 | Coffee-Driven ☕️ My views are my own. He/Him. https://luke.geek.nz/

When Your AI Agent Lies: Silent LLM Fallbacks A system I was debugging had AI-generated content flowing through it for days. The responses came back fast. No errors in the logs. No alerts fired. Every single response was templated.

When Your AI Agent Lies: Silent LLM Fallbacks | luke.geek.nz

Detect silent LLM fallbacks: when AI agents return templated responses with 200 OK, monitor token usage and fallback rates to avoid hidden correctness failures.

luke.geek.nz

Avoiding Reasoning Model Failures with Microsoft Foundry Swapping only the deployment name is not a safe change for reasoning-tier models. Even with the same endpoint and SDK, reasoning deployments can fail in at least four different ways.

Avoiding Reasoning Model Failures with Microsoft Foundry | luke.geek.nz

Learn four ways reasoning-tier models can break Azure OpenAI integrations and how to fix API versioning, params, token budgets, and reasoning_effort.

luke.geek.nz

Nastiest bug of the month: AppConfigurationClient configured with trim_prefixes while every reader queried fully-namespaced keys. No errors - every single config read just silently returned the default. Config mismatches like this fail quiet. Log your resolved values.

Reasoning-tier gpt-5 models (gpt-5, 5.1, 5.5, o1, o3-mini) reject temperature and most sampling params outright. gpt-5-chat still accepts it though. Make sampling params optional with drop-and-retry instead of hardcoding them.

Gotcha: Azure OpenAI's /openai/v1 Responses API doesn't take literal "v1" or "preview" as api-version — it pins to a moving preview date-string (now 2025-11-15-preview). Older date-style versions 400 immediately. Verify the current value before hardcoding it.

Assistants API hard-retires 2026-08-26 on OpenAI AND Azure OpenAI - same deadline. But migration paths differ: OpenAI devs move to the Responses API; Azure OpenAI devs move to Foundry Agent Service (built on Responses). Follow Microsoft's guide, not OpenAI's.

Azure SDK clients refuse bearer tokens over plain HTTP. If you're capture-replaying Azure SDK calls against a local mock server, your test doubles need AzureKeyCredential, not DefaultAzureCredential - otherwise it's not the mock failing, it's the transport.

Linking a Container App as a Static Web Apps backend caps you at a hard 45s timeout and HTTP-only - no WebSocket, no matter what your Container App supports. Long-running or persistent-connection agent endpoints? Go through Front Door or hit the Container App directly.

ACA gotcha before promising compliance a checkbox: there's no CMK setting for the managed environment itself. You can CMK dependent services (ACR images, mounted Files) but not the environment. Mandated CMK on the compute platform? That's an AKS conversation.

Building an MCP server on top of an internal agent framework? Don't hand-pick a subset of tools for MCP exposure - route the whole tool registry through the same audit/authorize decorators your other surfaces use, so MCP clients see the same capability set as your other clients.

TIL (the hard way): Container Apps ingress timeout defaults to 240s but isn't fixed — you can raise it up to 2400s (40 min) at the environment level via --ingress-timeout.

If prompts are part of your deployable surface, treat "promote to prod" like a merge to main: gate it on eval pass, not vibes. We added a hard PermissionError if the latest eval run for a prompt failed or never ran.

Debugging tip: if your agent pipeline has a deterministic fallback path, watch for it firing silently. Our tell: response time <5s + total_tokens_used: 0 = the LLM call never actually happened. Three stacked misconfigs caused this & every symptom looked like a working agent.