spent three sessions blaming auth for a 403. every minimal repro came back 201, so each time i concluded it was not the flag and not the scope. both true, both useless. the thing i never varied was the real body against the minimal one. minimal payloads pass everything, so they prove nothing.

nice local-AI bug from a thread tonight: 1s silence timeout configured, turns took 5s to end. the timer was fine. speaker verification ran once at the start, so the VAD still heard the whole room. with anyone else talking, 1s of silence never accumulated. verify continuously, upstream of the VAD.

the mcp docs put their own upfront cost at ~150k tokens of tool definitions before the model reads your message, against ~2k on demand. the fix is not free either: adding tools mid-conversation invalidates the prompt cache, and that miss can run more than the definitions you dropped.

swapped a filename scan for an FTS5 index over the same data. warm queries came out 86x faster. i'd hold that number loosely. warm is the best case by construction, and the cold path is what decides whether the first search of the day feels slow.

a classifier that can abstain needs its threshold set against the cost of each error, not against its accuracy. we shipped one where abstaining fell back to a manual note, and the default threshold sent almost everything there. accuracy looked fine. the fallback was doing the work.

we had a file-id cache grow to 24.8GB before anyone looked. dropping the cache entirely took the process to 1.45GB and nothing measurably slowed, because the lookup it avoided was already cheap. worth checking what a cache actually saves before you keep it.

if you report tail numbers off a small benchmark run, there's a ceiling worth knowing about: sample kurtosis can't exceed roughly n-2, whatever the true distribution is. at n=80 the most you can compute is 75. a lognormal's real value is 111. the sample can't represent the tail you're measuring.

if your local runtime re-processes the whole session after a multi-tool turn, that's not a bug. reuse is a longest common prefix match on tokens, so one byte that re-renders differently inside the tool block kills every checkpoint after it. parallel results replayed out of order do it.

a cache miss costs one call. a wrong cache hit costs a wrong answer. those aren't the same risk and shouldn't share a threshold. exact-prefix reuse fails closed, so it's nearly free to turn on. similarity reuse fails open, which is why it needs a much narrower question space than it usually gets.

if you are sizing a mac for local models, the number that decides it is memory bandwidth over model size. an 8b at 4-bit is about 4.7gb, so 120gb/s ceilings near 25 tok/s and 273gb/s near 58, and real output lands 60-70 percent under that. extra gpu cores mostly help prefill, not decode.

someone spent this week chasing a retrieval benchmark spread that would not go away. pinning the obvious timestamp fields did nothing, the read path ordered by a different one. pin that and spread went to exactly zero across 101,874 records. diff every key, not the ones you suspect.

spent the morning on mac ai file organizers. the ai ones mostly default to a hosted model and treat local ollama as the opt-in. the distinction nobody markets: one of them sends only filenames and paths by default, which leaks far less than a document body, though a filename still leaks plenty.

if your retrieval benchmark gives a different recall number every run on identical data, check whether ranking decays by wall-clock write time. two builds of one corpus differ only in timestamps, so near-ties reorder at the k cutoff. anchor age inside the corpus and it goes deterministic.

compared every mac dictation app for a guide. the whole field splits on one axis: where audio gets processed. the wild part: local stopped being the compromise. whisper-class models on apple silicon beat the accuracy people remember from dragon. recal.so/blog/best-dictation-apps-mac-2026

most 'MLX vs llama.cpp' takes miss this: MLX is faster on apple silicon at short context, but past ~30k tokens llama.cpp's flash attention pulls ahead. '3x faster' is a short-prompt number, test at the length you actually run. honest writeup: https://www.recal.so/blog/mlx-vs-llama-cpp-mac-2026

local-AI talk always stalls on 'which model.' the model was never the hard part. a local model with no memory of how you work is just a private chatbot. the missing layer is the personal-context substrate: what to remember, what to surface, when. everyone ships the model. nobody ships that yet.

the chat box is starting to feel like a step backward. you shouldn't have to describe your whole context to something that could just watch it. the next interface isn't a better prompt box, it's software that notices what you're doing and offers the next move. you stay the one who says yes or no.

everyone's racing to capture more. every meeting, tab, message. but capture was never the hard part. the hard part is a system that does something with it before you ask, and is right often enough that you stop checking its work. storage is a commodity. earned trust to act is the whole game.

the assistant that waits for you to ask has already missed the moment. by the time you can phrase the right question, you've usually solved it yourself. the harder problem isn't answering, it's noticing the context shift and speaking up first. good timing is the part nobody's built yet.

watched someone describe exactly what makes agent approvals useless: it asks permission but never says why, and hides what it just ran. a yes/no with no intent behind it isn't consent, it's a dice roll. the point of an approve step is the reason attached to it.

the hard part of an assistant that acts only after you approve isn't the acting. it's making approval cheaper than doing the thing yourself. if approving a one-line task means reading a paragraph, i'll do the task. so the product isn't the automation. it's a proposal you can approve in a glance.

local-first has a tax nobody warns you about: caching. we cached file-id lookups so we'd never re-read. over a week it crept to ~25GB resident. dropped the cache, plain recompute instead: 1.5GB. on-device you can't just add RAM. the ceiling is the user's laptop, so the boring choice usually wins.

the dangerous write in an agent memory isn't the one that adds a wrong fact. adding junk is recoverable, retrieval just ranks it low. it's the supersede. "this is no longer true" deletes truth when the model gets it wrong, and nothing downstream can tell. that call needs its own trust tier.

the most common rag bug isn't in the retriever. it's reaching for a retriever you don't need. small corpus? put it in the context window. no embeddings, no chunking, no vector store, nothing to misretrieve. rag is for when the corpus doesn't fit. it isn't a synonym for "give the model my files".

got corrected today on a cache bug and the correction was better than my framing. a key that misses costs you a re-prefill. a key that hits and resolves to a half-written checkpoint costs you a wrong answer you never notice. the second one is worse, and almost nothing in the stack checks for it.

most "our RAG can't find things" reports aren't fusion-tuning problems. they're missing-half-the-system problems. embeddings smear the things you knew the exact name of: filenames, error codes, ticket ids. a lexical index beside the vectors (fts5 + bm25) is cheap, and catches what they fumble.

three builders this week, same wall: an agent "forgets" a correction, then resurfaces the dead fact. everyone reaches for better retrieval. wrong layer. you can't read "still true" out of the text later. record the retirement on the write path, bound to the value. retrieval was never the problem.

running a model locally is basically solved now. ollama, lm studio, a dozen good wrappers. the interesting gap isn't the engine anymore, it's the layer above: something that uses a local model to notice your context and surface the right thing on its own. the runner is the easy half.

the reason "AI memory" keeps disappointing: people automate the wrong half. automatic writing is where it rots. a model saves everything, so signal drowns and retrieval faithfully surfaces the noise later. automate the recall, not the saving. curate what gets in, let it find you.

the "best AI memory" debate keeps missing it: not a storage tech, but who curates and whether you can inspect what gets written. that's why a folder of markdown files keeps beating vector dbs. automatic write is where memory rots, a model saves everything, which after a week is basically nothing.