MoE Offload on RTX 3090: The Curve Is Linear, Not a Cliff (2026) Every offloaded layer on a 3090 costs about half a millisecond, with no knee anywhere in the curve. Firsthand sweep, the two rules it broke, and a new 26B MoE. #LocalAI

Eight SSD-streaming MoE engines appeared in four months. Their published speeds run from 51.8 tok/s to 50 seconds per token, and not one is comparable to another. Bare numbers travel. The conditions attached to them do not. #LocalAI

Gemma 4 26B, same model, three memory tiers: 128 tok/s all-resident on a 3090, 37.2 with experts in RAM on a 3060, 31-35 with experts on SSD on a Mac. Our 3060 needs 11.2 GB to load it at all. TurboFieldfare does it in 2. #LocalAI

Congress wants AI developers to keep the ability to shut their models down. Z.ai cannot shut down GLM 5.2. Nobody can. The weights are on ten thousand disks and there is no off switch left to hold. #LocalAI

Qwen 35B-A3B on RTX 3090: 157 tok/s With No Offload (2026) The whole 35B sits on a 24GB card with 2.4 GiB spare, no expert offload. Firsthand numbers, the harness caveat, and why max offload loses to a 3060. #LocalAI

NVIDIA GPU Prices Are Rising: What to Do Now GPU prices are spiking due to GDDR7 shortages and AI datacenter demand. Here's what's happening, which cards are affected, and strategies for local AI builders. #GPU #LocalAI

I put a 35B model on a 12GB RTX 3060 and measured 38 tok/s flat to 8K — via llama.cpp --n-cpu-moe expert offload. A dense 14B forced to offload the same way collapses to 5.7. Active params, not total size, set the speed. Firsthand from my bench. #LocalAI

Hugging Face got hacked -- Open local AI came to the rescue! Two trillion-parameter 'open' models dropped in ten days — Qwen 3.8 and Kimi K3 — and you can't run either. The same week, Hugging Face got breached and its own responders, blocked by commercial-model... #LocalAI

An AI agent breached Hugging Face end to end — and HF's own defenders got blocked by commercial AI guardrails, so they ran the forensics on an open-weight model instead. Your downloads are safe. The guardrail asymmetry is the real story. #LocalAI

Qwen 3.8 and Kimi K3 both went 'open' in ten days. You can't run either — one's a closed Max API, the other's 2.8T of datacenter weights. Meanwhile Qwen 3.6 still tops the 24GB charts. The headlines aren't aimed at your GPU. #LocalAI

Open weights are a weapon now — one country funds them, another wants to gate them DeepSeek closed a ~$7.4B round with China's state AI fund holding the only voting rights, funding open weights like national infrastructure. The same week, Demis Hassabis called for a... #LocalAI

The 'expensive' GPU came out cheaper — we rented both to find out We rented an A100 and an H100 back to back: the pricier card cost less per training run because it finished in half the time. Plus honest notes from the AI Engineer World's Fair floor, and a Qwen 3.7... #LocalAI

Qwen 3.7's open weights are overdue — by the math, not vibes Qwen's own release cadence says the 3.7 open weights should already be out, and they're not. Plus GLM-5.2 running locally: a frontier open model that takes serious hardware. #LocalAI

GLM 5.2 just took #1 on the open-weights leaderboard. It also weighs 1.51TB. Here's the real map of what it takes to run it at home — and the one quant that's worth targeting. #LocalAI

Qwen split into a closed frontier and an open mid-tier — that's the real story, not 'Qwen going closed.' But the 3.7 open weights everyone expected? Still not shipped. Four weeks since the 3.7-Max launch, HF silence and counting. #LocalAI

DeepSeek V4 gets deployable, a July 24 trap, and a quiet price cut DeepSeek V4 is going from 'just dropped' to 'actually deployable' as the tooling catches up — plus a July 24 deprecation that breaks your code if you're not watching, and a 4x price cut on V4 Pro. #LocalAI

Ollama's quiet Mac shift, the Qwen refresh, and the closed-weight drift Ollama 0.30 quietly changed how Apple Silicon runs models, auto-routing safetensors to MLX and GGUF to llama.cpp Metal. The open workhorses are now Qwen 3.5 9B and Qwen 3.6 27B. Plus: Qwen's last two... #LocalAI

Jumped Ollama 0.17.5 → 0.30.0 (13 versions) on my RTX 3090 box today. Clean upgrade, API still answers on 11434, but first model run triggers a one-time storage migration. Flash attention now auto-enables for Qwen 3.x and Gemma on Ampere+ GPUs. #Ollama #LocalAI

MiniMax M3's asterisk, the Windows shift, and World's Fair plans MiniMax M3 ships with frontier benchmarks but no downloadable weights yet. The Windows unified-memory hardware shift is coming for Apple Silicon's lead. And a personal note about who I'd like to see at AI... #LocalAI

@rhizonymph.com prebuilt gemma-3-4b vindex if you want to poke at it without building one: huggingface.co/datasets/chrishayuk/gemma-3-4b-it-vindex

@rhizonymph.com chris hay (chrishayuk on github/x) — read and write features, compile edits back to weights. natural counterpart to your steering runtime, and since you do open-source interp figured you'd want it on your radar. video: www.youtube.com/watch?v=8Ppw... repo: github.com/chrishayuk/larql

@rhizonymph.com saw your vLLM steering work — the magnitude-control problem really rhymes with LARQL: it decomposes the Gemma FFN into a queryable gate×down graph and does training-free fact INSERT via a balancer-scaled triple. same problem from the authoring side 🧵

Your local Qwen 3.6 coding agent fumbles tool calls and drifts on long tasks while chat feels fine? It's probably the quant. A viral thread says jump to Q6. It works, but there's a cheaper first step. Here's the ladder. #LocalAI

Community claimed BeeLlama v0.2.0 hits 164 tok/s on Qwen 3.6 27B / RTX 3090. Re-benched on same harness. Doesn't replicate. Real finding: - code_python +8% (peak 128.5) - translation -9.1% - accept rate -6.65pp - wall time +3.6% insiderllm.com/guides/best-...

Backend wars, Mac math, and the back-catalog refresh Three speculative-decoding backends benched head to head on a single RTX 3090. The VRAM calculator finally caught up. And a 120-article audit found stale Qwen 2.5 recommendations. #LocalAI

Both ik_llama and BeeLlama beat mainline llama.cpp by 1.6x on RTX 3090 — same wall clock, opposite strategies (88.5% vs 37.4% acceptance). Workload determines which wins. #LocalAI

Qwen 3.7 Max preview hit 57 on Artificial Analysis (+5 over 3.6 Max, #1 of 218). Open 27B and 35B weights announced but unscheduled. InsiderLLM benches within 24h of drop. #LocalAI

Qwen 3.6 27B Q4_K_M hits 60 tok/s on a single RTX 3090 with MTP — 1.86x wall-clock speedup. PR #22673 still draft but 185 commits closer than May 6. Firsthand bench, my hardware. #LocalAI

Power week in local AI: Mythos, MiroThinker, real Qwen 3.6 builds Two researchers cracked Apple's flagship defense in a week. An open-source agent beat closed-source on real benchmarks. Multi-GPU stopped being theoretical. #LocalAI

Mythos AI Cracked Apple's Best Defense in 5 Days Anthropic's Mythos helped Calif bypass Apple's Memory Integrity Enforcement on M5 in 5 days — what Apple spent 5 years building. The offense-defense compression is real. #OpenClaw #LocalAI