David Manheim
@davidmanheim.alter.org.il
Humanity's future can be amazing - let's make sure it is. Visiting lecturer at the Technion, founder https://alter.org.il, Superforecaster, Pardee RAND graduate.
I was really happy to appear on the @futureoflife.org podcast this week to talk about @evals-consensus.ai and the challenges of evaluations of AI systems, along with discussions of Goodhart's law, AIxBiosecurity, AI persuasion, forecasting, and human oversight of AI.
Why AI Evaluations Are Broken and How to Fix Them (with David Manheim)
YouTube video by Future of Life Institute
youtu.be
Over on the bird site, Anthropic employees are using their slop machine that 'can't think' to solve longstanding mathematical open questions which have eluded humans for close to a century.
I keep seeing claims AI companies will get rich, or won't make any money, or that open source models will be cheaper and undercut prices, and similar claims that ignore the simple fact that the economic pricing power is all about compute and token generation costs. davidmanheim.com/AI-Economics/
The Future Economics of LLMs — Interactive Companion
An interactive companion to The Future Economics of LLMs.
davidmanheim.com
"Technologies that have first order impacts on coordination and production, or that empower groups in other ways, tend to differentially benefit the powerful in ways that are harmful to others, either directly or indirectly"
David Manheim@davidmanheim.alter.org.il · 3mo agoNew post: Huge technological revolutions aren't usually positive for those living through them - and this bodes poorly for AI even if it is a normal technology. davidmanheim.substack.com/p/if-ai-is-n...
New post: Huge technological revolutions aren't usually positive for those living through them - and this bodes poorly for AI even if it is a normal technology. davidmanheim.substack.com/p/if-ai-is-n...
If AI is normal technology, history is not reassuring.
Technological revolutions turn out well eventually, but they go badly first.
davidmanheim.substack.com
As a forecaster, strongly disagree with @kulveit.bsky.social on this. 🧵 "Massive change" is pushing narratives over base rates and trends; in the past 20 years, inequality increased in developed countries. Predictions defying trends based on "this changes everything" are a common error.
GPT-5.5 seems to do more 'hacking per dollar' ...Mythos does more 'hacking per token'" - @peterwildeford.bsky.social Thinking about risks from bad actor's access to scaled models, the costs involved here are still absolutely tiny compared to other attack modes.
It's hard to avoid the conclusion that Bluesky has been a net negative for US politics. They corralled everyone on the left into a little glass fishbowl where they shout at one another & everyone else ignores them. Meanwhile, all the pols & institutions stayed on X & are being dragged farther right.
The cognitive costs of choosing between different LLM versions or subagents for different tasks, picking or managing the thinking levels, whether to use /fast, and managing token budgets is an amazing illustration of why firms bundling labor into employee salaries is so much more efficient.
I think it's entirely appropriate that the people calling AI a tool are also people who would be insulted if we called them a bunch of tools.
Great new piece by a bipartisan team of Ben Buchanan (Biden admin White House Special Advisor for AI) and @deanwb.bsky.social (former Trump admin WH OSTP Senior advisor) saying that AI has national security implications which deserve, but aren't getting, a careful and bipartisan government response.
Opinion | A.I. Is a National Security Risk. We Aren’t Doing Nearly Enough.
nytimes.com
@ioda.live Question: Is February 2021 Texas historical data available? (We keep getting empty responses for historical US bgp data from the API.)
The proportion of Philosophy articles I'm asked to review that have undeclared LLM writing is too damn high! (I think LLM usage for writing is often fine, even / especially if the writing itself is about LLM's ability to reason. But it's supposed to be disclosed, so disclose it!)
"Chinese companies cannot legally fire employees simply to replace them with cost-saving artificial intelligence, courts in the country have ruled, setting a significant precedent for labor rights as automation sweeps the tech sector." There goes the "China is trying to win the AI race" narrative.
Chinese Courts Rule Companies Cannot Fire Workers Simply to Replace Them With AI
Judges classify AI adoption as a controllable business strategy rather than an unavoidable disruption, shielding employees from automation-driven layoffs
caixinglobal.com
We’re raising funds to print a brand new book of compiled SMBC comics on the topic of parenting, alongside our preorder for Sawyer Lee! Check out the project here : www.kickstarter.com/projects/wei...
Hot take from @henryshevlin.bsky.social from over on the bird site, about comparisons of LLMs and humans: "Not a fan of these clichéd “we used to think the mind was clockwork” analogies. Sometimes science just makes progress... Some mechanistic explanations were wrong; others are just true."
It's not just in the prompt, it's there twice! (Mistake, or necessary overkill? Who knows!) bsky.app/profile/emol... github.com/openai/codex...
Ethan Mollick@emollick.bsky.social · 3mo agoThis is an actual line that was added to the official system prompt for Codex for GPT-5.5 by OpenAI. Usually the system prompt is as minimal as possible, so I assume it would otherwise mention goblins a lot. AIs are weird.
"Draw me a highly detailed where’s Waldo image with people or items to find, but of an ISO SC42 standards conference. Make sure to make it funny with in-crowd jokes."
One problem with making predictions in public is that when I say I'm 75% sure of something, and someone else responds that they are 99% sure I'm wrong, they are using numbers as rhetoric, and I'm trying to make sure that I'd be right approximately 3 out of 4 times.
This is the whole alignment problem. All of it, encapsulated. Low-probability behaviors by the model, just make your training environment=your test environment, "what's a capability vs. a propensity", non-adversarial generalization, Goodhart's law. The whole damn thing
LLMs are missing an operating system! Great post by William Waites on the SoTA (Society for Technological Advancement) blog, laying out the argument for what the early history of computing tells us about current and future AI system design. sotaletters.substack.com/p/the-telety...
The Teletype of the Future
A brief history of memory: Training data is ROM, the context window is RAM, and tool-accessible storage is disk. What about the OS?
sotaletters.substack.com
Comment from a math professor on the quality of the latest proofs.
Claude refuses to help invent conspiracy theories. Then, after calling Claude a "glorified and electrified rock," the user complains that it's "being emotionally manipulative" and then claims that LLMs will be "used to fine tune human thought as it globalizes our will and ideals."
Sam Altman said he regrets calling the New Yorker profile "incendiary," but never edited his blog post. Now the SF DA is using the same term in a call for deescalation that implicitly frames journalism as a public safety threat. x.com/GerritD/sta...
Slightly contra @davidskrueger.bsky.social's post on the relative merits of AI treaties versus AI regulation, I argue different tools we have should be complementary, and any fights should be about details and different goals, not approaches. Also on LW here: www.lesswrong.com/posts/7CLL4K...
Treaties, Regulations, and Research can be Complements
Slightly contra David Kreuger
substack.com
"Every member of the TESCREAL movement accepts a posthuman eschatology, according to which we should introduce one or more new posthuman species, hopefully in the near future." Did Emile just try to kick me out of the (imagined) TESCREAL community because I don't pass his made-up purity tests?
Dr. Émile P. Torres@xriskology.bsky.social · 4mo agoNew article on how TESCREAL doomers like Yudkowsky offer us two options for the future: human extinction or, alternatively, human extinction. In both cases, our species will almost certainly die out. This is why you shouldn't lock arms with these people, if you're on Team Human.
