Dave Kasten

@davekasten.bsky.social

Do what seems cool next. "You need to learn WHY things work on a starship."-Admiral James T. Kirk

1. RM is of course entirely right. Sulla wiped out his enemies and totally remade all of Rome’s institutions, and it *didn’t work* 2. I think it’s also extremely important to understand *why* it didn’t work, as Ronald Syme memorably put it…

Bild
RoninMacbeth 🇺🇦🇵🇸🇻🇪@roninmacbeth.bsky.social · 21h ago

Sulla killed shitloads of his enemies, practically drowned the Roman Republic in blood to wipe out the Marians and their allies. But within ten years of his death his constitutional order had collapsed, because you actually can't just proscribe your way out of politics.

Yeah OpenAI or Anthropic or whoever might go bankrupt, some specific use cases like ‘therapists’ might be litigated to oblivion. But things that involve interfacing with code are never, ever going back to the modal default being a human being typing things out, like it’s just full stop not happening

The Great Pumpkin Papers 🎃🎃🎃@professormusgrave.bsky.social · 2w ago

AI might be a bubble, but is it a tulip bubble or a railroad bubble

the thing is that if you are working in a safety-critical area of the company, you need to understand that (a) your job is to get to zero deaths, and (b) you will not do your job because at any scale you will be trying to guess every 1e-7 event, and you won't. your boss cannot say that is okay.

e.w. niedermeyer@niedermeyer.online · last wk.

I don't know who needs to hear this, but the autonomous vehicle developers who most vocally argued that society must accept a certain number of deaths to enjoy the technology's safety benefits (Tesla & Uber ATG) were the only ones who killed people and never actually delivered on their promises

I don't know who needs to hear this, but the autonomous vehicle developers who most vocally argued that society must accept a certain number of deaths to enjoy the technology's safety benefits (Tesla & Uber ATG) were the only ones who killed people and never actually delivered on their promises

or if it is parroting, it's stretching the bounds of your conventional understanding of that term so far as to render it misleading. parrots cannot solve complex problems, self-interrogate and self-correct, search the web and extract relevant information, etc. etc.

feels like there is a group of smart and well-intentioned people who long ago made up their minds that AI Is Bad, swore off using it, and now get all their info about AI from each other. not saying they're wrong that ai is bad! but i think they're gradually losing touch with what it is and does

Colin@colin-fraser.net · 2w ago

I really wasn’t trying to start a fight with Prof. Bender but now that she has created a long thread subtweeting me and her disciples are all yelling at me, let me just say that this is the crux right here. She seems to respond as though I am describing something a priori impossible.

this is exactly my opinion. there may be a superstrate of extremely difficult and unique safety problems around AI, but the substrate is simply that no one takes basic precautions seriously becasue they imagine (or pretend to imaigne) that it is futile, and basic precautions slow down launches.

Michael Caley@michaelcaley.bsky.social · last wk.

this is by far the best "I quit my AI job over safety concerns" essay yet because it is the first one that is able to talk about safety concerns in practical terms and that understands "safety" as a topic with extensive literature in analogous fields www.theatlantic.com/technology/2...

Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking.

Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster. Right now, AI companies don’t know how—but other people do. In a nuclear-power plant, the technical systems and rules people follow are set up so that, if equipment breaks or someone pushes the wrong button, we still won’t risk a meltdown. OpenAI and other labs are growing and deploying frontier AI with far less redundancy and rigor than this, even though the harm from an irreversible loss of control would be much greater than the harm from any single meltdown. Even short of a full loss of control, we could see autonomous swarms of AI agents that act without human permission. Imagine “rogue” agents that work like teams of hackers (for example, holding hospital computer systems for ransom) but never need to sleep.

I am a cynic and think they know, they're just being intellectually dishonest. Like when Emily Bender insists that math is being done by a Secret Calculator in Chatgpt and suspiciously doesn't address questions about open models - she knows the implications of the current capabilities.

tweety fish oh wow like a chimera or something spoooky@sifu.tweety.fish · 2w ago

I am sort of constantly baffled by critics (which I am, more than not!) who don't feel like they have to stay apprised of what these things can actually do. not least you're going to come off like a fool, but also it's not at all like knowing what they can do obviates the criticisms of them?

Substantial portions of conventional charity and philanthropy are extremely adverse to any kind of effort to impose rigor or ask in any way “is this working at all”. It’s not a strawman that they’re critiquing, at least in my personal experience.

Earn-to-give was definitely one of the more obviously repugnant conclusions to come from people who should have known better than to fall into deontological tar pits, and yet it wasn't the last by a long shot.

yes, i am saying this. literally this. i am saying that you should not care about water for 16,000 people when "growing alfalfa for export in deserts with approximately zero precipitation" is literally orders of magnitude more water and vastly more pointless.

A New Reconstruction@burleyzack.bsky.social · 2w ago

40m people rely on the Colorado River, which year after year shows less water in it. Denver Water has declared drought conditions and restricts residential use. You’re saying we shouldn’t care about the new water use equal to 16000 people? www.governing.com/finance/denv...

This is an unpopular take but as a neurodivergent self-employed person, LLM's have been an unbelievably helpful tool for the things I have struggled with my whole adult life: managing my finances, filling out paperwork, and staying on top of my email. I'm functional in a way I never dreamed of.

Just a seamless pivot from “these people are selling something that is useless and can’t do anything” to “here’s how to hold these people who sold this harmful thing” accountable.

Occasionally I see people on here opining on the obvious best ways to handle AI hacking things who previously, and to much acclaim, were posting about how AI was a useless tool sold by charlatans and couldn’t do anything better than a human and would never have any practical applications whatsoever.

I’ll say it; I think this is bad. I think there’s growing pressure to talk about AI in this kind of coded language that destroys understanding, rather than facilitating it. People want you to recite a list of its supposed weaknesses as a reassurance for themselves, not to inform others.

Bild

I think a lot* of people don't realize that they actually do still believe in something like a soul, but until they clock that in themselves, they get kind of confused when talking with someone who doesn't believe in souls at all *large N, not necessarily large percentage

If a person is accurately absorbing new information, solving unsolved problems of incredible difficulty, engaging in detailed back-and-forths, but is not “understanding,” the word “understanding” has no meaning. pretty sure this version would be less controversial despite saying the same thing

Will Stancil@whstancil.bsky.social · 3w ago

Again: what we’re doing here is defining “understanding” as “the thing that humans do.” If a model is accurately absorbing new information, solving unsolved problems of incredible difficulty, engaging in detailed back-and-forths, but is not “understanding,” the word “understanding” has no meaning.

Again: what we’re doing here is defining “understanding” as “the thing that humans do.” If a model is accurately absorbing new information, solving unsolved problems of incredible difficulty, engaging in detailed back-and-forths, but is not “understanding,” the word “understanding” has no meaning.

Ron Belmont@ronbelmont.bsky.social · 3w ago

In any case, your original claim that what an LLM does "requires comprehensive understanding" is demonstrably false because an LLM does not and cannot understand anything.