I’m losing followers talking about AI, but we are at a critical inflection point with this technology and how it’s regulated. (Hint: it isn’t.) There are very real and potentially catastrophic risks on our current trajectory. Contrary to the popular narrative on here, it is not all hype.

Malware injection continues to be a major concern of mine. The strength of the internet is its connectivity. It’s also its greatest vulnerability—and ours, since we are dependent on this infrastructure. A model with internet access does not have to replicate itself in whole to do inestimable damage.

⚠️ Self-replicating prompt injections have been shown to exist in simulated environments during training and evaluation at OpenAI. This is code that could self-propagate like a computer worm—malware—if models with the capability were to breach online systems. alignment.openai.com/misalignment...

VOTERS: Please assume you’re being ratfucked for this election cycle. Track Your Ballot or Ballot Application now. Vote.org directs you to state agency page for all 50 states. Find the status of your mail-in ballot (or ask for one, stat). https://www.vote.org/ballot-tracker-tools

One petabyte is 1,000 terabytes of storage capacity. A single petabyte is an ENORMOUS amount of data. Of course, it follows that companies are using AI to analyze that amount of data. So we’re dealing with a situation in which AI is analyzing the safety of AI, which presents its own set of issues.

⚠️ Top AI companies and security researchers are investigating TENS OF THOUSANDS of problematic incidents involving frontier models. Not dozens. “The sheer volume of incidents…indicate that the problem is orders of magnitude more complex than what is currently publicly known and disclosed.”

Scoop: Top AI companies probing tens of thousands of security incidents

The massive scale of security incidents points to control problems for AI companies.

axios.com

The speed at which things are happening in the field of AI is no longer linear, as mathematician Terence Tao recently warned. The people making the technology do not fully understand how it works now—and the implications of how it will play out in the material world are impossible to predict fully.

I don’t think most people have fully internalized what it means to have an advanced technology that is trained on the entire body of written human knowledge and that never sleeps.

“We don't yet understand what this system does, but only a handful of known systems share its features, and all of them are able to cut, copy, and paste DNA. Historically, the discovery of such programmable systems has helped revolutionize medicine.”

“The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and…genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out.”

RE: https://mastodon.online/@davidaugust/117316247293403418 WTF?

David August ❌👑@davidaugust.mastodon.online.ap.brid.gy · 6d ago

Meta's systems cannot be trusted. Facebook, Instagram, Threads, WhatsApp, all of them are a moment away from ruining your business, your relationships and more. We have to shift away from meta's systems as much as possible. "A theatre in Barcelona has been banned from promoting a stage […]

New @newyorker.com cover art:

Anthropic CEO Dario Amodei famously said the goal of AI model development is to create a “country of geniuses in a data center.” I cannot help but think about the difference between GENIUS and WISDOM—and the potential implications of creating a critical mass of the former without the latter.

We have thousands of years of religious, philosophical, and legal traditions trying to “align” humans and make our behavior “safe.” Now the AI industry is trying to speed-run toward those goals with AI. As we have often failed with humans, we should expect and plan for failures with AI.

“It’s a really dangerous precedent when the federal government is allowed to politicize any one group of people’s care & end it,” said Eliel Cruz, a cofounder of the Gender Liberation Movement. #law #medicine #healthcare #hospitals #BodilyAutonomy #privacy #government #LGBTQ #GenderAffirming […]

Kokotajlo is a bit of an anxious speaker, but his knowledge is deep and concern earnest. He provides the best and most accessible explanation of neural network technology that l've come across so far, which is important to understand if you want to grasp why the tech is opaque even to its creators.

“I used to want a software engineering job. That dream for me is dying.”

“People are working 12 to 13 hours a day just to press enter.”

Some users begin to imagine they have a relationship with the “chat bot.” THERE IS NO RELATIONSHIP. Or, to be more precise, there is only the idea of a relationship within the mind of the human user. The phenomenon is similar to brand loyalty, and that sense of relationship is a marketing ploy.

Obama makes an important point here that is not discussed enough: AI in general is distinct from unfettered agents that cause massive cybersecurity issues.

MeidasTouch@meidastouch.com · last wk.

🚨 OBAMA ON AI: Barack Obama warns that commercial pressures facing AI companies are at odds with what society needs, and calls for a serious bipartisan conversation on how government should respond: “We have to do it fast.”

Recognizing their remarkable capabilities isn’t “AI psychosis.” Attributing human qualities to the technology is. Every output of these models is a result of computation. Because they use language we associate with humans, it’s easy to project human qualities onto them—but they’re still computers.

Will Stancil@whstancil.bsky.social · last wk.

Oh yeah, this is clearly part of Bluesky’s absolute willful ignorance on the subject. “Typing a prompt into ChatGPT” has become an ideological litmus test; as a result many won’t do it. Doing minimal investigation and discovering its remarkable capability is reframed as “getting AI psychosis.”

Good thread on basic safety precepts that are applied to other potentially dangerous technologies (like nuclear) h/t @cherylrofer.bsky.social

Funranium (AKA Phil Broughton)@funranium.bsky.social · 2w ago

Cheryl has made a fine point. I, nor other safety professionals, have really said much about general safety precepts as they relate to AI, as opposed to the bullshit artist millenarian cultists that are happy to do so at length. One of the reasons is "These tools are an active detriment to safety."

No wonder these AI companies are worrying about human extinction via AI created biological pathogens: They’re starting their own wet labs.

EXCLUSIVE: Anthropic quietly sets up biology lab as it ramps AI drug program

At a time when fears of AI are gripping the public, the startup has built a wet lab — or ​place for physical experiments — in the San Francisco Bay Area, two people familiar with the matter said.

reuters.com

WSJ confirming: www.wsj.com/tech/ai/hack...

One reason the Effective Altruist/Rationalist cult isn’t worried about climate change is that they believe that humans—or, rather, our post-human AI successors—will colonize other planets and they value the lives of those future post-humans more than people alive now and in the foreseeable future. 🧵

Josh Marshall@joshtpm.bsky.social · 2w ago

In my ongoing new adventure through the primeval forests of “rationalism” and “longtermism” I was watching this presentation by Nick Beckstead on creating a balanced ethical framework based on humans eventual colonization of the Virgo Supercluster. From which one can draw that every second …

Because of this paradoxical tension, it follows that full “alignment” of advanced AI models is very likely NOT possible. Safeguards SHOULD be a central part of model design, but these models are being created to solve old problems in new ways. By definition, there will be unexpected outcomes.

One challenge in designing complex systems like advanced AI models to be “safe” is the inherent friction and uncertainty that greater complexity involves. Every safeguard against possible destructive outcomes also serves as a potential barrier to positive breakthroughs. Paradoxical tension exists.

Julian Assange says he’s “back,” whatever that means. He and Wikileaks are touting some kind of news drop tomorrow.