New podcast episode: @simondgoldstein and @petersalib on why liberal institutions may be hard to beat, even after AGI. Podcast apps: t.co/kLG1UsA9XT YouTube: www.youtube.com/watch?v=PVi...

Liberalism Forever – Peter Salib & Simon Goldstein | ForeCast

Peter Salib and Simon Goldstein return to discuss their new paper "...

youtube.com

Governments are going to deploy frontier AI in areas where existing checks on government power are already weak — the military, intelligence, surveillance, policing.

How much would automating AI R&D accelerate AI software progress, even in the absence of a software intelligence explosion? We've created a tool to calculate this speed up, with adjustable inputs so you can test your own scenarios. Try it here! newsletter.forethought.org/p/speed-up-...

Speed-up calculator: How much will automating AI R&D speed up AI software progress, absent an software intelligence explosion?

A tool for calculating how much automating AI R&D will speed up AI progress, even if there’s no software intelligence explosion.

newsletter.forethought.org

Claude Fable’s deliberately triggered sandbagging shows that training-time targets are, by themselves, insufficient to guarantee particular LLM behaviors.

New post: @Benthamsbulldog considers whether it will be possible to get AI that is highly competent at philosophy, such that we could trust its answers to philosophical questions in domains that aren't empirically verifiable. Read it here: newsletter.forethought.org/p/can-ai-do...

Can AI do philosophy?

A guest post by Bentham’s Bulldog, created while they were a visiting scholar at Forethought.

newsletter.forethought.org

New podcast episode: Avi Parrack and Tom Davidson discuss the plausibility of AI data centers being built in space. newsletter.forethought.org/p/will-we-p...

Will We Put Data Centers In Space?

A podcast episode from Forethought

newsletter.forethought.org

(1/2) A new blog post argues that eventually handing off high-stakes decisions about the future to philosophically competent, reflective AIs will result in much better outcomes than locking in current human values, retaining human control,

A new report argues that training AIs to be risk-averse – to treat resources as having diminishing marginal utility – could both preserve AIs’ usefulness (if they turn out aligned) and provide an extra line of defense (if they turn out misaligned).

New podcast episode: Wei Dai on the importance of making sure AI is philosophically competent. newsletter.forethought.org/p/could-ai-...

Could AI Help Solve Philosophy?

A podcast conversation with Wei Dai

newsletter.forethought.org

AI companies face a tangle of competing considerations when deciding what goes into a model spec.

New post: Will we really put data centers in space? Read it here: www.forethought.org/research/wi...

Will We Really Put Data Centers in Space?

How soon could AI data centers move to orbit? Forethought analyzes launch costs, cooling physics, and other engineering and governance factors needed to make space compute viable.

forethought.org

AI model specs are usually aimed at shaping the behaviors of present and near-future models. But what if current model behaviors transfer into future models by default?

For humans and advanced AI systems to be able to make honest deals and avoid negative-sum conflict, AIs will need reasons to trust us. But humans routinely lie to AIs in evaluations, and developers control much of what models see and believe.

As AI gets smarter, people will increasingly turn to it for advice on important decisions, so the quality of AI advice really matters.

How is moral diversity valuable for achieving a near-best future? A new post introduces several models for thinking about the value of moral diversity as the number of powerholders scales. Read it here: newsletter.forethought.org/p/the-value...

The value of moral diversity

Several models for thinking about the value of moral diversity as the number of powerholders scales.

newsletter.forethought.org

AI could dramatically transform how we collectively determine what's true—for better or worse. In a new post, the authors map out the possible impacts of AI on society's epistemics: the good, the bad and the ugly.

Near-term AI could make it dramatically easier for groups to find deals, resolve disputes, and hold each other accountable. But the same tools could enable collusion and worse.

New post: AIs should (sometimes) be proactively prosocial. Read it here: www.forethought.org/research/ai... x.com/willmacaski...

AIs Should Have Proactive Prosocial Drives

Forethought argues that AIs should (sometimes) take proactive actions to benefit society, not just follow instructions.

forethought.org

What if... we could use AI to help build the kind of AI that would empower us to work out what's true? Introducing: AI for AI for epistemics. www.forethought.org/research/ai...

AI for AI for Epistemics

AI could help us to build stronger AI-powered systems to help people track what is true. This brings important opportunities and risks.

forethought.org

New post: concrete projects to prepare for superintelligence. Read it here: www.forethought.org/research/co... x.com/willmacaski...

Concrete Projects in AGI Preparedness

Eight concrete projects to help prepare for superintelligence, including AI character evaluation, automated macrostrategy, and tools for improving epistemics.

forethought.org

New post: William MacAskill and Tom Davidson argue that AI character is a big deal. Read it here: www.forethought.org/research/the...

The importance of AI character

Forethought argues that AI character—e.g. how obedient, honest, or altruistic AI systems are—will shape power, conflict, and society far more than is recognized. Work to shape AI character could be hu...

forethought.org

New post: should we lock in post-AGI agreements under uncertainty? Read it here: www.forethought.org/research/sh...

Should We Lock in Post-AGI Agreements Under Uncertainty?

Some mutually beneficial agreements, between major powers or individuals, depend on shared uncertainty about post-AGI outcomes. We consider which deals are worth enabling before an intelligence explosion.

forethought.org

"Moral public goods" are things many people value for ethical reasons, but where no individual's contribution is worth it unless others contribute too, creating large potential gains from coordination.

New podcast episode: chatting with economist Sam Hammond about what happens to public institutions when AI collapses transaction costs. www.youtube.com/watch?v=grG...

Can Liberal Democracy Survive AGI? — Sam Hammond | ForeCast

Sam Hammond and Fin Moorhouse discuss discuss how AGI could reshape the nation-state, drawing on Sam's “AI and Leviathan” essay series.Read a transcript of t...

youtube.com

Tools for strategic awareness could deepen people’s understanding of what’s actually going on around them, making it easier for them to make good decisions in their own interests. This would have big implications both for individuals and for collective decision-making.

We’ve recently published two pieces of background research that informed our thinking on an international AGI project: • An overview of some international organisations, with their voting structures • The UN Charter: a case study in international governance

Technologies powered by near-term AI systems could transform our ability to reason and coordinate, significantly improving our chances of safely navigating the transition to advanced AI. Last week, we launched series of design sketches for specific technologies that we think could help.

Imagine having a technological analogue to an ‘angels-on-the-shoulder’: a customised tool or tools that help you make better decisions in real time, decisions that you more deeply endorse after the fact.

Should we focus on worlds where AGI comes in the next few years? People often argue yes, because short timelines have higher leverage. We're not so sure. New post arguing that for many people, 2035+ timelines might be highest leverage: www.forethought.org/research/sh...

Short AI Timelines Aren’t Always Higher-Leverage

Are 2–10 year AGI timelines really highest leverage? We compare 2027/2035/2045 scenarios and explain when medium timelines can offer higher-impact.

forethought.org

Wondering whether to apply to our open roles? Research Fellow Mia Taylor joined 5 weeks ago. We just released a new episode of ForeCast, hearing from her about why she joined, what it's like to work here, and who the work is likely (and unlikely) to suit. pnc.st/s/forecast/...

Forethought is Hiring Researchers (with Mia Taylor)

This is a bonus episode to say that Forethought is hiring researchers. After an overview of the roles, we hear from Research Fellow Mia Taylor about working at Forethought. The application deadline has been extended to November 1st 2025. Apply here: fore

pnc.st