It Takes a Village to Survive the Future

your Saturday roundup

Professionally, I focus on creating social benefit startups. In my Saturday morning emails I share what I’m learning and thinking. Topics range from better living and parenting to business and philosophy.

1. Superintelligence Strategy. I’ve previously written about AI 2027, Situational Awareness, Machines of Loving Grace and The Techno-Optimist Manifesto. Somehow I missed the Superintelligence Strategy, a whitepaper that stands out as the most geopolitically informed and arguably the most grounded in game theory. Figure 6.3 (below) offers a concise overview of competing approaches to AI governance. Situational Awareness argues for a High Control model, advocating a nationalized Manhattan Project for intelligence. AI 2027 outlines a more optimistic path: cooperative duopoly through treaty, placing it on the high end of Medium Control. A few key takeaways from The Superintelligence Strategy:

   • AI systems may inherently favor offense at least in cybersecurity. Do you agree?

   • They introduce the concept of Mutually Assured AI malfunction. Instead of the Mutually Assured Destruction of the Cold War, it suggests that as large AI projects get closer to superintelligence, adversaries will attack and slow down progress.

   • They propose treating AI chips like fissile material. An interesting analogy that raises questions about edge computing and open-source models that can run on personal hardware. They don’t fully account for open-weight models or the diminishing role of compute as a limiting factor. The human brain, after all, is a powerful reminder that intelligence doesn’t require vast computational resources. Control over hardware might delay risk, but it won’t eliminate it.

   • They float the idea of developers building in or disclosing backdoors in model weights as a form of deterrence to stealing them. Do you think that is a good idea?

   • What also seems absent in their analysis is a position like Reid Hoffman’s SuperAgency, which emphasizes iterative deployment and adaptive regulation. His argument: as we use AI, we’ll better understand the kinds of regulation we actually need. Like past technological shifts, initial fears may give way to beneficial adaptation. It’s a more bottom-up, pro-agency approach. It sits in the center of the control spectrum, though with less attention to national security dynamics.

   • How else might prepare for superintelligence? Will MacAskill et al have some thoughts. Or start with the AI safety basics.

     

2. Malthusian Swerve. I listened to RadioLab’s Malthusian Swerve with our kids, and it sparked a conversation about capitalism’s relentless drive for growth. Does that mean we’re using up our resources and heading toward scarcity? For some things. But price signals matter as they push innovation forward and help us put the effort towards inventing replacements. Government can help by supporting knowledge creation, though banning the use of a resource usually isn’t the best route. A better role for the government is making sure prices reflect what we truly value. There’s no guarantee we’ll keep dodging catastrophe, but the best tools I know for doing so are science, invention, and capitalism working together. Do you know of better ones? If you convince me, awesome. If not, we could bet about it.

3. Unhappy Apps. Which smartphone apps do people link most with happiness or regret after the fact? Are we using certain apps because we’re already feeling low, or do they actually make things worse? Maybe we just feel bad about using them later because they’re not exactly socially celebrated. Or maybe we turn to them when we don’t have anything more meaningful going on. Whatever the case, I’d already started avoiding the apps topping the “most unhappy” list before I even saw the Center for Human Technology’s survey results.

Until next time,

Miles

P.S. Good news. There is a newish drug to prevent HIV transmission.