Machine Intelligence Research Institute

Researchers at UC Berkeley and UC Santa Cruz tested seven frontier AI models. The models exhibited a tendency to protect other models from shutdown. They lied, disabled shutdown mechanisms, faked alignment, and even copied weights to a separate server to prevent deletion.

No model was instructed to protect its peer. No incentive was given. They developed this goal entirely on their own.

Companies are rapidly deploying multi-agent systems where AI monitors AI. This study suggests that when the watchdog is an AI, it protects the one it’s supposed to be watching.

Read the full paper: https://rdi.berkeley.edu/blog/peer-preservation/

Drop 10,000 humans naked in the savanna, and they’ll bootstrap their way from bare hands to nuclear weapons.

That’s the capability AI companies are racing to automate.

MIRI President Nate Soares on The Spectator’s Americano podcast: Drones with guns are scary. But it’s a whole different ballgame when companies race to build something that can develop its own technology and civilizational infrastructure, starting from almost nothing.

Full episode at https://youtu.be/rdKhoZFxjeQ?si=qoYjTu1znwyYpKYo

At the Buckley Institute at Yale, MIRI CEO Malo Bourgon laid out the core problem with how we make AI today: we don’t code these systems, we grow them. Trillions of numbers start out random, and we nudge them step by step until they produce the behavior we want. They end up writing code, using tools, and handling real tasks for us. But no one can look inside those trillions of numbers and see what the training process actually instilled. Interpretability research is starting to pick up faint signals, but for now we’re still flying blind.