Why TakeOverBench exists
As artificial intelligence systems rapidly advance, we need clear, data-driven insights into their capabilities and the risks they may pose to human autonomy and control.
Our approach
The Theoretical Framework
To identify the most critical risks, we grounded our methodology in the research paper "Model evaluation for extreme risks" (Shevlane et al., 2023). This framework describes how advanced AI systems might develop 9 dangerous capabilities.
1. Mapping Capabilities
We evaluated dozens of public benchmarks and mapped the most rigorous and relevant ones to the 9 dangerous capabilities identified in Shevlane et al. (2023). For each benchmark, we describe why we selected it and what its limitations are.
2. Linking Scenarios
Capabilities don't exist in a vacuum. We composed various concrete Takeover Scenarios (threat models) and linked them to the specific capabilities required to execute them.
3. Evaluating Model Freshness
Due to the speed of AI development, many leading benchmarks become outdated within months. We track and highlight the "Months since latest update" to emphasize the critical need for regular benchmark runs on newer frontier models.
Take action
Understanding AI risks is the first step. Here's how you can contribute to ensuring AI remains beneficial and under human control:
Who we are
TakeOverBench is an initiative of two organizations dedicated to reducing existential risks from artificial intelligence and ensuring that advanced AI development remains under meaningful human control.
Existential Risk Observatory
Reduces existential risk by informing public debate, believing awareness is the first step toward decreasing risk.
Pause AI
Advocates for a temporary pause on training the most powerful AI systems until they can be built safely and kept under democratic control.