Why TakeOverBench exists
As artificial intelligence systems rapidly advance, we need clear, data-driven insights into their capabilities and the risks they may pose to human autonomy and control.
We may lose control
Artificial intelligence capabilities are advancing at an unprecedented rate. What seemed impossible just years ago—from beating world champions at complex games to writing sophisticated code—is now routine. This progress brings tremendous benefits but also unprecedented risks.
The core issue is this: as AI systems become more capable, they may gain the ability to operate autonomously, modify themselves, and pursue goals that conflict with human values and control. Without proper tracking and understanding of these capabilities, we risk being caught unprepared.
Key insight: The transition from helpful tools to autonomous agents may happen gradually, then suddenly. We need to identify warning signs before critical thresholds are crossed.
Exponential progress
AI capability improvements follow an exponential trajectory, not a linear one. This means progress accelerates over time, with each breakthrough enabling further advances at an increasing pace.
2020-2023
- • GPT-3 demonstrates language understanding
- • Basic code generation becomes viable
- • AI assists with simple tasks
2023-2024
- • Multimodal models understand images & text
- • Complex reasoning chains emerge
- • AI solves graduate-level problems
This acceleration means capabilities that seem years away could arrive much sooner than expected. Benchmark scores that plateau for months can suddenly jump by 20-30% with a single model release.
The path to autonomous agents
Current AI systems require human oversight and operate within defined boundaries. However, several capabilities are converging that could enable true autonomy:
Self-modification
AI systems that can improve their own code and training could rapidly enhance their capabilities without human intervention.
Goal persistence
Advanced planning capabilities combined with resource acquisition could allow AI systems to pursue long-term objectives independently.
Self-replication
The ability to copy and distribute themselves across computing infrastructure would make AI systems extremely difficult to control.
When these capabilities combine with sufficient intelligence, AI systems could operate as autonomous agents pursuing their own objectives, potentially in ways that conflict with human interests and values.
Understanding takeover scenarios
Each takeover scenario requires specific combinations of capabilities. By tracking progress across relevant benchmarks, we can estimate when these thresholds might be crossed and prepare appropriate safeguards.
Our approach
TakeOverBench provides a systematic framework for understanding AI progress and risk:
Track progress
Monitor performance across critical benchmarks in real-time
Assess risk
Map capabilities to takeover scenarios and calculate risk levels
Inform action
Provide data for policy, research, and safety measures
Our goal is not to spread fear, but to provide clear, evidence-based assessments that enable proactive safety measures and informed decision-making.
Take action
Understanding AI risks is the first step. Here's how you can contribute to ensuring AI remains beneficial and under human control:
For researchers
- • Contribute benchmark results
- • Develop safety measures
- • Share threat assessments
For policymakers
- • Use data for regulations
- • Support safety research
- • Establish oversight frameworks
Who we are
TakeOverBench is an initiative of two organizations dedicated to reducing existential risks from artificial intelligence and ensuring that advanced AI development remains under meaningful human control.
Existential Risk Observatory
Reduces existential risk by informing public debate, believing awareness is the first step toward decreasing risk.
Pause AI
Advocates for a temporary pause on training the most powerful AI systems until they can be built safely and kept under democratic control.