Potential Risks from Advanced Artificial Intelligence

We aim to support research and strategic work that could reduce risks and improve preparedness.

In recent years, we’ve seen rapid progress in artificial intelligence. And within a decade or two, we think it’s plausible that AI systems will arrive that can outperform humans in nearly all intellectual domains (as do many of the world’s foremost AI experts).

These systems could have enormous benefits, from accelerating scientific progress to vastly increasing global GDP. However, they could also pose severe risks from misuse, accidents, or drastic societal change — with potentially catastrophic effects. We’re interested in supporting technical, strategic, and policy work that could reduce the risk of accidents or help society prepare for major advances in AI.

Funding opportunities and requests for proposals

• RFP on technical AI safety: Support work across 21 research areas to help make AI systems more trustworthy, rule-following, and aligned, even as they become more capable. We’re open to proposals for grants of many sizes and purposes, from rapid funding for API credits to seed funding for new research organizations. Submit an expression of interest by April 15, 2025.
• RFP on improving capability evaluations: Supports work that addresses major challenges to the field of AI capability evaluation by building GCR-relevant benchmarks, advancing the science of evaluations, or improving third-party access infrastructure. The RFP will be open until April 1, 2025, though we may extend the application period.
• RFP on AI governance: Supports work in six areas: technical AI governance, policy development, frontier company policy, international AI governance, law, and strategic analysis and threat modeling. We’re evaluating expressions of interest on a rolling basis.