Executive Summary
A field guide, in brief.
AI both threatens and promises to reshape our society. To ensure it goes well, we’ll need to preserve checks and balances on powerful institutions, including frontier AI companies and national governments, even as the capabilities of those institutions change rapidly.
This is a difficult problem. We think that a solution includes:
- Establishing government processes for overseeing AI companies, without providing arbitrary power over the industry.
- Modernizing legislatures and the courts so they can check the executive, without causing procedural gridlock.
- Empowering citizens politically, economically, and technologically to ensure that government institutions remain aligned to the public interest, despite widening power differentials.
- Supporting countries around the world, especially democracies, to maintain a stake in a future with widespread automation.
- Increasing the attack surface area of all of these solutions: breaking them down into specific projects, articulating feasible and desirable goals, and bringing them into contact with relevant experts.
We’re supporting projects to achieve all of the above.
Areas of work
Some areas we’re especially excited for.
Checking government in the AI era.
Lawfare and LawAI’s recent agenda on AI and the rule of law highlights many of the same problems we’re concerned about, and outlines a number of worthwhile areas for legal research and policy design. For example, how should AI be overseen and restricted in national security contexts, where it already serves as a valuable and increasingly autonomous tool? No one organization can answer even a fraction of these questions. We’re eager to support more experts capable of reasoning carefully about AI’s implications for society’s constitutional software.
We’re also interested in work that takes this early conceptual end of the policy pipeline to its practical endpoint: identifying stakeholders’ interests and political constraints, drafting legislative text, and identifying practical vehicles for enacting these changes. While this process may be thankless at first, it will ultimately be incredibly important. Policy change often happens slowly, then all at once: starting now means that we are more likely to have vetted proposals ready to enact when the Overton Window opens.
Technical guardrails for sensitive AI applications.
National governments are already using frontier AI systems for administrative work, military operations, and intelligence. AI companies are using them to develop yet more powerful systems. AI has the potential to continue automating this work, making these institutions more powerful while concentrating the organizations’ oversight more tightly in the hands of their leaders.
As AI takes over the tasks of human employees in sensitive settings, we need to establish standards for its behavior. It seems like common sense to train AI systems to obey the law, but what should they do in legally ambiguous situations? What input information should they require to ensure that they are obeying the law? Are there cases where AIs should be equipped with a communication line to oversight organizations, and what should they be trained to disclose to them? We’re eager to support work that develops, tests, and implements answers to these questions.
Structured transparency for cross-institutional oversight.
In principle, modern AI systems could be used to conduct sophisticated audits of sensitive data while preserving its privacy. This could enable much stronger oversight relationships between powerful institutions. For example, legislatures could use this technology to validate that national-security agencies are not violating citizens’ rights. Oversight capabilities that scale with AI will be increasingly important as the overseen institutions themselves make use of AI. This could enable sophisticated forms of misuse that both require careful inspection to catch and create large amounts of data to validate.
AI for better reasoning and truth-seeking.
AI-powered tools could enable citizens to keep pace with an increasingly fast-moving world. They could actively improve the power balance of society by parsing financial and governmental data to identify misbehavior, making the actions of the powerful more legible. Coupled with coordination tools, they could help citizens identify and support prosocial policies and projects.
These are only a few of the ideas in this RFP. We think these problems will benefit from a wide variety of participants with different viewpoints and solution concepts; we’re eager to see applicants propose ideas that aren’t in this document.