Increasing the pace of preparation and research is vital. People and projects should aim to deliver results sooner rather than later, even if more deliberation would yield a cleaner outcome. If your results will only come out in 3 years’ time, build intermediate prototypes or pivot to a different project that yields (possibly-preliminary) outputs earlier.

Some of the most important work will ultimately require substantial, multi-year budgets. The opportunity now is to launch proof-of-work pilots with clear milestones that can start immediately and scale – many are viable at six- to seven-figure levels. A staged, evidence-driven portfolio lets funders de-risk their spend and double-down on what works.

Government agencies with budget, teeth, and technical staff should be created to monitor and, when needed, restrict further development by AI labs. This is politically difficult, for good reason. Conveying the reasoning for these decisions is as necessary as establishing the legal and institutional structure itself.

Political views on AI risk will vary over time. Consider the design of programs to be durable across administrations – non-partisan governance, multi-year funding, and risk-triggered escalation criteria.

Increasing general technical understanding among governments and leading company directors seems valuable but hard. A specific organization dedicated to explaining the current state of AI and likely near-term trajectories to influential people seems immensely valuable if it works, but it runs into the problem of being hard to distinguish from less credible actors.

Some effort should be dedicated towards preparation for vital opportunities in the near future. The focus should be on response speed and ability to quickly evaluate a project’s necessity. Examples of organizational structures include funds intending to finance promising projects within days of them being proposed.

Economic transition seems quite inevitable. The exact nature is somewhat uncertain, but it is very likely that entire areas of the job market will evaporate. The task is to identify which areas are likely to get hit sooner and, as a stretch goal, what to do about it.

Competent, dedicated, and charismatic leaders are crucial to ensuring projects run smoothly.

Competitive salaries and benefits, including good living environments for those working on governance and alignment projects, are crucial.

Diversify organizational types – nonprofits, businesses, governmental departments – so the benefits of incentives inherent to all of these structures can be captured.

Adversarial relationships with labs, such as attempts to draft coercive laws with leading AI labs, are strongly discouraged. Working relationships are crucial, including labs in other countries. (This is somewhat at odds with other recommendations, such as ones requiring strict training data control.)

The current ecosystem is quite disconnected – a lot of work is duplicated without much reason beyond failure to notice it is being done competently and reliably elsewhere. A system should be created to make parallel work more efficient. As a second-best, a more defined system for propagating the concrete conclusions from existing work would also be useful, but this is less relevant, as existing mechanisms have partial functionality. Internal sharing of alignment and control approaches among all labs might be a viable substitute if other approaches fail.

Currently, there is no shared global roadmap in place that is on track to fully address AI risk. Guarantees are not especially useful, but having a rough roadmap to ensure 80%, 95% and 99% chance of avoiding catastrophic outcomes would be beneficial. This is very challenging due to the tradeoffs between setting realistic goals and having realistic odds of avoiding catastrophe, but it is still worth attempting. This report outlines priority projects, likely owners, and early milestones that can seed a roadmap.

To be worthwhile, global coordination does not need to resolve every potential issue, but rather to establish a necessary condition – to reach a state where all major world players, both private and governmental, have some incentives to meet baseline, auditable safeguards.

There is a need to determine how to allocate authority and responsibility for deciding which alignment and control measures should be optimized. Questions include how equality is prioritized, and how to account for non-human experiences. More broadly, we may ask whether a majority of humans deciding on an answer to questions like these is sufficient for that to become a guiding principle.

A general interface for auditing, both internal and for third-parties, is required. Opinions vary on whether this should be achieved through centralized legislation or voluntary agreements. The results of using this general interface should be intuitive and usable by laymen.

While laws passed by governments and international organizations are the default, other avenues should also be considered. Examples include research agreements where labs share novel breakthroughs contingent on alignment measures being reliably followed.

A lot of value lies in a neat, accessible user interface for questions like capabilities. Dynamically answering user questions regarding what the system can actually do, when asked, is a widely desired property for audit software or even user-facing models.

There is currently no agreed-upon definition of what exactly AGI means. Getting authorized representatives from leading AI organizations in the same room to agree on the terms that refer to specific real-world consequences would be valuable, turning slogans into operational definitions, enabling consistency of action, and reducing the risk of talking past each other.

Current evaluations and red-teaming efforts are inadequate in terms of frequency and sophistication. Massive upscaling is necessary.

Technically-skilled people should be in positions of leadership. This doesn’t need to be ubiquitous, but a majority of people with a hand on the chisel, having a sense for what sculpture looks like, is necessary. Otherwise, the necessary research taste will be missing.

It is probably too late to train promising but inexperienced talent. Finding people already experienced in relevant fields who are currently working elsewhere is recommended. This doesn’t necessarily exclude young candidates, but they are much less likely to have the relevant technical skills.

Funding is likely to be constrained in the short term, especially if competitive salaries are used. A widely shared but not ubiquitous opinion was that this would become less of a constraint as time goes on due to the increased obviousness of posed dangers.

A widely noticed practical use case for alignment measures would be very useful, but it is hard to come by before reaching critical levels of danger. More effort should be devoted to finding examples of current failures that are likely indicative of future catastrophic failures.