aises_8_5

<h1 id="sec:nat-gov">8.5 National Governance</h1>
<p><strong>Overview.</strong> Government action may be crucial for AI
safety. Governments have the authority to enforce AI regulations, to
direct their own AI activities, and to influence other governments
through measures such as export regulations and security agreements.
Additionally, major governments can leverage their massive budgets,
diplomats, intelligence agencies, and leaders selected to serve the
public interest. More abstractly, as we saw in the Collective Action Problems chapter, institutions
can help agents avoid harmful coordination failures. For example,
penalties for unsafe AI development can counter incentives to cut
corners on safety.<p>
This section provides an overview of some potential ways governments may be able to advance AI safety including safety standards and regulations, liability for AI harms, targeted taxation, and public ownership of AI. We also describe various levers for improving societal resilience and for ensuring that countries focused on developing AI safely to do not fall behind less responsible actors.</p>
<h2 id="standards-and-regulations">8.5.1 Standards and Regulations</h2>
<p>To ensure that frontier AI development and deployment is safe, two
complementary approaches are formally establishing strong safety
measures as best practices for AI labs, and requiring the implementation
of strong safety measures. This can be done using standards and
regulations, respectively.</p>
<p><strong>Standards are specified best practices.</strong> Standards
are written specifications of the best practices for carrying out
certain activities. There are standards in many areas, from
telecommunications hardware to country codes to food safety. Standards
often aim to ensure quality, safety, and interoperability; for instance,
the International Food Standard requires the traceability of products,
raw materials and packaging materials.</p>
<p><strong>The substance of AI safety standards.</strong> In the context
of frontier AI, technical standards for AI safety could guide various
aspects of the AI model life cycle. Before training begins, training
plans could be assessed to determine if training is safe, based on
evidence about similar training runs and the proposed safety methods.
After training begins, models could be evaluated to determine if further
training or deployment is safe, based on whether models show dangerous
capabilities or propensities. During deployment, particularly powerful
models could be released through a monitored API. Standards could guide
all aspects of this process.</p>
<p><strong>Standards are developed in dedicated standard-setting
organizations.</strong> Many types of organizations, from government
agencies to industry groups to other nonprofits, develop standards. Two
examples of standard-setting organizations are the National Institute of
Standards and Technology (NIST) and the International Organization for
Standardization (ISO). In the US, standard setting is often a
consensus-based activity in which there is substantial deference to
industry expertise. However, this increases the risk that standards
over-represent industry interests.</p>
<p><strong>The impact of standards.</strong> Standards are not
automatically legally binding. Despite that, standards can advance
safety in various ways. First, standards can shape norms, because they
are descriptions of best practices, often published by authoritative
organizations. Second, governments can mandate compliance with certain
standards. Such “incorporation by reference” of an existing standard may
bind both the private sector and government agencies. Third, governments
can incentivize compliance with standards through non-regulatory means.
For example, government agencies can make compliance required for
government contracts and grants, and standards compliance can be a legal
defense against lawsuits.</p>
<p><strong>Regulations are legally binding.</strong> Regulations are
legal requirements established by governments. Some examples of
regulations are requirements for new foods and drugs to receive an
agency’s approval before being sold, restrictions on the pollution
emitted by cars, requirements for aircraft and pilots to have licenses,
and constraints on how companies may handle personal data.</p>
<p><strong>Regulations are often shaped by both legislatures and
agencies.</strong> In some governments, such as the US and UK,
regulations are often formed through the following process. First, the
legislature passes a law. This law creates high-level mandates, and it
gives a government agency the authority to decide the details of these
rules and enforce compliance. By delegating rulemaking authority,
legislatures let regulations be developed with the greater speed, focus,
and expertise of a specialized agency. As we discussed, agencies often
incorporate standards into regulations. Legislatures also often
influence regulation through their control over regulatory agencies’
existence, structure, mandates, and budgets.<p>
Regulatory agencies do not always regulate adequately. Regulatory
agencies can face steep challenges. They can be under-resourced: lacking
the budgets, staff, or authorities they need to do well at designing and
enforcing regulations. Regulators can also suffer from regulatory
capture—being influenced into prioritizing a small interest group
(especially the one they are supposed to be regulating) over the broader
public interest. Industries can capture regulators by politically
supporting sympathetic lawmakers, providing biased expert advice and
information, building personal relationships with regulators, offering
lucrative post-government jobs to lenient regulatory staff, and
influencing who is appointed to lead regulatory agencies.<p>
Standards and regulations give governments some ways to shape the
behavior of AI developers. Next, we will consider legal means to ensure
that the developers of AI have incentives in line with the rest of
society.</p>
<h2 id="liability-for-ai-harms">8.5.2 Liability for AI Harms</h2>
<p>In addition to standards and regulations, legal liability could
advance AI safety. When AI accidents or misuse cause harm, liability
rules determine who (if anyone) has to pay compensation. For example, an
AI company might be required to pay for damages if it leaks a dangerous
AI, or if its AI provides a user step-by-step instructions for building
or acquiring illegal weapons.</p>
<p><strong>Non-AI-specific liability.</strong> Legal systems including
those of the US and UK have forms of legal liability that would
plausibly apply to AI harms even in the absence of AI-specific
legislation. For example, in US and UK law, negligence is grounds for
liability. Additionally, in some circumstances, such as when damages
result from a company’s defective product, companies are subject to
strict liability. That means companies are liable even if they acted
without negligence or bad intentions. These broad conditions for
liability could apply to AI, but there are many ways judges might
interpret concepts like negligence and defective products in the case of
AI. Instead of leaving it to a judge’s interpretation, legislators can
specify liability rules for AI.</p>
<p><strong>There are advantages to holding AI developers liable for
damages.</strong> Legal liability helps AI developers internalize the
effects of their products on the rest of society by ensuring that they
pay when their products harm others. This improves developers’
incentives. Additionally, legal liability helps provide accountability
without relying on regulatory agencies. This avoids the problem that
government agencies may be too under-resourced or captured by industry
to mandate and enforce adequate safety measures.</p>
<p><strong>Legal liability is a limited tool.</strong> There are
practical limits to what AI companies can actually be held liable for.
For example, if an AI were used to create a pandemic that killed 1 in
100 people, the AI developer would likely be unable to pay beyond a
small portion of the damages owed (as these could easily be in the tens
of trillions). If the amount of harm that can be caused by AI companies
exceeds what they can pay, AI developers cannot fully internalize the
costs they impose on society. This problem can be eased by requiring
liability insurance (a common requirement in the context of driving
cars), but there are amounts of compensation that even insurers could
not afford. Moreover, sufficiently severe AI catastrophes may disrupt
the legal system itself. Separately, liability does little to deter AI
developers who do not expect their AI development to result in large
harms—even if their AI development really proves catastrophically
dangerous.<p>
Ensuring legal liability for harms that result from deployed AIs helps
align the interests of AI developers with broader social interests.
Next, we will consider how governments can mitigate harms when they do
occur.</p>
<h2 id="taxation">8.5.3 Targeted Taxation</h2>
<p>Although taxes do not directly force risk internalization, they can provide government revenues that can be reserved for risk mitigation or disaster relief efforts. For example, the Superfund is a US government program that funds the cleanup of abandoned hazardous waste sites. It is funded by excise taxes--—a special tax on some good, service, or activity---on chemicals. The excise tax ensures that the chemical manufacturing industry pays for the handling of dangerous waste sites that it has created. Special taxes on AIs could support government programs to prevent risks or address disasters.</p>

<p><strong>Taxation is the most straightforward redistribution
policy.</strong> Wealth redistribution or social spending is most often
funded by taxes. Progressive tax policies, adopted today by most but not
all nations, require that those who earn more money pay a greater
proportion of their earnings than those who earn less. Governments then
seek to redistribute wealth through a wide variety of programs, from
healthcare and education to direct checks to people. In light of massive
earnings by tech companies, economists and policymakers have already
proposed specialized taxes on robots, digital goods and services, and
AI. If AI enables big tech companies to make orders of magnitude more
money than any previous company, while much of the population is unable
to pay income tax, targeted taxes on AI may be necessary to maintain
government revenues.<p>
Seeking to encourage innovation, the US’s tax landscape currently favors
capital investment over labor investment. If a firm wants to hire a
worker, they have to pay payroll taxes and employees have to pay a
number of separate taxes. If a firm replaced their worker with an AI,
they would presently only pay corporate tax, which was incurred anyway
<span class="citation" data-cites="acemoglu2020does">[8]</span>. As with
other redistributive policies surveyed in this chapter, there are
political barriers to high taxes, including the ability of companies to
lobby the government in favor of lower taxes, as well as long-standing
and contentious debates over the economic effects of taxation.</p>

<h2 id="public-ownership">8.5.4 Public Ownership</h2>
<p>Another approach to aligning AI development with societal interests would be public ownership over some AI systems. Public ownership means that the public receives both the benefits and the costs (risks) of AIs, addressing moral hazards. Governments might seek to assume partial or full ownership of powerful AI systems both in order to control their operations safely and to guarantee equitable revenue distribution.
<p>
Public utilities are often nationalized because they benefit from
economies of scale, where larger organizations are cheaper and more
efficient. For example, building power plants and transmission lines is
expensive, so governments are interested in maintaining one large,
well-regulated company. The French government owns the utility company
Électricité de France, whose nuclear plants power the majority of the
country’s electricity. AIs may be similar to public utilities if they
are ubiquitous throughout the economy, essential to everyday activity, and
require special safety considerations.The potential tendency of AI to strengthen concentration of market power is discussed further in the Economic Engine section in the Beneficial AI and Machine Ethics chapter.</p>

<h2 id="improving-resilience">8.5.3 Improving Resilience</h2>
<p>The government actions already discussed focus on preventing unsafe
AI development and deployment, but another useful intervention point may
be mitigating damages during deployment.  We discuss this briefly here and at greater length in the "Systemic Safety" section (3.5). </p>
<p><strong>Resilience may protect against extreme risks.</strong>
Governments may be able to improve societal resilience to AI accidents
or misuse through promoting cybersecurity, biosecurity, and AI
watchdogs. Measures for increasing resilience may raise the level of AI
capabilities needed to cause catastrophe. That would buy valuable time
to develop safety methods and further defensive measures—–ideally enough
time for safety and defense to always keep pace with offensive
capabilities. Sufficient resilience could lastingly reduce risk.</p>
<p><strong>Policy tools for resilience.</strong> To build resilience,
governments could use a variety of policy tools. For example, they could
provide R&amp;D funding to develop defensive technologies. Additionally,
they could initiate voluntary collaborations with the private sector to
assist with implementation. Governments could also use regulations to
require owners of relevant infrastructure and platforms to implement
practices that improve resilience.</p>
<p><strong>Tractability of resilience.</strong> If governments defend
narrowly against some attacks, rogue AIs or malicious users might just
find other ways to cause harm. Increasingly advanced AIs could pose
novel threats in many domains, so it may be hard to identify or
implement targeted defensive measures that make a real difference.
However, perhaps there are a few domains where societal vulnerabilities
are especially dire and tractable to improve (cybersecurity or
biosecurity, for example), while some defensive measures could provide
broader defenses (such as AI watchdogs).</p>
<p><em>Cybersecurity</em>. AIs could strengthen cybersecurity. AIs could
identify and patch code vulnerabilities (that is, they could fix faulty
programming that would let attackers get unauthorized access to a
computer). AIs could also help detect phishing attacks, malware and
other attempts to attack a computer network, enabling responses such as
blocking or quarantining malicious programs. These efforts could be
targeted to defend widely used software and critical infrastructure.
However, AIs that identify code vulnerabilities are dual-use; they can
be used to either fix or exploit vulnerabilities.<p>
<em>Biosecurity</em>. Dangerous pathogens can be detected or countered
through measures such as wastewater monitoring (which might be enhanced
by anomaly detection), far-range UV technology, improved personal
protective equipment, and DNA synthesis screening that is secure and
universal.<p>
<em>AI watchdogs</em>. AIs could monitor the activity of other AIs and
flag dangerous behavior. For example, AI companies can analyze the
outputs of their own chatbots and identify harmful outputs.
Additionally, AIs could identify patterns of dangerous activities in
digital or economic data. However, some implementations of this could
harm individual privacy.<p>
Defensive measures including cybersecurity, biosecurity, and AI
watchdogs may mitigate harms from the deployment of unsafe AI systems.
However, defensive measures, regulation, and liability may all be
insufficient for safety if the countries that implement them all fall
behind the frontier of AI development. Next, we will consider how
countries can remain competitive while ensuring safety in domestic AI
production.</p>
<h2 id="not-falling-behind">8.5.4 Not Falling Behind</h2>
<p>If some countries take a relatively slow and careful approach to AI
development, they may risk falling behind other countries that take less
cautious approaches. It would be risky for the global leaders in AI
development to be within countries that lack adequate guardrails on AI.
Various policy tools could allow states to avoid falling behind in AI
while they act to keep their own companies’ AIs safe.</p>
<p><strong>Risks of adversarial approaches.</strong> Adversarial
approaches to AI policy–—that is, policies focused on advancing one
country’s AI leadership at the expense of another’s—–have risks.
Adversarial policies could rely on wrong assumptions about which states
will adequately guardrail AI, and they could also motivate
counter-actions and increase international tensions (making cooperation
harder). Competitive mindsets can also encourage de-prioritizing safety
in the name of competing–—in the Collective Action Problems chapter, we consider this problem in
greater depth using formal models. Additionally, AI technologies might
proliferate quickly even with strong efforts to build national leads in
AI.<p>
International cooperation, as explored in section 8.7, may enable states to keep their AIs safe, preserve national
competitiveness, and avoid the pitfalls of adversarial AI policy. Still,
as options for cases where cooperation fails, here we consider several
policy tools for preserving national competitiveness in AI.</p>
<p><strong>Export controls.</strong> Restrictions on the export of
AI-specialized hardware can limit states’ access to a key input into AI.
Due to the extreme complexity of advanced semiconductor manufacturing,
it is very difficult for states subject to these export controls to
manufacture the most advanced hardware on their own. Additionally, the
AI hardware supply chain is extremely concentrated, perhaps making
effective export controls possible without global agreement. We explore
this further in the section.</p>
<p><strong>Immigration policy.</strong> Immigration policy affects the
flow of another important input into AI development: talented AI
researchers and engineers. Immigration could be an asymmetric advantage
of certain countries; surveys suggest that the international AI
workforce tends to have much more interest in moving to the US than
China <span class="citation"
data-cites="zwetsloot2021winning">[1]</span>. Immigrants may be more
likely to spread AI capabilities internationally, through international
contacts or by returning to their native countries, but many immigrants
who are provided with the chance choose to stay in the US.</p>
<p><strong>Information security.</strong> Information security measures
could slow the diffusion of AI insights and technologies to countries or
groups that lack adequate guardrails. For example, governments could
provide information security assistance to AI developers, and they could
incentivize or require developers’ compliance with information security
standards.</p>
<p><strong>Intelligence collection.</strong> Collecting and analyzing
intelligence on the state of AI development in other countries would
help governments avoid both unwarranted complacency and unwarranted
insecurity about their own AI industries.<p>
Governments can use a range of measures to remain internationally
competitive while maintaining the safety of domestic AI development.</p>
<h3 id="information-security">Information Security</h3>

<p><strong>Model theft as a national security concern.</strong> If model weights from advanced AI systems are stolen or leaked, this could allow state or non-state actors to misuse these models. For example, they could maliciously use models for offensive purposes such as cyberattacks or the development of novel weapons (as discussed in more detail in Chapter 1). Assuming AI systems were sufficiently capable and valuable from a military and economic perspective, the leaking of this intellectual property to competitors would represent a major blow. 

<p><strong>The likelihood of theft or leaks of model weights appears high.</strong> First, the most advanced AI systems are likely to be highly valuable due to their ability to perform a wide variety of economically useful activities. Second, there are strong incentives to steal models given the high cost of developing state of the art systems. Lastly, these systems have an extensive attack surface because of their extremely complex software and hardware supply chains. In recent years, even leading technology companies have been vulnerable to major attacks, such as the Pegasus 0-click exploit that enabled actors to gain full control of high-profile figures' iPhones and the 2022 hack of NVIDIA by the Lapsus group, which claimed to have stolen proprietary designs for its next-generation chips. </p>

<p><strong>There are various attack vectors that could be exploited to steal model weights.</strong> These include running unauthorized code that exploits vulnerabilities in software used by AI developers, or attacking vulnerabilities in security systems themselves. Attacks on vendors of software and equipment used by an AI developer are a major concern, as both the hardware and software supply chains for AI are extremely complex and involve many different suppliers. Other techniques that are less reliant on software or hardware vulnerabilities include compromising credentials via social engineering (e.g. phishing emails) or weak passwords, infiltrating companies using bribes, extortion or placement of agents inside the company, and unauthorized physical access to systems. Even without any of these attacks, abuse of legitimate Application Programming Interfaces (APIs) can enable extraction of information about AI systems. Research has shown that it is possible to recover portions of some of OpenAI's models using typical API access. </p>

<p><strong>Securing model weights is complex and challenging.</strong> Basic approaches to thwart opportunistic attacks include using multifactor authentication, developing incident detection and response capabilities, and limiting the number of people with access to model weights. More advanced threats require more elaborate measures such as penetration testing with a well-resourced external team, establishing an insider threat program, reviewing vendor and supplier security, and hardening interfaces to weight access against weight exfiltration. These responses illustrate several of the safe design principles discussed in \nameref{chap:safety-engineering}. The principle of least privilege is directly applied by limiting access to model weights. Red-teaming is an example of the concept of antifragility. The need for multiple independent security layers demonstrates the importance of defense in depth in securing advanced AI systems against potential threats.</p>

<p><strong>Defending model weights likely requires heavy investments and a strong safety culture.</strong> The measures discussed above are likely insufficient to rule out successful attacks by highly resourced state actors looking to steal model weights. This might require novel approaches to the design of the hardware and facilities used to store model weights. A long-term commitment to building an organisational safety culture (further discussed in the Safety Engineering chapter) is also crucial. One major challenge of information security is to build buy-in from employees through effective communication and building a company culture that values security. Without this, measures that are important from a security perspective, but significantly reduce productivity and convenience, might prove impossible to enforce or be bypassed.</p>

<h3 id="conclusions-about-national-governance">Conclusions About
National Governance</h3>
<p>National governments have many tools available for advancing AI
safety. Standards, regulations, and liability could stop dangerous AIs
from being deployed, while encouraging the development of safe AIs.
Improved resilience could mitigate the damage of dangerous deployments
when they do occur, giving us more time to create safe AIs and
mitigating some risk from dangerous ones. Measures such as strong
information security could allow governments to ensure domestic AI
production is both safe and competitive. Each of these approaches has
largely distinct limitations—for example, regulations may be held back
by regulatory capture, while liability might impose too few penalties
too late—so effective governance may require combining many of the
government actions discussed in this section.<p>
With robust AI safety standards and regulations, a well-functioning
legal framework for ensuring liability, strong resilience against
societal-scale risks from AIs, and measures for not being outpaced
internationally by unconstrained AI developers, there would be multiple
layers of defense to protect society from reckless or malicious AI
development.</p>

<br>
<br>
<h3>References</h3>
<div id="refs" class="references csl-bib-body" data-entry-spacing="0"
role="list">
<div id="ref-zwetsloot2021winning" class="csl-entry" role="listitem">
<div class="csl-left-margin">[1] R.
Zwetsloot, <span>“Winning the tech talent competition,”</span>
<em>Center for Strategic and International Studies</em>, p. 2,
2021.</div>
</div>
</div>