Europe’s AI Act Moves Forward: What’s Happened, What’s Next, and How to Prepare for Compliance

Introduction

In December 2022, the Council of the European Union (Council) adopted its common position (Common Position) on the Artificial Intelligence Act (AI Act).¹ The Council’s action marks a milestone on the EU’s path to comprehensive regulation of artificial intelligence (AI) systems, building on the restrictions on automated decision-making in the General Data Protection Regulation.² The AI Act categorizes systems by risk and imposes highly prescriptive requirements on high-risk systems. In addition, the legislation has a broadly extraterritorial scope: it will govern both AI systems operating in the EU as well as foreign systems whose output enters the EU market. Companies all along the AI value chain need to pay close attention to the AI Act, regardless of their geographic location.

The European Commission (Commission) proposed the AI Act (EC Proposal) in April 2021.³ Since then, the EU’s co-legislators—the European Parliament (Parliament) and the Council—have separately considered potential revisions. The Council’s deliberations culminated in adoption of its “Common Position” (or version of the legislation). Parliament hopes to arrive at its position by the end of March.

After the Parliament adopts its position, the Council and Parliament will negotiate over the final text of the legislation along with the Commission, a process called the “trilogue.” In these negotiations, the parties will have to resolve some key disputes, including how to supervise law enforcement use of AI systems and, relatedly, how much latitude (if any) governments will have to use real-time and ex-post biometric recognition systems for law enforcement and national security. After the trilogue results in an agreed-upon text, that version will go before the Parliament and Council for final approvals.

In the meantime, there is enough clarity on the EU’s direction that businesses should begin considering whether their internal AI systems or AI-enabled products and services will be covered by the AI Act and, if so, what steps they will need to take for compliance.

The Council’s Major Changes

The Common Position contains several important changes from the EC Proposal.

Narrower Scope

The EC Proposal defined AI systems as: “software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with.”⁴ Critics have argued this definition is overly broad and sweeps in statistical and other processes used by businesses that are not generally considered to be AI.

In response, the Council revised the definition of an AI system:

a system that is designed to operate with elements of autonomy and that, based on machine and/or human-provided data and inputs, infers how to achieve a given set of objectives using machine learning and/or logic- and knowledge[-]based approaches, and produces system-generated outputs such as content (generative AI systems), predictions, recommendations or decisions, influencing the environments with which the AI system interacts.⁵

“[S]oftware” has been entirely removed from the definition while the concept of autonomous operation was added. In addition, the Council excluded “Statistical approaches, Bayesian estimation, search[,] and optimization methods,” limiting AI systems to those employing either machine learning or logic- and knowledge-based approaches.

The Council reinforced the importance of autonomy to the definition in a few ways. First, the Common Position removes from the definition of an AI system the categorization of the system as “software” and categorically excludes any system that uses “rules defined solely by natural persons to automatically execute operations.”⁶ Second, unlike the Commission Proposal, the Common Position introduces the concept of machine learning as one of several “approaches” an AI system may use to “achieve a given set of objectives . . . and produce[] system-generated outputs” such as content and decisions, and emphasizes that machine learning does not include “explicitly programm[ing a system] with a set of step-by-step instructions from input to output.”⁷ Third, the Common Position clarifies that “[l]ogic- and knowledge[-]based approaches . . . typically involve a knowledge base [usually encoded by human experts] and an inference engine [acting on and extracting information from the knowledge base] that generates outputs by reasoning on the knowledge base.”⁸

Beyond the definitions, the Common Position narrows the scope of the AI Act by excluding the use of AI systems and their outputs for the sole purpose of scientific research and development, for any R&D activity regarding AI systems, and for non-professional purposes.⁹ As discussed below, the Council also introduced various exemptions for national security and law enforcement.

Expansion of Prohibited Practices

The EC Proposal of April 2021 proscribed four categories of AI uses: distortion of human behavior through subliminal techniques, behavior manipulation exploiting age or disability, real-time remote biometric identification by law enforcement, and governmental social scoring, defined as the evaluation or classification of the trustworthiness of a person based on their social behavior in multiple contexts or known or predicted personal or personality characteristics.

The Common Position makes two revisions to these prohibitions: it now bans behavior manipulation exploiting not only age or disability, but also a person’s social and/or economic situation, and now prohibits social scoring not only by governmental entities, but also by private actors.¹⁰ (In several charts below, we summarize selected aspects of the AI Act, showing the Council’s additions in bold blue text and deletions in struck-through bold red text.)

High-Risk AI Use Cases

The EC Proposal listed eight broad categories of high-risk uses while empowering the Commission to add to the list. While the Common Position retains the eight categories, it made other significant revisions, including adding “digital infrastructure” to its definition of critical infrastructure, adding life and health insurance as new high-risk uses, and removing deep fake detection by law enforcement, crime analytics, and verification of the authenticity of travel documents.¹¹ Significantly, the Council would allow the Commission not only to add high-risk use cases, but also to delete certain high-risk use cases under certain conditions.¹²

Of course, the risk posed by an AI system depends upon how its output contributes to an ultimate action or decision. To prevent overdesignation of systems as high-risk, the Common Position requires consideration of whether the output of an AI system is “purely accessory in respect of the relevant action or decision to be taken and is not therefore likely to lead to a significant risk to the health, safety[,] or fundamental rights.”¹³ If so, the system is not high-risk.

Requirements for High-Risk AI Systems

Once an AI system is classified as high-risk, the AI Act subjects it to numerous detailed requirements. The Common Position makes certain obligations clearer, more technically feasible, or less burdensome than they were under the EC Proposal.¹⁴ For example, it limits the risks subject to the risk management system requirements to “only those which may be reasonably mitigated or eliminated through the development or design of the high-risk AI system, or the provision of adequate technical information.”¹⁵ In addition, the Common Position would not require risk management systems to address risks under conditions of reasonably foreseeable misuse.¹⁶

New Plan to Address General Purpose AI Systems

The EC Proposal did not contemplate “general purpose AI”—AI systems that can be adapted for a variety of use cases, including cases that are high-risk. This omission has triggered much debate over how to classify general purpose AI and where in the value chain to place compliance obligations.

The Council addressed these concerns in a few ways. First, the Common Position introduces a definition of general purpose AI: “[A]n AI system that . . . is intended by the provider to perform generally applicable functions, such as image and speech recognition, . . . [and] in a plurality of contexts.”¹⁷

Second, the Common Position delegates to the Commission responsibility for determining how to apply the requirements for high-risk AI systems to general purpose AI systems that are used as high-risk AI systems by themselves or as parts of other high-risk AI systems. The Commission would do so by adopting “implementing acts.”¹⁸

Third, providers of such general purpose AI systems would have to comply with many of the obligations of providers of high-risk AI systems, but not all. For instance, they would not have to have a quality management system.¹⁹ However, they would have to “cooperate with and provide the necessary information to other providers” incorporating their systems into high-risk AI systems or components thereof to enable compliance by the latter providers.²⁰ General purpose AI system providers can exempt themselves from these obligations altogether if they, in good faith considering the risks of misuse, explicitly exclude all high-risk uses in the instructions or other documentation accompanying their systems.²¹

These provisions do not apply to qualifying microenterprise, small-, or medium-sized providers of general purpose AI systems.²²

Additional Carveouts for National Security and Law Enforcement

Much debate has centered on the degree to which the Artificial Intelligence Act should apply to national security and law enforcement uses of AI systems. The governments of EU states have sought to protect their flexibility, especially in exigent circumstances, and the Common Position reflects this objective. For instance, the Council expressly excluded use of AI systems for national security, defense, and military purposes from the scope of the legislation.²³ The Common Position also exempts sensitive law enforcement data from collection, documentation, and analysis under the post-market monitoring system for high-risk AI systems.²⁴ And the Common Position somewhat expands permissible law enforcement uses of real-time remote biometric identification systems.²⁵

These changes set up what is expected to be perhaps the most difficult issue for resolution in the trilogue. The Parliament has been moving in the other direction—towards banning real-time remote biometric identification systems altogether²⁶—and there has been speculation the Council added some exemptions for law enforcement as bargaining chips for the upcoming negotiations.²⁷

Regulatory Sandboxes and Other Support for Innovation

The Common Position expands the regulatory support for innovation. It provides greater guidance for establishment of “regulatory sandboxes” that will allow innovative AI systems to be developed, trained, tested, and validated under supervision by regulatory authorities before commercial marketing or deployment. In addition, the Common Position allows for testing systems under real-world conditions both inside and outside the supervised regulatory sandboxes—the latter with various protections to prevent harms, including informed consent by participants and provisions for effective reversal or blocking of predictions, recommendations, or decisions by the tested system.²⁸

The Council also exempted qualifying microenterprise providers of high-risk AI systems from the requirements for quality management systems.²⁹

Emerging Global AI Regulation

Final passage of the AI Act (quite possibly later this year) will bring a massive increase in regulatory obligations for companies that develop, sell, procure, or use AI systems in connection with the EU. And other jurisdictions are stepping up their regulation of AI and other automated decision-making.

In the United States, the federal, state, and local governments all have focused on the risks AI poses. The leading bipartisan congressional privacy bill would regulate algorithmic decision-making.³⁰ The Federal Trade Commission is considering how to formulate rules on automated decision-making (as well as privacy and data security)³¹ while other federal agencies are cracking down on algorithmic discrimination in employment,³² healthcare,³³ housing,³⁴ and lending.³⁵ Meanwhile, California, Colorado, Connecticut, Virginia, and New York City have laws on automated decision-making that took effect on January 1 or will become effective this year.³⁶

The Chinese Cyberspace Administration has adopted provisions governing algorithms that make decisions or create content while Shanghai and Shenzhen have enacted laws as well.³⁷ The United Kingdom is working on its AI-governance strategy,³⁸ the Canadian government has introduced the Artificial Intelligence and Data Act within broader legislation,³⁹ and Brazil is developing its own AI regulatory legislation.⁴⁰ Meanwhile, the EU, UK, Brazil, China, South Africa, and other countries already regulate automated decision-making systems under their privacy laws. Indeed, the Dutch Data Protection Authority recently announced plans to begin supervising AI and other algorithmic decision-making for transparency, discrimination, and arbitrariness under the GDPR and other laws.⁴¹

A Practical Approach to Global Compliance

The AI Act’s final contours remain uncertain, as does the exact shape of AI regulation in other jurisdictions. Yet, we have enough clarity for companies to ready their global compliance programs.

A company can begin by taking stock of the systems it develops, distributes, or uses to make automated decisions affecting individuals or safe operations. Then, assess the risks each one poses. For help in spotting risks, adapt a checklist such as The Assessment List for Trustworthy AI⁴² to the particulars of the business and system. No single person or team may have a full understanding of how the system was created, trained, and tested, so probe assumptions and dependencies on work performed by others. Take care to vet each component of the system, including components purchased from vendors, because they may introduce risks, too.

Next, the business should mitigate the identified risks reasonably. There are many aspects to mitigating risks from automated decisions.

Explainability is a good starting point. Explanation facilitates appeal of an adverse outcome that does not make sense or acceptance of the result if it does. Moreover, having a wide range of explanations for a system eases oversight and gives the company’s leaders assurance the system is accurate and satisfies regulatory obligations and business objectives. Explaining Decisions Made with AI by the UK Information Commissioner’s Office and The Alan Turing Institute⁴³ provides practical advice on explainability.

After explainability, focus on bias. The Algorithmic Bias Playbook⁴⁴ is a useful guide for bias audits, which require social-scientific understanding and value judgments, not just good engineering. “[T]here is no simple metric to measure fairness that a software engineer can apply. . . . Fairness is a human, not a mathematical, determination.”⁴⁵ When an audit reveals disparate impacts against protected classes, the company’s lawyers will have to determine whether the distinctions may be justified by bona fide business reasons under the applicable laws. Even if so, leadership should contemplate whether the justification reflects the company’s values.

In addition to explainability and bias, a global AI compliance program should address documentation. For high-risk AI systems, the AI Act will mandate extensive documentation and recordkeeping about the system’s development, risk management and mitigation, and operations. Other AI laws—both adopted and proposed—have similar requirements. Even where not legally required, a business may choose to document how their AI and other automated decision-making systems were developed, trained, tested, and used. Records of reasonable risk mitigation can help defend against government investigations or private litigation over a system’s mistaken decisions. (In the fall, the Commission proposed a directive on civil liability for AI systems; once adopted, it will be easier for Europeans harmed by AI systems to obtain compensation.⁴⁶) Of course, greater document retention has downsides, so companies need to strike the right balance if going beyond legal requirements.

For a comprehensive approach to managing AI risks, consult the Artificial Intelligence Risk Management Framework (AI RMF) recently released by the US National Institute of Standards and Technology (NIST).⁴⁷ Accompanying the AI RMF is NIST’s AI RMF Playbook (which remains in draft).⁴⁸ The AI RMF Playbook provides a recommended program for governing, mapping, measuring, and managing AI risks. While prepared by a US agency, the AI RMF and AI RMF Playbook are intended to “[b]e law- and regulation-agnostic.”⁴⁹ They should support a global enterprise’s compliance with laws and regulations across jurisdictions.

Conclusion

The Council’s Common Position brings the AI Act one big step closer to adoption. Parliament’s passage of its version and then the trilogue negotiations remain ahead this year. Still, the final legislation is likely to include highly prescriptive regulation of high-risk AI systems. Companies should do what they can now to protect against the regulatory and litigation risks that are beginning to materialize—if they want to stay ahead of the curve.

© Arnold & Porter Kaye Scholer LLP 2023 All Rights Reserved. This Advisory is intended to be a general summary of the law and does not constitute legal advice. You should consult with counsel to determine applicable legal requirements in a specific fact situation.