EU AI Act (Part 3): High-Risk AI & Mandates

The Artificial Intelligence Act (“AI Act”) has now gone into effect in the European Union. In Part One of this series, the Internet Law Attorneys at Revision Legal summarized the applicability of the AI Act, some exemptions from coverage, and the effective date timetable. Part Two summarized the Act’s enforcement mechanisms, risk-level framework, and banned types of AI programs and systems. This final Part Three summarizes the AI Act’s regulations of high, limited, and low-risk AI systems.

High-risk AI programs/systems under the AI Act

Looking at the AI Act, high-risk AI programs/systems can be seen to apply to three broad categories:

Societal risk — for example, risks to energy generation and provision, transportation systems, law enforcement, banking, government, etc.
Risks to fundamental rights/freedoms — for example, dangers to groups based on group characteristics, to individuals based on application of AI systems, etc.
Specific risks to individuals and businesses — for example, AI systems for automobiles, airplanes, machinery, medical devices, etc.

Further, looking at the AI Act, clearly, EU regulators have identified several sources of risk including:

From the AI programming itself as developed, modified, and as it might modify itself
From the inputs used, including inputs that may be false/fake, inaccurate, biased, etc.
From the lack of human input and supervision
From external threats such as hacking and cybercrime and
From internal misuse, accident, sabotage, etc.

To combat and attempt to mitigate these risks, the AI Act imposes a long list of mandates on developers of AI systems. Some of these mandates include pre-marketing testing and validation of the AI system, including accuracy of input data sets and generated outputs, proof of human oversight in design and implementation, proof of sufficient and adequate cybersecurity procedures and protocols (both internal and external), the requirement of obtaining — and then affixing — a “Conformité Européenne (“CE”) certification to the product and the registration of the the AI system with EU AI regulators. Other mandates include:

Having risk management policies and personnel in place
Establishing quality training programs for individuals involved with the AI program/system
Maintaining technical documentation
Providing transparency on how the AI system functions
Having robust monitoring programs
Promptly providing documents and access when demanded by regulators
And more

To whom do these mandates apply?

The AI Act distinguishes four types of persons/businesses involved in AI provision: Developers, deployers, distributors, and importers. Most of the obligations — those listed above, for example — are imposed on developers. Deployers have fewer obligations, most of which relate to proper use and oversight. Distributors and importers have even fewer obligations but are responsible for ensuring that the products are properly labeled with the CE markings.

Note that the categories can be fluid. If, for example, a distributor licenses an AI system from a developer and then modifies it, the distributor could become a developer (and, thus, subject to the higher requirements of the AI Act).

Limited and low-risk AI systems

The AI Act imposes relatively few mandates on developers of limited and low-risk AI programs. Generally, developers of low-risk AI systems must provide technical documentation, instructions for use, comply with the EU Copyright Directive and publish a summary about the content used for training. Developers of limited-risk AI products must also conduct model evaluations, adversarial testing, track and report serious incidents and ensure proper and adequate cybersecurity protections.

High-Risk AI in Employment and Human Resources

One domain where the EU AI Act’s high-risk designation will have immediate practical consequences for U.S. companies with EU operations is employment and human resources. The AI Act’s Annex III explicitly classifies AI systems used for recruitment, selection, or evaluation of candidates (including CV screening and ranking tools), making decisions about promotion, termination, and performance monitoring, and allocating tasks based on individual behavior or personal traits as high-risk AI systems.

For HR tech companies that sell AI-driven recruiting or performance management tools to EU-based employers, this means their products must comply with the full suite of high-risk AI mandates — including conformity assessments, CE marking, technical documentation, human oversight requirements, and registration in the EU’s AI database. Buyers (the deployers) must implement governance procedures, conduct a fundamental rights impact assessment before deploying the tool, and ensure that the AI system is only used in the way the provider documented.

Employees in the EU are entitled to notification when their employer uses a high-risk AI system to make or assist in making decisions that significantly affect them. They also have the right to explanation and the right to human review of AI-assisted employment decisions. These transparency obligations are distinct from — and additional to — GDPR transparency obligations.

GPAI Models: A Special Category

The EU AI Act introduces a new regulatory category that does not fit neatly into the provider-deployer framework: General Purpose AI (GPAI) models. A GPAI model is an AI model trained on a broad set of data at scale that demonstrates significant generality and is capable of performing a wide range of distinct tasks. Large language models fall within this definition.

All GPAI model providers must prepare technical documentation, comply with EU copyright law with respect to training data, and make certain information available to deployers who build applications on top of the GPAI model. GPAI models that present systemic risk — because of their scale, capability, or the range of domains in which they are deployed — face additional obligations, including model evaluation, adversarial testing, incident reporting, and cybersecurity measures.

The concept of systemic risk in GPAI models is assessed primarily based on training compute exceeding 10^25 FLOPS — a threshold that currently applies to the largest frontier models. As AI capabilities advance, this threshold and the list of obligations may be revised by the European AI Office, which has been established to oversee GPAI regulation and facilitate cross-border enforcement coordination.

Enforcement Mechanisms and the European AI Office

The AI Act creates a European AI Office within the European Commission to serve as the primary enforcement body for GPAI models and to coordinate enforcement across member states. For other AI systems, enforcement is primarily handled by national market surveillance authorities in each EU member state, with the European AI Office providing coordination and guidance.

The AI Act also creates a Scientific Panel of independent experts to assist the AI Office with evaluating GPAI models and identifying systemic risks. Businesses that develop or deploy AI systems in the EU should monitor the AI Office’s guidance, which will be issued through delegated and implementing acts over the coming years and will flesh out many of the Act’s more general requirements.

Contact the Internet Law and Social Media Attorneys at Revision Legal

For more information, contact the experienced Internet Law and Social Media Lawyers at Revision Legal. You can contact us through the form on this page or call (855) 473-8474.