Technical Documentation under the EU AI Act: What Providers Must Prepare Before Entering the Market

Developing an AI system is only part of the journey to the EU market. Equally important is demonstrating that the system complies with the applicable legal requirements. For providers of AI systems, assurances regarding a product’s safety, reliability or accuracy are not sufficient.

The EU AI Act requires such claims to be supported by technical documentation that makes it possible to trace how the system was developed, how its performance was tested and how the associated risks are managed. This documentation must be drawn up before the system is placed on the market or put into service, rather than only when the relevant documents are requested by a regulatory authority.

So, what exactly must technical documentation contain under the EU AI Act, how should it be prepared, and what requirements must providers consider before entering the EU market? Let us take a closer look.

Who Must Prepare Technical Documentation and When?

First, it is important to note that the EU AI Act does not impose a uniform technical documentation obligation on all AI systems and models. The requirements for preparing such documentation depend on the type of product concerned.

The first category comprises high-risk AI systems.

Under Article 11 of the EU AI Act, providers of such systems must draw up technical documentation before placing the system on the market or putting it into service and must subsequently keep that documentation up to date.

Article 6 of the EU AI Act determines which AI systems are classified as high-risk. These include certain systems used in employment, education, critical infrastructure, law enforcement and other areas specified in the Regulation. However, the fact that a system is used in one of these areas does not automatically mean that it is classified as high-risk, as the system’s intended purpose and the classification criteria laid down in the EU AI Act must also be taken into account.

The second category comprises general-purpose AI models.

Article 53 of the EU AI Act establishes separate technical documentation requirements for providers of such models. In particular, it requires them to document the model’s training and testing processes and the results of its evaluation, as well as to keep the relevant documentation up to date.

In summary

Before determining the content of the technical documentation, a provider must establish which category its product falls into. For high-risk AI systems, the requirements concerning the content of technical documentation are set out in Annex IV to the EU AI Act, while those applicable to general-purpose AI models are set out in Annex XI. In addition, Annex XII specifies the information and documentation that providers of such models must make available to other providers intending to integrate those models into AI systems.

What Must the Technical Documentation for High-Risk AI Systems Contain?

Annex IV to the EU AI Act sets out the minimum content of the technical documentation for high-risk AI systems. In practice, the provider must compile information that makes it possible not only to understand how the system works but also to assess its compliance with the applicable requirements.

In particular, depending on the characteristics of the AI system concerned, the technical documentation must cover:

  • Description of the system and its intended purpose. Information about the provider, the system’s intended purpose, its current version and its relationship with previous versions. The documentation must also describe how the system interacts with other software and hardware, including other AI systems, the relevant software versions and the requirements for their updates. It must also specify the forms in which the system is placed on the market or put into service, the hardware required, the main elements of the user interface and the instructions for use. Where the system is a component of another product, images of that product’s external features, marking and internal layout must also be included, where applicable.
  • Architecture and development process. A description of the methods and stages of development, the system’s main components and their interactions, the general logic of the algorithms, and the key design choices and assumptions on which they are based. The documentation must also specify the main classification approaches, optimisation parameters, expected outputs of the system and their quality, and explain the technical trade-offs made to comply with the requirements of the EU AI Act. The computational resources used for development, training, validation and testing must also be documented. Where pre-trained systems or third-party tools have been used, the documentation must explain how they were used, integrated or modified.
  • Information about the data. Information on the training methodologies and techniques, the training datasets used, their provenance, scope and main characteristics, and the methods used to obtain, select, label and clean them. The data used for validation and testing of the system, together with their main characteristics, must be described separately.
  • Testing and evaluation results. A description of the validation and testing procedures, the metrics used to measure accuracy, robustness and compliance with other applicable requirements, as well as an assessment of potentially discriminatory impacts. The suitability of the selected performance metrics for the particular AI system must also be justified. The documentation must include testing logs and all relevant test reports, dated and signed by the persons responsible.
  • Capabilities, limitations and safety measures. Detailed information on the system’s capabilities and limitations, its expected level of accuracy, including in relation to relevant persons or groups of persons, foreseeable unintended outcomes and sources of risks to health, safety and fundamental rights, including risks of discrimination. The documentation must also describe the risk management system referred to in Article 9 of the EU AI Act, an assessment of the human oversight measures needed, the technical measures facilitating the interpretation of the system’s outputs, input data requirements and the cybersecurity measures implemented.
  • Demonstration of compliance with the requirements of the EU AI Act. A list of harmonised standards applied in full or in part, the references of which have been published in the Official Journal of the European Union. Where such standards have not been applied, the documentation must provide a detailed description of the solutions adopted to meet the relevant requirements of the Regulation and identify any other standards and technical specifications applied. A copy of the EU declaration of conformity must also be included.
  • Information on changes and post-market monitoring. A description of relevant changes made by the provider to the system throughout its lifecycle. Where predetermined changes to the system or its performance have been envisaged, these changes and the technical solutions ensuring continued compliance with the applicable requirements must be described in detail. In addition, the documentation must contain a detailed description of the system used to evaluate the AI system’s performance after it has been placed on the market, including the post-market monitoring plan referred to in Article 72 of the EU AI Act.

However, Annex IV does not require each of these elements to be presented as a separate document with a prescribed title. What matters is that the technical documentation, taken as a whole, contains all information applicable to the particular system and makes it possible to assess its compliance with the requirements of the EU AI Act.

When Is a Single Set of Technical Documentation Required, and Who Can Use a Simplified Form?

A specific rule applies to high-risk AI systems that form part of products covered by the EU legislation listed in Section A of Annex I to the EU AI Act. This may concern, for example, medical devices, machinery, lifts or toys. Under Article 11(2), a single set of technical documentation must be drawn up for such systems, covering both the requirements of the EU AI Act and those of the relevant EU product legislation. In other words, the provider does not need to prepare two separate sets of documentation.

In addition, the EU AI Act provides for the possibility of using a simplified form of technical documentation for smaller companies. This option is available to small and medium-sized enterprises, including start-ups, as well as small mid-cap enterprises. The European Commission is required to establish a specific form enabling them to present the information required under Annex IV in a simplified manner. However, this does not exempt providers from the obligation to provide all the necessary information about the high-risk AI system.

Overall, the provider’s task is not merely to collect technical files but to document how the system was developed and tested and how its compliance with the applicable requirements is ensured. For this reason, drawing up technical documentation should be part of the system development process rather than a final formality before entering the market.

What Must the Technical Documentation for General-Purpose AI Models Contain?

The EU AI Act establishes separate technical documentation requirements for general-purpose AI models. Under Article 53(1)(a), their providers must draw up and keep up-to-date technical documentation of the model, including its training and testing process and the results of its evaluation. The minimum content of such documentation is set out in Annex XI to the EU AI Act. However, the scope of the required information depends on the size and risk profile of the model concerned.

In particular, the technical documentation must cover:

  • General description of the model and its intended purpose. Information on the tasks the model is intended to perform, the types of AI systems into which it can be integrated, acceptable use policies, the release date, methods of distribution and licensing terms.
  • Technical characteristics and integration capabilities. Information on the model’s architecture, number of parameters, modality, and the format of inputs and outputs. The documentation must also describe the technical means, infrastructure and tools required to integrate the model into AI systems.
  • Development and training process. A description of the training methodologies and techniques used, key design choices and assumptions, optimisation objectives and the values of the relevant parameters. In addition, the computational resources used to train the model, the duration of training and other relevant characteristics of the process must be documented.
  • Information on the data used. Information on the type, provenance, quantity, scope and main characteristics of the data used for training, testing and validation of the model. The documentation must also describe how the data was obtained, selected and processed, including cleaning and filtering, as well as measures taken to identify unsuitable data sources and methods used to detect bias.
  • Energy consumption of the model. Information on the known or estimated energy consumption. Where actual figures are unknown, the estimate may be based on information about the computational resources used for training.

For general-purpose AI models with systemic risk, Annex XI provides for additional requirements. Their technical documentation must also include:

  • Model evaluation strategies and results. A detailed description of the evaluation carried out, the criteria, metrics and methodologies used to identify the model’s limitations, including the use of publicly available evaluation protocols and tools or other appropriate methods.
  • Testing and adaptation measures. A detailed description of internal or external adversarial testing, as well as model adaptations, including alignment and fine-tuning.
  • Detailed description of the system architecture. An explanation of how the software components interact with each other and are integrated into the overall functioning of the system.

However, the obligations of a provider of a general-purpose AI model are not limited to drawing up technical documentation under Annex XI.

Under Article 53(1)(b) of the EU AI Act, the provider must also draw up, keep up-to-date and make available information and documentation to providers of AI systems who intend to integrate the model into their products. These materials must enable such providers to understand the model’s capabilities and limitations and comply with their own obligations under the Regulation. The minimum content of this documentation is set out in Annex XII to the EU AI Act.

In particular, it must cover:

  • General description of the model. Information on its intended purpose, the tasks it can perform, the types of AI systems into which it can be integrated, acceptable use policies, the release date, methods of distribution and licensing terms.
  • Technical characteristics and compatibility. Information on the model’s architecture, number of parameters, modality, and the format of inputs and outputs, including their maximum sizes, such as the context window length. Where applicable, the documentation must also specify how the model interacts with other software or hardware and the versions of software associated with its use.
  • Integration requirements. A description of the technical means, infrastructure, tools and instructions required to integrate the model into AI systems.
  • Information on the data used. Information on the type and provenance of the data used for training, testing and validation of the model, as well as the methods used to prepare and process that data, including cleaning and filtering.

Thus, Annexes XI and XII provide for two sets of documentation serving different purposes. Annex XI concerns the technical documentation of the model itself, including for submission to competent authorities upon request. Annex XII is intended to provide other providers with the information necessary to integrate the model into their own AI systems.

At the same time, Article 53(2) of the EU AI Act provides for an exception to these two obligations for certain general-purpose AI models released under a free and open-source licence. To qualify for this exception, the licence must allow access to, use, modification and distribution of the model, and the model’s parameters, including its weights, information on its architecture and information on model usage must be made publicly available. However, this exception does not apply to general-purpose AI models with systemic risk.

Accordingly, providers of general-purpose AI models must take into account not only the requirements for documenting the model’s development and training process but also the obligation to provide other market participants with the information necessary for the proper use and integration of the model.

What Must Be Considered After Preparing the Technical Documentation?

A provider’s obligations do not end with drawing up technical documentation. It is also important to ensure that the documentation remains up to date, is properly retained and is available to the competent authorities.

For high-risk AI systems, Article 18 of the EU AI Act requires technical documentation to be retained for 10 years after the system has been placed on the market or put into service. In addition, under Article 43, such systems must undergo the applicable conformity assessment procedure before entering the market. Depending on the category of the system, this procedure may involve internal control or the involvement of a notified body.

As regards general-purpose AI models, Article 53 of the EU AI Act requires providers to keep technical documentation up to date and make it available to the AI Office and national competent authorities upon request. At the same time, providers must make the information and documentation specified in Annex XII available to those intending to integrate the model into their own AI systems, enabling them to understand its capabilities and limitations and comply with the requirements of the Regulation.

In conclusion, technical documentation is not merely a formality before entering the EU market. It is a means of demonstrating compliance with the requirements of the EU AI Act and requires both timely preparation and ongoing updates.

Tags
  • AI
  • AI Act
Do you have any questions for the lawyers?
up to 500 characters
An error occurred
The request has been sent Thank you for your message! We will process it as soon as possible.

Articles on the topic