I Paesi all’interno dell’UE e in tutto il mondo stanno cominciando a regolare l’uso dell’automazione nei processi decisionali pubblici. Il quadro giuridico è molto differenziato e il suo sviluppo è in una fase iniziale. Questo contributo definisce un possibile quadro di ricerca comparativa; in altre parole gli elementi per confrontare le diverse soluzioni sviluppate dai diversi sistemi giuridici di fronte alle sfide poste dall’automazione dei processi decisionali pubblici.
Jurisdictions within the EU and countries around the world are beginning to regulate the use of public Automated Decision Making (ADM). The legal framework thereof differs considerably, and its development is at an early stage. This contribution sets out a possible comparative research framework, with other words elements to compare the different solutions developed by the legal systems in the face of challenges of ADM.
Summary: 1. Introductory remarks.- 2. Background to the differentiated regulation of ADM in public law.- 3. A conceptual framework for comparison.- 3.1. (a) Regulation of data sources and input to ADM.- 3.2. (b) Regulation of the programming of procedures leading to ADM.- 3.3. (c) Regulation of individual decision-making procedures in the context of ADM. 3.4. (d) Privatisation of regulatory tasks to be conducted by ADM.- 4. Some tentative conclusions
1. Introductory remarks
The automation of public decision-making procedures is progressing in ever more policy areas. Automation affects procedures leading to administrative acts and administrative rule making where elements of the procedure are either fully or partly executed without a direct human intervention by sophisticated computer software, increasingly using machine learning and other tools referred to loosely as Artificial Intelligence (AI). The regulation of the use of Automated Decision Making (ADM) systems in public law, although in its infancy, has already spurned various differentiated approaches in various jurisdictions.
The EU has reacted to the spread of automated decision making and the rise of AI with proposing and in some cases adopting a number of legislative acts, some of which are also applicable to the public administration. The range of tools proposed is to date is just being explored in the legal literature. The various acts in themselves contain various addressees often not distinguishing between public and private recipients. They also frequently call for complex multilevel composite cooperation schemes for their enforcement.
Some EU Member States are also actively regulating ADM in the exercise of public functions. Other states rely on the regulation of AI powered ADM systems through more traditional public law tools.
Overall, the various reactions to the rise of AI and ADM in public law are an important object of study. Each jurisdiction’s approach can serve as a laboratory of ideas displaying different regulatory techniques for addressing common problems of the spread of ADM in public law. Therefore, all the papers published together in this section of issue 1/2023 of CERIDAP Journal, which allow for a comparative oversight over various jurisdictions’ reactions to ADM in public law, are a highly welcome contribution to the debate of mapping and assessing regulatory options and their respective effects. This introduction to the comparative overview of possibilities explained in the following papers sets out a possible comparative research framework: with other words, elements to compare the different solutions developed by the legal systems in the face of challenges of ADM.
2. Background to the differentiated regulation of ADM in public law
The challenges to understanding the effects of ADM in public decision-making are common to public law systems around the world but have also specific relevance in the highly integrated European system of close cooperation within multiple levels of government and administration. Accordingly, reactions to the digitalisation of public law exist on various levels – the European, the national and the sub-national. On the EU level approaches include an ever-growing list of approaches to legislation of data sources, data collections and data sharing. On the Member States level, the questions of the application of ADM in the public sectors may, on one hand, be concerned with specific services such as decisions related to benefits, housing, education, employment, and other areas that have a significant impact on people’s lives. AI systems have been increasingly used for ensuring public order, support decision making and combating crime and terrorism. AI use is especially advanced in such sectors as law enforcement, migration and asylum, and especially in tax-related investigations. On the other hand, also general legislation on administrative procedure and organization has reflected on new challenges arising from ADM.
In this dynamically developing field, comparing approaches across legal systems and levels of government is a helpful tool. It allows for an orientation of various regulatory approaches. Comparative approaches widen the pool of ideas and show how various systems have conceptualised elements of these changes and which approaches have been deemed appropriate to address challenges in this context. Obviously, there are different speeds in that process, some jurisdictions deliberately waiting before regulating, others moving proactively ahead. There also will be different implications of solutions chosen. Where decision making in public law systems across the EU and beyond is increasingly supported by automation, concepts of how to establish the legal framework for ADM are still evolving. In this sense, a comparative approach will be in line with the functionalist model of comparison under which actual solutions chosen by one legal system can provide inspiration for another legal system. Each legal system or each solution within such system can act as a kind of regulatory sandbox, capable of exploring advantages and disadvantages of chosen approaches. Thereby, debates arising in the context of the creation or use of certain principles in one jurisdiction can inform the search for solutions in another. Despite the different competences on EU and Member State levels, a comparative approach to the regulation of ADM in public law may provide for a valuable learning experience in a relatively new field with to date mostly experimental approaches.
3. A conceptual framework for comparison
Studying the diversity of these possible approaches and their consequences through a comparative lens, requires a comparative framework along which the various approaches can be assessed. In the following, I suggest a framework for comparison of various jurisdictions’ approaches to ADM in public law, which will be developed in several case studies to be published in the coming weeks in issue 1/2023 of CERIDAP Journal. This framework will be further developed by taking into account various factors of existing ADM regulation arising from the comparative studies. But a starting a comparison requires considering various fundamental factors of ADM regulation in public law, along which the reactions can be studied. I suggest four basic dimensions as basic common denominators for legal research.
3.1. (a) Regulation of data sources and input to ADM
The relation between ADM systems and the data sources from which they draw their input is linked to evolving notions of law on data collections. Such law of information is thus relevant in the context of understanding, through a comparative approach, public law conceptualisations of ADM. In this context, the EU legal system is striving to move towards broadening available data pools for public decision making. Considerations will be important to ensure availability but also quality and protection of certain types of data in such sources.
The current wave of EU data related legislation addresses many of the matters relating to data availability for data-driven public administration. In this context following its European Strategy for Data of February 2020, the European Commission introduced numerous data-related draft regulations, some of which by now have passed the legislative procedure and entered into force. The Commission’s European Strategy for Data foresaw an approach to regulate the use of data and data services but also to foster data sharing across economic, government, cultural and scientific sectors in areas such as health, mobility, and agriculture to create various European data spaces. A prime example for a push in this direction is the European Commission’s draft regulation, the “Interoperable Europe Act” of November 2022 seeking to link data sources across Europe for use by public decision making, however being at the same time remarkably silent on discussing means to ensure data quality in such exchanges. Another legislative initiative by the Commission is the Regulation on Data Governance (DGA), which intends to support data flows between countries and sectors thus benefitting public actors next to increasing data availability of public sector data to private parties including business. Finally, in the draft regulation on a European Data Act, the Commission seeks to align rules on data transfers to outside the EU (and EEA) of non-personal data with those rules applicable in the GDPR, an aspect particularly important for cloud service offers including those used by public administrations.
Other fields of EU data related legislation are concerned with linking various administrative levels taking place by creating joint data bases on which automated administrative procedures are built upon. This requires careful design not just of the software for the automated decision-making but also the responsibility for their use in relation to the data sources. Examples for this come from the large-scale information systems in the field of the EU’s Area of Freedom, Security and Justice (AFSJ) such as the Schengen Information System (SIS II). The principle of interoperability enables interconnectivity of data collections and thereby enlarges the ‘data lake’ available to processing by automated decision-making technology. For example, the AFSJ’s Electronic Travel Information and Authorisation System (ETIAS) and the Passenger Name Record (PNR) system will become linked with interoperability functions, allowing for searches to take place within these databases to be enriched with data from certain other interconnected databases..Maintaining large scale data collections and ensuring the interoperability of various such collections from different levels, the EU, Member States and private parties allows for an integration of automated decision making technologies into procedures. Often, this approach is EU driven, but Member States might have similar approaches for other policy areas.
The latter example as well as the cited acts from the new wave of data related and ADM governing legislative acts in the EU take many forms but are generally not published with the explicit wish of providing for general administrative law or even public law. Many, like the EU’s AI act, cover the use of advanced algorithms in public law, but are equally declared applicable for public and private applications of AI.
This raises the question as to the relation between general (administrative) law solutions in various legal systems versus policy-specific regulation – especially to whether a legal system requires transparency about the use of data from certain sources. Are there any generally applicable or policy-specific requirements for disclosing how ADM systems have used specific data points in reaching certain conclusions used in decision-making? Regulating interfaces between data use and machine elements of ADM as well as between ADM and human elements of decision making can become especially relevant when considering situations in which public participation is possible and must be fed into a decision-making process or where individual hearing rights are affected.
3.2. (b) Regulation of the programming of procedures leading to ADM
ADM relies on programming of a software system identifying the automated steps of the procedure. In public law the question arises as to the definition of the legal characteristic of such software, which becomes a tool integrated into a formal decision-making procedure. Identifying the necessities of the software, the programming of the software tool underlying a system, and the definition of its role within a decision-making procedure are thus decisive elements of the automation of individual decisions to be taken on that basis. The identification of the software system then needs to be distinguished from the actual process of decision making with the help of an automated system.
The comparative question that arises here is how such ADM system is conceived of and integrated into the structure of a constituted legal system. Generally, the latter consists of a cascade beginning with a generally enabling constitutional norm, over a policy-defining legislative act to more precise administrative rulemaking identifying in ever more detailed way the considerations to be taken into account and procedures to be followed in individual decision making applied to a specific set of facts. Irrespective of modes of possible judicial review or other accountability mechanisms, distinguishing, on one hand, an ADM-system contributing to the general norms governing individual decision making from, on the other hand, the individual decision-making process, is central for an understanding of the public regulatory approach to ADM.
The nature of the ‘software element’ of ADM systems to date appears rarely explicitly addressed in legislation – neither on the EU level nor on the Member State level. Case law will be more likely to be addressing this matter where it has been developed. On the EU level, for example, the matter has been addressed by CJEU case law identifying requirements of pre-defined factors identifying what must be addressed in a legal text and what may be subject to software design. In that context, the principles evoked by the CJEU are in principle technology-neutral requiring simply that legal texts address matters to be later developed in software defining the ADM components used in public law. However, at second sight, CJEU solutions suggested in the case law so far are to a certain degree technology-specific in making additional requirements for machine learning technology. An important comparative question is thus whether approaches sought are predominantly technology neutral or whether they are designed specifically for certain challenges arising from specific technological solutions.
Further, as observed from the national judicial practice, a question that has been raised is whether the addressee shall be given an access to the source code or algorithms the automation is based on. This information, it is often argued may be necessary to the addressee of the decision to assess how and what parameters have been decisive to issue the certain decision and in this line of argument concerns directly the right to defence. Yet, whether the access to the source code can indeed contribute to the right to defence realisation is questionable as the understanding of such information requires specific technical knowledge and, in advanced AI systems or with respect to ‘general purpose AI’ – systems not specifically programmed for an administrative task – the source code might give rise to understanding of the functioning of the system as such, but not to how a specific decision was made.
3.3. (c) Regulation of individual decision-making procedures in the context of ADM
The distinction between the general rules for the ADM element of a decision-making procedure and the single case decision making being based on that general rule, also have an effect on the legality of individual decision-making. Rights and principles governing decision making procedures and their protection will be assessed following individual decision making and thus might implicitly address the general normative level. But here also the question arises whether legal systems practically develop specific approaches to decision making by ADM, or – normatively speaking – should do so. In the alternative, one might ask whether it would not be sufficient to review decisions whether with ADM or without by the same standards in order to hold public decision making to account.
Both the interaction between data sources and ADM systems as well as between the programming of ADM systems and their application within decision making procedures turns the focus also on questions of inter-faces. How does a legal system understand and regulate the use of data and the selection of data by ADM systems? How, on the other hand, does a legal system govern the interface between the automated part of a decision-making procedure and the human input into decision-making? These interfaces can arise at various points during a procedure. At this point in time, at least, the use of ADM does not cover entire decision-making cycles from the initiation of a procedure over investigation, possible adoption of a measure to its implementation. Instead, ADM systems generally are employed in specific phases of decision making necessitating the creation of interfaces between human interventions and ADM or, as the case might be in the future, different ADM systems covering different phases of decision making.
Finally, questions important for subjecting to a comparative approach are matters of ensuring procedural rights of individuals including the right to a fair hearing, the right to access to one’s file, the right to a reasoned decision – the latter often related to the possibility of discerning whether there would be sufficient grounds to seek judicial review.
3.4. (d) Privatisation of regulatory tasks to be conducted by ADM
Privatisation of public functions is an important element of the regulation of ADM. The degree and limitations of private involvement are especially relevant in areas in which fundamental rights are to be balanced in decision making.
On the EU level, for example, limitations of what has amounted to a de-facto delegation of powers to balance fundamental rights in the pursuit of a public policy objective have been subject to CJEU case law from on the interpretation of elements of the EU’s IP Directive. The latter had conferred on private internet service providers liability for IP violations in the context of uploading of content. It was well understood that this task contained the balancing of fundamental rights including free speech and property. It was also understood that the internet service providers were able to dispose of this task only with the help of ADM systems. According to the CJEU, therefore, the obligation of the internet service providers obliged to balance fundamental rights positions had to be framed by legislative definition of basic elements of such balancing exercise and must contain sufficient guarantees to protect individual rights effectively.
The CJEU’s approach points towards first conceptual considerations for setting a framework for the conferral of tasks to private parties using ADM. Many more questions concerning private involvement in public obligations may arise. These will include matters of programming or servicing of software used by public authorities for ADM purposes by private parties. Additional questions which may serve as comparative factors include the question whether the data collections as well as the programming of information sources is predominantly undertaken by private parties or by public administrations themselves and which possibilities of oversight the public bodies have over such private activities. How to ensure that the systems which are in use do not discriminate against certain groups of people and criteria such as listed in Article 21 of the Charter of fundamental rights of the European Union are complied with in that they are not used as distinguishing factors.
4. Some tentative conclusions
The above outline sets out several research questions for comparative studies of ADM in public law. Jurisdictions within the EU and countries around the world are beginning to regulate the use of public automated decision making (ADM). The legal framework thereof differs considerably. Not surprisingly at this relatively early stage of development of legal reactions to the emerging technologies of ADM and in view of the fast-paced developments of ADM technologies themselves, answers to the challenges are developing in various legal systems. The fast-paced technological developments, especially in the available AI capabilities, result in novel regulatory questions. This makes for a compelling case for comparative learning and experimenting. The comparison of reactions in various legal systems functions similar to laboratories of experimentation for each-other. The interaction of legal systems in Europe’s multi-level legal systems arguably makes such comparative approach ever more important. However, in order to undertake a meaningful comparison, it is necessary to establish a framework of comparison in order to study and mutually compare various solutions developed on the national and supranational levels. This paper has suggested several avenues for a framework of comparison, therefore outlining fundamental topics along which legal systems’ reactions to the emergence of ADM with ever more sophisticated technological approaches can be studied and meaningfully compared. The framework of comparison thus serves as a mapping exercise of the main possible issues.
- Herwig C.H. Hofmann is Professor of European and Transnational Public Law at the University of Luxembourg, FDEF, and Head of its Department of Law. He is one of the co-founders of the Research Network on European Administrative Law (ReNEUAL) and is the principle investigator of the project “Information in the EU’s Digitalized Governance, INDIGO” (Norface Horizon 2020). I would like to thank Ms Laura Valtere for her valuable input into the development of this chapter. ↑
- E.g. Directive 2019/1024 of the European Parliament and Council of 20 June 2019 on open data and the re-use of public sector information (Open data directive, OJ 2019 L 172/56) which governs the re-use of existing documents held by public sector bodies and public undertakings of the Member States. ↑
- Annexes to the Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions. Foster in a European approach to Artificial Intelligence, pp. 49-50. ↑
- See for a conceptual development of various frameworks for comparative public law research, see H. C.H. Hofmann, Imagining Theoretical Frameworks, in: P. Cane, H. C.H. Hofmann, E. C. Ip and P. Lindseth (eds.) Oxford Handbook of Comparative Administrative Law , OUP, Oxford: 2021, pp. 1009-1026 with further references. ↑
- See e.g. European Commission Proposal for a Regulation of the EP and the Council laying down measures for a high level of public sector interoperability across the Union (Interoperable Europe Act) of 18 November 2022, COM(2022) 720 final 2022/0379 (COD). ↑
- See e.g. Regulation of the European Parliament and of the Council on European data governance (Data Governance Act) of 25 November 2020, COM(2020) 767 final 2020/0340(COD). ↑
- S. Demková, T. Quintel, Allocation of Responsibilities in Interoperable Information Exchanges: Effective Review Compromised?, in Cahiers Jean Monnet, 6, 2020, p. 589. ↑
- A large-scale information system for border management in operation in all EU Member States (with the exception of Ireland and Cyprus) and 4 associated countries (Switzerland, Norway, Liechtenstein and Iceland) based on Regulations (EU) 2018/1860-1862 (OJ 2018 L 312/1, 14 and 56). Other large-scale information systems exist for example in the areas regulating risk in food, animal feed, plant health (see Commission Implementing Regulation (EU) 2019/1715 (OJ 2019 L 261/ 37), human and veterinary medicine products (See with further references S. Demková, The Decisional Value of Information in European Semi-Automated Decision Making, in Review of European Administrative Law, 2, 2021, p. 29). ↑
- T. Quintel, Connecting Personal Data of Third Country Nationals: Interoperability of EU Databases in the Light of the CJEU’s Case Law on Data Retention, in University of Luxembourg Law Working Paper Series, n.2/2018, available at <https://ssrn.com/abstract=3132506 or http://dx.doi.org/10.2139/ssrn.3132506>. ↑
- Regulation (EU) 2018/1240 of the European Parliament and of the Council of 12 September 2018 establishing a European Travel Information and Authorisation System (ETIAS) and amending Regulations (EU) No 1077/2011, (EU) No 515/2014, (EU) 2016/399, (EU) 2016/1624 and (EU) 2017/2226, OJ L 236, 19.9.2018, p. 1–71, pursuant to which visa free Third Country Nationals (TCNs) have to apply for an electronic authorization in order for the risk they pose to be assessed in advance. ↑
- Directive (EU) 2016/681 of the European Parliament and of the Council of 27 April 2016 on the use of passenger name record (PNR) data for the prevention, detection, investigation and prosecution of terrorist offences and serious crime, OJ L 119, 4.5.2016, p. 132–149. ↑
- N. Vavoula, Consultation of EU Immigration Databases for Law Enforcement Purposes: A Privacy and Data Protection Assessment, in European Journal of Migration and Law, 2, 2022, pp. 145–146. ↑
- E.g. Travel, communications, banking and finance institutions face certain data retention obligations in order to allow for subsequent access of data by public authorities. See Joined Cases C-203/15 and C-698/15 Tele2 Sverige AB v Post- och telestyrelsen ECLI:EU:C:2016:970; C-698/15 Secretary of State for the Home Department v Tom Watson, Peter Brice, Geoffrey Lewis ECLI:EU:C:2016:970; C-511-520/18 La Quadrature du Net ECLI:EU:C:2020:791. ↑
- See e.g. C-511-520/18 La Quadrature du Net ECLI:EU:C:2020:791; Opinion 1/15 (EU-Canada PNR Agreement) EU:C:2017:592. ↑
- I am writing this contribution in January 2023. ↑
- C‑401/19 Republic of Poland v. European Parliament and Council of the European Union ECLI:EU:C:2022:297. ↑
- For example, see Article 17(4) of Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market and amending Directives 96/9/EC and 2001/29/EC, OJ 2019 L 130/92 under which “online content-sharing service providers shall be liable for unauthorised acts of communication to the public, including making available to the public, of copyright-protected works”. Internet service providers facing such potential liability undertake searches for IP protected content by ADM systems, thereby potentially affecting artistic freedoms, freedom of expression, freedom to conduct a business and other individual rights. ↑
- C‑401/19 Republic of Poland v. European Parliament and Council of the European Union ECLI:EU:C:2022:297, para 67 with reference to C‑311/18 Facebook Ireland and Schrems (Schrems II) EU:C:2020:559, para. 176. ↑