Responsibly Buying Artificial Intelligence: A ‘Regulatory Hallucination’
1. Introduction: ‘Confidently and Responsibly’ Buying AI
Much like other advanced economies, the UK seeks to position itself as a global superpower in Artificial Intelligence (AI) by 2030,[1] ‘to diffuse AI across the whole economy to drive the highest amount of economic and productivity growth due to AI’.[2] In the parallel race to AI regulation,[3] beyond its participation in the international efforts of, for example, the OECD,[4] the G7,[5] and the Council of Europe;[6] the UK aspires to global leadership in AI safety[7] and has invested significant effort in promoting the international summit that led to the Bletchley Declaration in November 2023.[8] This builds on the realisation that ‘without appropriate guardrails, this technology also poses significant risks in ways that do not respect national boundaries.’[9] The UK is thus trying to promote international collaboration to develop broadly consistent regulation to tackle AI risks, while simultaneously placing AI and its desired competitive advantage as the major plank for its post-Brexit economic strategy,[10] and seeing AI as an almost magical solution to the acute problems arising from the continued underfunding of its public services after decades of austerity. These goals are in tension, at least to the extent that light-touch regulation is considered necessary to facilitate AI-related innovation—which the UK government does.[11] Indeed, the government has adopted a light-touch regulatory strategy on the conviction that ‘[i]ntroducing binding measures too soon, even if highly targeted, could fail to effectively address risks, quickly become out of date, or stifle innovation and prevent people from across the UK from benefiting from AI’.[12] Such domestic regulatory strategy does not seem well aligned with international calls for (stronger) regulatory intervention.[13] Ultimately, then, the credibility of the UK’s claims to global leadership in AI safety hinges on the (internationally perceived) effectiveness of the UK’s domestic approach to AI regulation.[14]
In this article, I focus on the UK’s approach to regulating AI procurement and the use of AI in the public sector as a yardstick for the lack of credibility of its claim to global leadership in AI safety. I argue that UK policymaking is trapped in a ‘regulatory hallucination’. The UK Government has described hallucinations by generative AI as the fact that ‘AI systems regularly produce plausible yet incorrect answers and state these answers with high confidence’.[15] I borrow this description and use it as a metaphor to highlight how the UK Government’s current response to the challenge of regulating public sector AI procurement and use, although plausible and expressed with high confidence, is incorrect.
In 2021, the UK’s National AI Strategy[16] highlighted that government ‘has a role to play when it comes to the use of AI, both as a significant market pull in terms of public procurement… but also in terms of using the technology to solve big public policy challenges’.[17] Under this approach, public buyers are expected to ‘confidently and responsibly procure AI technologies for the benefit of citizens’,[18] and ‘government must lead from the front and set an example in the safe and ethical deployment of AI’.[19] The National AI Strategy ultimately expected the public sector to act as a responsible AI buyer and a role model in AI use. Earlier steps had suggested a commitment to supporting through guidance the process of AI adoption in the public sector,[20] and AI procurement in particular,[21] as well as a recognition of the need to facilitate accountability and contestability of public sector AI use through mandatory transparency.[22] However, following the publication of the National AI Strategy, progress stalled and the commitment to transparency was significantly watered down by rendering it voluntary for public sector users of algorithmic tools, including AI.[23] Although the most recent policy steer indicates that government will promote use of the algorithmic transparency recording standard,[24] this is far from pointing towards mandatory disclosure of public sector AI use.[25] There has also been no further review or development of the initial (thin) guidance on AI procurement. It seems as if the government considers that all that needed doing to ensure that the public sector can ‘confidently and responsibly procure AI technologies’ is already done. The Procurement Act 2023,[26] which represents a ‘once in a generation’ review of procurement law, contains no specific provisions on AI procurement. Amendments that sought to introduce them in the Procurement Bill[27] during its legislative passage were either not moved or defeated.[28] Public buyers are thus ultimately expected to self-regulate to ‘responsibly buy AI’, building on limited existing guidance,[29] and despite a growing gap in public sector digital skills.[30]
Similarly, despite seeking to actively leverage AI procurement as a tool of digital industrial policy,[31] and the existence of numerous challenges requiring regulatory attention,[32] the government eventually made it clear in its March 2023 AI White Paper[33] that it intends to stick to a ‘pro-innovation approach’ to AI regulation. This has been further confirmed in February 2024 following the public consultation.[34] No new rules or regulatory structures have been proposed—and no new constraints on the public sector’s AI adoption and deployment are forthcoming.[35] This is bound to perpetuate a situation where oversight of public sector digitalisation is both disperse across regulatory remits, and subject to oversight by inadequately resourced regulators—such as the Equality and Human Rights Commission.[36] If anything, existing constraints under the data protection regime may be further watered down,[37] thus also reducing the influence of the Information Commissioner’s Office.[38] Given the current gaps in existing legislation and regulatory structures,[39] in the absence of new rules (or with the watering down of existing ones), public sector AI users are thus also expected to largely self-regulate to ‘set an example in the safe and ethical deployment of AI’. The need to self-regulate the use of AI compounds, or at least directly influences, the self-regulation of the procurement of those technologies.
This light-touch approach to the regulation of public sector digitalisation is risky.[40] Whether it will be successful is a high-stakes bet, as AI and other digital technologies quickly penetrate most areas of public sector activity and the provision of public services—with 70% of respondents to a recent survey by the National Audit Office currently piloting or planning AI use cases, and all government departments required to create AI adoption plans by June 2024.[41] The government is investing at least £8bn in digital, data and technology transformation between 2022 and 2025.[42] The Chancellor has also told ministers to quicken AI adoption to boost the economy,[43] and technology is being considered as a ‘fix’ for many a challenge in crucial public services such as the National Health Service (NHS),[44] or education.[45] Enormous public resources will thus be invested in AI and other digital technologies in the short and medium term. If procurement is not capable of ‘confidently and responsibly’ acquiring AI, and of ensuring that the public sector is a role model user, many problems, risks, and harms loom in the horizon.
I argue that, unfortunately, unless this regulatory strategy is abandoned and an alternative regulatory strategy is promptly implemented, those problems, risks, and harms will materialise. Continuing with the pretence that public buyers can ‘confidently and responsibly procure AI technologies for the benefit of citizens’ is an unjustified ‘regulatory hallucination’ that can generate individual harms and broader negative social effects as the public sector ramps up AI adoption and accumulates a potentially significant stock of AI deployments in a wide variety of areas of public sector activity. Importantly, these significant risks are exacerbated by two factors that put significant pressure on the public byer ‘getting it right’ the first time around. First, the procurement activity underpinning public sector digitalisation is unlikely to be subjected to demanding standards of judicial review in relation to the technical characteristics or attributes of the digital technologies being procured, or their potential effects. Judicial review of procurement decisions involves deference to the exercise of technical discretion by public buyers.[46] Relatedly, judicial approaches to the review of algorithmic automation are severely under-developed and constrained[47] and, even where the courts engage with mechanisms for the assessment of potential impacts (e.g., under data protection rules),[48] their approach is highly formalistic and entirely reliant on the existence of specific procedures for the carrying out of such assessments.[49] Although there is a possibility that the UK courts could take a ‘hard stance’ to the review of AI procurement and/or AI deployment by the public sector, this is but a guaranteed development. And even then, and second, public sector digitalisation risks creating mass harms that can immediately arise from the AI deployment, without traditional (ex post) judicial review mechanisms and remedies being suited for the timely (prevention or) correction of downstream effects.[50] This means that, even a stringent approach to judicial review would most likely provide ‘too little, too late’ by way of protection of individual rights and collective interests. This further justifies the need for an alternative anticipatory regulatory approach, as public buyers are very unlikely to be able to effectively steer public sector digitalisation towards adequate compliance with the overarching regulatory goals.
To support this view, this article shows that the public buyer is inadequately placed to take on the role of public sector digital gatekeeper and (self-)regulator. After this introduction, the article conceptualises the UK’s light-touch approach to regulating public sector digitalisation as a strategy of ‘regulation by contract’, or contract private ordering (Section 2). It then highlights the two main shortcomings of this regulatory strategy. First, the ‘two-sided gatekeeping’ role required to ensure that the public sector is a responsible buyer and a role model user of AI undermines procurement’s position as a regulator (Section 3). Second, the ‘regulatory tunnelling’ arising from procurement’s reliance on industry-led standards, and the risks of capture and commercial determination arising from skills and power imbalances within (to be expected) procurement negotiations, hollow out the public contract as regulatory instrument (Section 4). The article then sketches the need for an alternative regulatory strategy based on the creation of the ‘AI in the Public Sector Authority’ (Section 5). The article concludes that the credibility of the UK’s claims to global leadership in AI regulation is significantly undermined by the weakness of its domestic approach, which is trapped in a ‘regulatory hallucination’; only a change of regulatory tack at home could strengthen the UK’s position in the global race to AI regulation (Section 6).
The article focuses on the regulatory strategy and environment in the UK but, given the (perhaps surprising) commonality in regulatory approach between the UK and the EU, most of the analysis is also relevant in the broader European context.[51] This is perhaps not as much the case of the US, where more active steps are being taken to promote responsible AI procurement and use in the public sector.[52] However, embedding those changes will take time and the analysis here can be relevant as a reminder of the negative effects that could follow from a reversal of the nascent policies and strategies.
2. ‘Regulation By Contract’ as the Underlying Strategy
In the absence of general AI regulation setting substantive requirements for the development and deployment of these technologies,[53] the seemingly simple proposition that public buyers should ‘confidently and responsibly procure AI technologies for the benefit of citizens’ requires some unpacking and conceptualisation. A first question arises in relation to what ‘confidently’ and ‘responsibly’ mean here. I argue that ‘confidently’ refers to effectively, in the sense of the public buyer being able to put in place effective mechanisms to ensure responsibility in AI procurement and use. As to ‘responsibly’, I would take it to mean compliance with the principles of AI regulation formulated in the AI White Paper,[54] where the government stated that its regulatory approach is underpinned by five principles to guide and inform the responsible development and use of AI in all sectors of the economy:
Safety, security[,] and robustness
Appropriate transparency and explainability
Fairness
Accountability and governance
Contestability and redress.[55]
In this context, ensuring responsible AI procurement hinges on the public buyer’s ability to determine and enforce substantive requirements to operationalise those higher-level principles through the design of public tenders and contracts. In other words, it requires the public buyer to leverage public tenders and contracts as regulatory tools—or, more simply, to engage in AI ‘regulation by contract’.[56]
The use of procurement as a means of regulation by contract is not new. There is a long tradition in the context of the outsourcing of public services, or the use of procurement to promote environmental and social goals.[57] It could thus seem that relying on procurement to regulate AI by contract is a continuation of a tried and tested approach likely to lead to positive outcomes. However, this would be too simplistic an assessment. First, because the use of procurement as a tool of regulation by contract has proven problematic in those contexts.[58] Second, and perhaps more importantly, because there are two primary contextual presumptions that make ‘regulation by contract’ in the digital context particularly challenging.
At its simplest, the use of procurement as a tool of regulation by contract presumes that the public buyer (and the public sector more broadly) has a clearly defined goal that is at odds with the interests of the private provider[59] (e.g., because it has cost implications), and that such goal can be imposed through explicit contractual obligations binding the private provider. The creation of those obligations, the monitoring of compliance therewith, and the imposition of contractual sanctions for unacceptable deviations would empower the public buyer to ensure the desired regulatory outcomes. The presumption is thus that only the behaviour of the private provider needs disciplining through the contract, and that the public buyer holds a position of superiority that enables it to do so. Relatedly, such an approach also presumes that the provider has no influence in establishing the contractual obligations or determining how to achieve them, as it is seen as a ‘rule-taker’ that must comply with the contractual requirements defined by the public buyer as the ‘rule-maker’.
Even at this level of simplification, the logic underpinning this regulatory use of procurement shows that this is a strategy unlikely to deliver positive outcomes where not only the behaviour of the private provider, but also that of the public sector user of the technology, needs to be regulated by contract—because the contract as regulatory instrument does not hold any particularly salient properties as an adequate mechanism to bind the public sector to the pursuit of (external) regulatory goals. Relatedly, this is likely to prove a weak regulatory strategy where the private provider can influence the setting, monitoring, or sanctioning of its own contractual conduct (e.g., due to higher levels of technology skills and/or commercial acumen). I argue that these weaknesses are particularly challenging in the digital context, where both presumptions underpinning the logic of regulation by contract are significantly eroded—if not outright rebutted in practice.
The salience of the challenges in the digital context is perhaps best exemplified by contrast to relatively simpler uses of regulation by contract. Take the example of the use of procurement to enforce wage requirements beyond prescriptive legal rules.[60] If the government wants to ensure that the public sector is a ‘responsible employer’, it can adopt a policy requiring public buyers to ensure compliance with a wage level above any existing legal minimum by demanding that private providers pay such enhanced wages to workers involved in the delivery of the contract. In that case, the interest of the public sector is clearly defined and relatively easy for the public buyer to translate into a contractual requirement, which is also relatively easy to monitor (and sanction). It also seems that the public entity buying the goods or services delivered through such contracts has limited reasons to disagree with the policy (other than in relation to funding and costs), or seek to deviate from it.
Now, by contrast, take the example of using procurement to make sure that facial recognition AI solutions used by the police,[61] Home Office,[62] or intelligence services[63] are fair and explainable. Let us take the example of the Home Office. First, it is unclear that the Home Office will be accepting of the need to ensure (certain conceptions of) fairness, or sufficiently committed to ensuring explainability, at least where prioritising them clashes with the (expected) functionality of facial recognition solutions (i.e., where more explainable AI performs worse than ‘black-box’ AI). In this context, the role of the Home Office’s procurement department (or procurement function[64]) in seeking to operationalise regulatory principles will be severely under strain. Imagine the procurement department—which is a back office department dependent of the commercial director, which is itself dependent of a director general—seeking to push back against the adoption of a (specific) facial recognition AI solution for a specific use, on the basis that it would breach regulatory goals such as fairness. How likely is it that such pushback would prevail where, for example, the relevant ‘operational’ or front-line department, the relevant minister, or the Home Secretary, had made a decision (and public promises) on the ways the AI would improve Home Office operations? Would an external centralised procurement function, such as the Crown Commercial Service,[65] fare any better in that regard? What levers could the internal or external procurement function use to oppose facial recognition use? I argue that there are no effective levers in that regard, which means that pinning our hopes on a regulatory role for procurement is unwarranted. Second and relatedly, even if the Home Office was committed to such general regulatory goal and the procurement department was allowed to try to design the tender and the future contract in ways that embedded safeguards—those would in any case likely make reference to ‘state-of-the-art’ benchmarks, or to technical standards heavily (or solely) influenced by the tech industry.[66] Given the open-endedness and technical complexity of the issues subject to contractual regulation (by reference to standards), and that these are unlikely to be ‘off-the-shelf’ contracts and will thus require negotiations, it is unclear that the tech provider/s would not be able to determine or heavily steer the contractual specification (e.g., as part of contract negotiations), and/or the monitoring of compliance with the relevant contractual obligations (which would likely operate on the basis of self-assessment). On the contrary, it seems clear that, especially if the Home Office procurement department had more limited digital skills than the facial recognition technology providers willing to tender for the contract—which it most probably does[67]—the contracts would end up reflecting goals and metrics with which the tech providers felt comfortable—which are unlikely to (or at least cannot be presumed to) fully align with the overarching regulatory goals. Once again, this implies that pinning our hopes on a regulatory role for procurement is unwarranted.
In this context, and if nothing else by comparison, it seems clear that the reliability of both the presumption that the public sector entity ultimately using the AI is committed to the regulatory goal, and the presumption that the private provider is a rule-taker, cannot without more be taken to apply. This is further developed in the next two sections.
3. ‘Two-Sided Gatekeeping’ Undermines Procurement’s Position as Regulator
The first presumption requiring deeper analysis is that AI regulation by contract can discipline the behaviour of all parties influencing whether the procured AI is capable of being used in a responsible manner, and effectively used in that way. In that regard, it is useful to revisit the logic underpinning the presumption, which stems from agency theory.[68] In simple terms, regulation by contract is seen as a mechanism whereby the public buyer (as principal) imposes regulatory requirements on the private provider (as its agent). This relies on a presumption of public buyer superiority, as well as the related presumption that, once it has set the relevant requirements, the public buyer monitors and controls the behaviour of the private provider throughout the life cycle of the contract and can sanction any deviations from the imposed obligations. Neither of these presumptions necessarily or easily holds true in the digital context. First, it is necessary to acknowledge the overlap of agency relationships that arise between the public sector entity seeking to adopt AI, the public buyer, and the tech provider—as this will bring to light the limitations of the strategy of regulation by contract in relation to binding the user public sector entity to goals of AI regulation. Second, it is necessary to acknowledge that the contract has limited effectiveness as a regulatory mechanism where the interests of the user public entity and those of the tech provider align in ways that deviate from the goals of AI regulation—as this will stress the need for independent oversight. The remainder of this section further unpacks the public–public (Section A) and decentred (Section B) interactions involved in the procurement and use of AI.
A. Public–Public Interactions, Capture, and Procurement’s Institutional Embeddedness
i. Public–Public Interactions
The presumed position of superiority of the public buyer over the private provider needs to be assessed in the broader context of the interaction between the public sector entity seeking to use the AI and the public buyer—as public buyers generally do not procure AI for their own use, but rather for a use specified by the public sector entity in charge of the relevant public service or activity. In this context, and following agency theory, the public buyer acts as the agent of the adopting public entity as a higher-level principal. In this understanding, agency theory places the public buyer in a subordinated position—which explains the governance challenges in aligning the agent’s incentives to the principal’s goals,[69] as well as the governance difficulties that arise from any misalignment between the (policy) interests of principal and agent.[70]
From a regulatory perspective, the overlay of agency relationships would not be a problem where the public sector entity and the public buyer shared the relevant goals. The goals of the public sector entity imposed on the public buyer as agent would simply be cascaded down to the tech provider as second-tier agent. However, where the user public sector entity and the public buyer do not share the relevant goals, the picture is much more complicated. The complexity in the context of AI procurement is that the regulatory role assigned to the agent (the public buyer) unavoidably encompasses ensuring adherence to regulatory goals and disciplining (future) aspects of public sector behaviour and decision-making by its principal (the adopting public entity)—for example, in relation to adherence with the principles of AI regulation discussed above, or decisions on data governance or technology management frameworks related to the AI deployment.[71] Focusing on the public buyer as the guarantor that the procured AI will be responsibly used would require placing the public buyer in a position of (functional) superiority over the user public sector entity, in that it would (be expected to) be able to dictate (some of) the terms of the technological adoption. Thus, regulatory gatekeeping requires a reversal of positions within the vertical relationship between the adopting public entity and the public buyer, and flips agency theory on its head.[72]
Ultimately, both public–public interactions between the public buyer and the adopting public entity are conceived in a vertical fashion, but agency and gatekeeping frameworks operate on an opposite understanding of which actor holds a predominant position. Under agency theory, the public buyer is the agent and thus subject to the instructions of the public entity that will ultimately adopt the AI. The public buyer is thus mainly tasked with procuring what the entity requires for its operations in the best possible market terms. Conversely, under gatekeeping theory, the public buyer is the (independent) guarantor of a set of goals or attributes in public sector digitalisation projects and is thus tasked with ensuring compliance therewith. This could require, for example, directing the public entity to revisit and modify, or even abandon, a digitalisation project where the planned AI implementation falls short of ensuring compliance with the required goals and attributes.
This generates structural tensions, conflicts of interest, and governance challenges that procurement rules are not well-prepared for—as procurement law is not designed as an internal tool of public sector governance. It is unclear, for example, whether the procurement function can query (or stop) a digitalisation project, for example, because of insufficient adherence to the goals of AI regulation discussed above, the inadequate content of the impact assessments carried out by the adopting public sector entity, or on the basis of the inadequate management of other digital tools already deployed. The public–public relationship between the adopting public entity and the public function is not regulated by the procurement rules,[73] which places the procurement function in an awkward position when tasked with ensuring compliance with specific standards or expectations by public sector actors for which it is simultaneously expected to function as agent.
ii. The Adopting Public Sector Entity as a Potentially Captured Principal
The potential conflict between the procurement function and the adopting public sector entity is usually not acknowledged as a regulatory challenge for the effective deployment of ‘regulation by contract’ strategies. This could result from a presumption that, as the goals of digital regulation are concerned, both actors are in full alignment because, after all, they are both embedded within the public sector and should thus share a commitment to such goals. However, this is a problematic presumption because existing theories of governance—and in particular garbage can theory[74]—show that the adopting public sector entity is particularly exposed to risks that would trigger a deviation from such regulatory goals, due to the allure and ‘policy irresistibility’ that digital technologies generate in a context of policy- and decision-making characterised by top-down pressures, reinforcing effects between different policy strands in the digitalisation of the public sector, and limited institutional capacity.[75] Where the technology is presented as (and perceived to be) the solution to (persistent) organisational problems, the absence of robust governance structures exposes the organisation to hype-driven claims that can distort its decision-making processes. Combined with insights from policy entrepreneurship[76] and regulatory capture theory,[77] this sub-section highlights governance risks in this setting that underpin the potential for conflict between the procurement function and the adopting public sector entity—especially where the public sector entity ends up being captured by internal or external tech (policy) entrepreneurs.
Garbage can theory explains the institutional dynamics of ‘organised anarchies’ and evidences how organisational decision-making, even on high-stake issues, is shaped by the interaction of a series of elements that leave the organisation particularly exposed to capture by key participants in decision-making processes at the relevant time. This does not contribute to consistency in decision-making through time, or in relation to the overarching institutional goals, as each decision comes to be made under a relatively chaotic or anarchical dynamic. ‘Organised anarchies’ are organisations with a combination of: (1) problematic preferences, that is, ‘a variety of inconsistent and ill-defined preferences’[78] whereby the organisation ‘discovers preferences through action more than it acts on the basis of preferences’;[79] (2) unclear technology, whereby the organisation ‘operates on the basis of simple trial-and-error procedures, the residue of learning from the accidents of past experience, and pragmatic inventions of necessity’;[80] and (3) fluid participation, whereby participants in decision-making ‘vary in the time and effort they devote to different domains’[81] and change capriciously for any particular kind of choice. I submit that most public sector institutions, and especially those with a policymaking remit related to public sector digitalisation,[82] are examples of such organised anarchy. This is due to the complexity and lack of clarity on how to balance competing goals, the lack of clarity on how public sector digitalisation should proceed, as well as the fluid participation of several actors in multi-level governance settings.
The ‘garbage can process is one in which problems, solutions and participants move from one choice opportunity to another in such a way that the nature of choice, the time it takes, and the problems it solves all depend on a relatively complicated intermeshing of elements’;[83] where ‘decision-making in the public sector … reflects the serendipitous, and almost accidental, confluence of streams of problems, solutions, opportunities and actors’.[84] In other words, it is a model of decision-making where ‘solutions go looking for problems with which to link’.[85] As extended to policymaking, the model stresses the importance of ‘windows of opportunity’ for policy formulation and implementation, when there is alignment between the parallel movements of problems and solutions.[86] When those policy windows open, ‘policy entrepreneurs must be prepared to exploit the opportunities’.[87]
Relatedly, such policy entrepreneurs can be either internal or external. First, where a window of opportunity is seized by an internal policy entrepreneur, there is a risk that the views of that individual (or small group) will determine the decisions of the institution. In the context of digitalisation, there is a clear risk that decisions are in this way determined by those with higher levels of digital expertise, which may hold idiosyncratic views on the overarching regulatory goals and their importance. Given the specific motivations of those policy entrepreneurs, there is scope for conflict between such motivations and the overarching regulatory goals—for example, a results-driven policy entrepreneur could minimise the relevance of regulatory constraints that affected the technical performance of a given AI deployment, or more simply ‘cut corners’ to deliver a specific functionality. Second, windows of opportunity can be exploited by external agents, and especially by tech providers seeking to be awarded public contracts for the provision of AI solutions. Where the potential provider manages to persuade the public sector entity that its solution is the best that can do what is required, or that it is the only possible solution, it is clear to see scope for clashes between the public sector entity and the procurement function. There are also more subtle ways in which the public sector entity can be captured to steer its decision-making towards specific forms of digitalisation.
Therefore, there is significant scope for deviation from the overarching regulatory goals by the public sector entity, either as a result of policy entrepreneurship, regulatory capture, or other sources of friction (e.g., operational requirements, budgetary constraints, etc). This stresses the need for the procurement function to not solely discipline the behaviour of the tech provider, but also that of the adopting public sector entity. In other words, this stresses that the regulatory gatekeeping is challenging in relation to public–public interactions, as much as in public–private interactions.
iii. Procurement’s Institutional Embeddedness
At this point, it is important to consider the institutional embeddedness of the procurement function. The challenges outlined in the previous two sub-sections can be exacerbated by organisational structures. Despite having considered the public buyer as a separate agent for the purpose of isolating its regulatory role in the discussion so far, in reality, it is hardly ever the case that procurement is carried out by an entirely separate organisation. Procurement is usually carried out by a unit or team within the public entity seeking to adopt the technology, which places it within a hierarchical organisation and thus with limited (functional) independence.[88] As a result of the process of procurement centralisation, or in the context of collaborative procurement, it is possible for procurement to be carried out by a separate organisation.[89] However, in that case, the organisational relationship will still be highly dependent on the interests of the adopting public entity. In the case of collaborative procurement, the governance of the arrangement will generally be subject to veto by the adopting public entity. In the case of centralised procurement, barring the mandatory use of centralised frameworks or other procurement vehicles, the adopting entity retains the decision whether to partake in any centralised or collaborative procurement procedure. And, in any case, it generally retains discretion for the organisation of relevant aspects of the tender (call-offs or mini-competitions) and, possibly, in the negotiation of specific aspects of the resulting public contract. This reduces the possibility for the centralised or collaborative procurement function to dictate the terms to the adopting public entity,[90] which generally retains discretion to conduct a separate procurement or, at the very least, to tailor the use of the centralised tools.
The implication of such institutional embeddedness is that, where the procurement function sits within the same administrative unit or organisation as the adopting entity, the gatekeeping role will be substantially affected and potentially jeopardised, if not neutralised, by the hierarchical functioning of that unit—and it can well be that the gatekeeping role is thus driven by policy entrepreneurship or commercial capture. In other words, where the procurement function is embedded within the organisation that will use the AI, there is no meaningful chance for the inversion of the principal/agent position required to enable procurement to act as an effective regulatory gatekeeper; the lack of independence of the procurement function undermines the integrity and likely effectiveness of the gatekeeping. Similarly, where the procurement function is detached from the adopting public entity—as in the case of a central purchasing body or collaborative procurement—the discretion retained by the adopting entity and competing commercial interests can erode the integrity of the gatekeeping function, even if a lower level of dependence can be expected. Simultaneously, other institutional aspects (e.g., centralisation and thus standardisation of requirements) can also reduce the effectiveness of the gatekeeping through blanket measures incapable of sufficient adaptation to the case at hand. This can be particularly relevant in the process of AI adoption, given the potentially very high levels of customisation required at the point of deployment for a wide variety of AI solutions.
An additional consideration is that the institutional embeddedness of the procurement function needs to be understood in its decentralised context. There is no single procurement function or public buyer, but rather a multiplicity of them, embedded within different institutional contexts and, importantly, with large variations in terms of (digital) capacity, experience, specialisation, inter- and intra-institutional dynamics, etc. This raises further questions on the ability of the procurement function to discipline the behaviour of its principal in any given institutional setting, as it will be highly context dependent.
This institutional reality evidences the breakdown in regulatory logic mentioned earlier. The lack of functional independence and the atomisation of the procurement function raise important questions on the likely effectiveness of the approach to regulate public sector digitalisation through procurement, at least as the public sector side of the transaction is concerned. Overall, despite the common vertical conceptualisation of public–public interactions, there is a structural tension between agency and gatekeeping approaches in this setting. Such tension also needs to be considered in relation to decentred interactions between the adopting entity and the tech provider, as below.
B. Decentred Interactions
Indeed, a further complication from the perspective of the regulatory logic of procurement as a tool of regulation by contract is that it presumes the centrality (or at least the direct participation) of the public buyer in the relevant public–private and public–public interactions. The procurement function is conceptualised as the principal, agent, or gatekeeper (or given multiple roles) and is always present in the understanding of the interactions with the tech provider that underpins the relevant rules. However, in leveraging procurement to regulate by contract public sector AI adoption, the system would be creating decentred (public–private) interactions. In those, the public sector AI user and the tech provider would be jointly shaping the effective deployment of the technological solutions in a way that cannot be monitored or adequately influenced by the procurement function, or by existing mechanisms of procurement oversight and remedies.
This is not only the direct result of the impossibility of regulating technological development and deployment through rigid rules and the unavoidable use of open-ended and/or under-specified standards (to varying degrees).[91] It is also the result of the impossibility to force commitment to a strict (or demanding) interpretation of those standards by both the adopting entity and the tech provider during the contract implementation phase, coupled with the retraction or entire disappearance of the procurement function from that contractual regulatory space. Usually, the involvement of the procurement function ceases with the award of the contract. And, even if there is ongoing involvement of the procurement function in contract execution due to the specific organisation of the relevant public sector entity, conflicting (operational and commercial) interests are likely to condition or preclude effective continuous oversight from a gatekeeping perspective.[92] Given the adopting entity’s interest in the deployment of the technology, it (and a procurement function embedded within it) is not in an adequate position to effectively perform a gatekeeping role.
This decentring of decision-making creates a significant element of uncertainty (or rather, a moving target) for the procurement function, as regulatory gatekeeper, at the point of choosing specific tools to foster compliance with the regulatory contract. As specific goals or obligations can straddle the public/private divide, it may not be easy, or possible, to identify the appropriate setting or mechanism to regulate them. In the specific context of AI adoption, the procurement function may be unable, at the point of designing the contract for the development and deployment of the technology, to ascertain how the adopting entity will make decisions with a significant bearing on the outcome of the process of digitalisation and its compliance (or not) with the desired attributes and goals underpinning the gatekeeping function. This will be the case where, for example, the procurement is launched at an early stage of exploration of technological alternatives. Notably, it will also be the case in all, common, instances where the deployment of the technology requires significant tailoring to the specific context and needs of the adopting entity, with crucial decisions on, for instance, transparency or explainability left to later stages or, potentially, to decisions of the tech provider.[93]
In theory, the procurement process could result in a contract that imposed obligations on both the adopting entity and the tech provider—indeed, the uncertainty mentioned above would likely push for a rigid contractual design. However, in practice, it will not be possible to come up with operationally adequate strict rules and the (contractualised) regulation of the technology will need to remain standards-based (as further discussed below in Section 4). This implies that compliance with such obligations will depend on the willingness of the adopting entity and the tech provider to abide by the contract as interpreted or expected by the public buyer (as regulator and gatekeeper). This is not necessarily a given outcome.
It is indeed by no means clear that the adopting public entity will have the right incentives (or resources) to adhere to the overarching regulatory goals and the desirable goals and attributes of the technological deployment, or that the tech provider will have the incentives (or resources) to force such adherence on the part of the adopting public entity.[94] Moreover, even if the adopting entity was in principle committed to the relevant (contractualised) regulatory goals, its behaviour can be (partly or totally) influenced, determined, or functionally delegated to the tech provider. This is specially the case where compliance with the regulatory goals depends on decisions to be taken during the execution of the public contract (e.g., on the specific mechanisms to operationalise the explainability requirements of the specific technical solution chosen to perform the contract), or depend on self-assessments by the tech provider.
Further, it is possible that both the adopting entity and the tech provider find it in their common interest to ignore or bypass the contractual obligations and deviate from the regulatory goals (e.g., through the implementation of a less than fully explainable technological solution), in which case the question arises of who would have the incentives and effective possibility of enforcing the contractual terms on both the adopting public entity and the tech provider—especially where transparency of the use of AI or the underlying public contract is limited or non-existent. This is a rather different enforcement challenge than that of the ‘standard’ understanding of the use of ‘regulation by contract’, as the latter presumes that the interests of the contracting parties are opposed, and thus not usually susceptible of alignment contrary to contractual terms, because those terms have been imposed by the principal or result from a common approach under relational understandings of contracting.[95]
This points to a difficult interaction between the tools that the public buyer can deploy to try to impose specific behaviour on the tech provider through the public contract, and the unregulated and institutionally driven nature of the public–public interaction, which can leave the procurement function with no means to effectively influence decision-making by the adopting public entity once the procurement is completed. This is important because the adopting entity is bound to become the principal of the regulatory contract. Differently from other types of ‘regulation by contract’ where the obligations are solely incumbent upon the private provider as agent, in this setting the obligations would (also) be incumbent on the contractual principal and thus, potentially seen as an unwanted constraint on the way it digitalises (and thus, carries out) its operations.[96]
C. Recapitulation: ‘Two-Sided Gatekeeping’ as a Major Obstacle
By placing the focus on the multiple public and private interactions that take place in the regulation by contract of public sector AI acquisition and deployment, the analysis in Section 3 has shown how the emergent approach creates what can be labelled as a ‘two-sided gatekeeping’ challenge. This challenge encapsulates the difficulties for the procurement function to effectively influence regulatory outcomes where it needs to discipline both the behaviour of tech providers and adopting entities in the public sector, and where contract implementation depends on the decentred interaction of those two actors with the procurement function as a (toothless) bystander. To recapitulate, this challenge stems from two main presumptions within the regulatory logic that do not fit with the need for the procurement function to simultaneously align the behaviour of the relevant public and private actors.
First, procurement is presumed to hold a position of superiority in the vertical relationship with market agents that allows it to dictate the terms of tender procedures and to impose contractual obligations to regulate (future) behaviour.[97] However, this does not align with the position of subordination of procurement as agent of the public entity seeking to adopt a given digital technology. In this also vertical relationship, the subordination of procurement as agent runs counter to the functional need for superiority (or at least independence) in the exercise of the gatekeeping function. The unregulated nature of the relationship between the procurement function and the adopting public entity raises questions on the effectiveness of the gatekeeping role, which are exacerbated by the institutional embeddedness of the procurement function.
Second, the timing of intervention and the toolkit available to the procurement function are inadequate to regulate by contract decentred interactions between adopting public sector entity and tech provider. Given that both actors can have convergent incentives to deviate from the terms of the regulatory contract designed by the procurement function, there are also questions about the effectiveness of the gatekeeping function in this two-sided setting. Not least because the potential impossibility of disciplining the adopting entity can nullify the efforts to embed contractual obligations in the relevant contract when that entity replaces the procurement function and becomes the contractual principal. In the absence of an effective external enforcement mechanism, it is highly problematic that the contractual relationship is established between the two entities whose behaviour it seeks to discipline. The contract is not a self-executing regulatory mechanism. Leaving its implementation to the parties that can benefit from (surreptitiously) deviating from it is a risky regulatory approach, to say the least.
Functionally, then, the emerging approach cannot properly operate where the adopting public entity is not entirely committed to maximising the goals of digital regulation that are meant to be enforced by contract, and where the tech provider has a concurring interest in deviating from those goals by reducing the level of demand of the relevant contractual clauses.[98] In the setting of AI regulation, this seems a likely common case, especially considering that the main regulatory goals (e.g., explainability, fairness) are open-ended, and thus the question is not whether the goals in themselves are embraced in abstracto by the adopting entity and the tech provider, but the extent to which effective (and costly or limiting) measures are put in place to maximise the realisation of such goals.
Overall, this shows that gatekeeping and ‘regulation by contract’ may only be an effective approach where the procurement screening and contractual obligations are aimed at disciplining the behaviour of the tech provider, or to ensure compliance with external legal requirements that bind the tech provider or/and the adopting public sector entity. In such case, the role of the procurement function and the role of the principal under the contract are aligned towards compliance with the legal obligations, and so the gatekeeping role can straddle the public/private and the tendering/contractual implementation divides. However, this does not necessarily hold where there is a potential tension between the roles and incentives of the procurement function (as regulatory gatekeeper) and the adopting entity as the principal under the regulatory contract. It also does not necessarily hold where the contract cannot set rules but rather needs to rely on standards, as discussed in the next section.
4. Regulatory Tunnelling and Commercial Determination Hollow Out the Public Contract as Regulatory Instrument
The second presumption requiring deeper analysis is that AI regulation by contract creates a situation where the public sector is the rule-maker and tech providers are the rule-takers—so that the public sector can use contractual obligations to impose/ensure compliance with the principles of AI regulation. The logic of regulation by contract indeed requires control over the regulatory goals and the means for their delivery to remain with the public sector as regulator. This will not be the case where the regulatory contract does not set the relevant substantive requirements, but rather refers to external sets of (privately established) standards—at least to the extent that such standards deviate from the desired regulatory goals. This will also not be the case where tech providers are able to determine or heavily steer the specification and monitoring of compliance with the relevant contractual obligations, either due to bargaining power arising from their market position or superior technical skills, or through the disguise of normative choices as matters for technical judgement. This section will further explore how the public buyer’s regulatory gatekeeping role is undermined by its reliance on industry-led standards, which generates risks of regulatory tunnelling (Section A),[99] and by the tech providers’ upper hand in setting contractual benchmarks and their control over the ensuing self-assessments, which generates risks of commercial determination (Section B).[100]
A. Regulatory Tunnelling Through Reliance On (Technical) Standards
There is currently no legislative benchmark establishing specific requirements in relation to the procurement and use of AI by the public sector. There is also no guidance on what the principles of AI regulation imply or how they are meant to be operationalised.[101] This requires the public buyer tasked with ‘confidently and responsibly procuring AI technologies for the benefit of citizens’ to determine those requirements for the purposes of selecting tech providers and choosing specific AI solutions.
There is very little guidance on how the public buyer is meant to do so and the guidance that exists is so generic and lacking in detail that a public buyer cannot operationalise it without significant other sources of knowledge, or access to significant levels of digital skills and competences.[102] Crucially, the Guidelines for AI procurement[103] simply refer to the need to (1) develop a plan for governance and information assurance that adheres ‘to the Tech Code of Practice[104] and Government Design Principles,[105] the Data Ethics Framework[106]and other relevant standards’;[107] (2) avoid ‘Black Box algorithms and vendor lock in’;[108] and (3) ‘address technical and ethical limitations of AI deployment during … evaluation’[109]—which requires considering ‘whether the appropriate technical standards [have] been adhered to’.[110] The guidance thus points towards the need to rely on externally established (technical) standards to specify the requirements of the AI technology to be procured. However, there is no indication on how the public buyer is meant to assess the suitability of those standards to operationalise the general goals and principles of AI regulation.[111] The main issue here is, ultimately, that the seemingly large volume of guidance that has been issued does not provide any meaningful steer and does not consolidate any (possibly) existing institutional memory and knowledge of how to procure technology, leaving each public buyer to reinvent the wheel on each occasion. As it is, the existing guidance is not more than a funnel towards the (emerging) private standards,without providing any implementable approaches on how to judge their adequacy for any given procurement.
Ultimately, this creates (a risk of) regulatory tunnelling whereby all relevant decisions on how to interpret a given goal (e.g., fairness or explainability) and how to translate its requirements into technical and organisational arrangements within the regulatory contract hinge on decisions not taken by the public buyer or the broader public sector, but by industry-led standardisation organisations (where they exist) or simply by the tech industry where there is no ‘official’ standard and the relevant benchmark arises from commonly accepted ‘best practices’ or the state-of-the-art. And, differently from other areas where industry standards can (in principle) provide adequate safeguards because the relevant sources of risks can be objectively measured and tested, AI regulation requires engagement with incommensurable issues that procurement is ill-equipped to operationalise.[112]
Worryingly—at least from the perspective of the logic underpinning the use of regulation by contract as a regulatory strategy—this offloading of regulatory power onto industry-set or industry-led standards is not an accident, but rather a core feature of the UK’s current ‘pro-innovation’ approach to AI regulation. The AI White Paper reflects the same regulatory technique. There is meant to be no general legislative benchmark establishing specific requirements to operationalise the principles of AI regulation.[113] Regulators are meant to ‘lead the implementation of the framework, for example, by issuing guidance on best practice for adherence to these principles’[114] within their regulatory remits. The AI White Paper contains some additional guidance on the implementation of the principles by regulators,[115] and there is some additional guidance for regulators on how to develop their own guidance and implement this regulatory approach.[116] However, all guidance ends up pointing to standards. For all principles, the government expects regulators to ‘consider the role of available technical standards addressing’ the relevant goal—and explicitly refers to IEEE and ISO/IEC standards (some of them still under development) that regulators should take into consideration. This creates a direct risk of (automatic) adoption of those standards by regulators as part of their guidance, or at the very least for those standards to provide the basis for the regulatory guidance—which can have anchoring effects.[117] This is problematic because those standards have an inherent pro-industry bias[118]—while it is unclear to what extent regulators will be able to scrutinise in detail the implications of those standards from a public interest perspective and, where appropriate, deviate from them or set them aside.
By the same token, and given much more limited technical capabilities than those of (some) sector regulators, public buyers can be expected to (pragmatically) rely on those standards to establish tender and contract requirements seeking to achieve the overarching goals of ‘responsible’ AI procurement and use. Where that happens, the public buyer will not be a rule-maker, but rather clearly a rule-taker. Such an approach would hollow out the public contract as a regulatory instrument because, rather than setting the relevant rules, it would simply become a conduit for the adoption of the externally generated (private) standards.
B. Commercial Determination Through Negotiations and Technical Judgement
A related consideration is that, even setting the issue of the origin of the relevant standards aside for a moment, there are challenges in clearly specifying the desired regulatory attributes related to most principles of AI regulation. Most of those attributes are difficult to observe or measure, and the processes leading to their promotion are not easy to establish. The outcomes of those processes are not binary and determining whether a requirement has been met cannot be subject to strict rules, but rather to (yet to be developed) technical standards with an unavoidable degree of indefinition,[119] which may also be susceptible of iterative application in, for example, agile deployment methods, and thus difficult to evaluate at tender stage. Moreover, the desired attributes can be in conflict between themselves or with the main functional specifications for the digital technology deployment. There is, for instance, a growing understanding of the incompatibility (or unavoidable trade-off) between requirements for explainability and AI performance, in the sense that non-explainable AI solutions tend to have high(er) levels of functional performance.[120] Negotiating those trade-offs is complex and subject to non-technical decisions (e.g., how much more accurate must a solution be to justify a reduction in explainability?).[121] Additionally, other incompatibilities or tensions between goals of digital regulation may be more difficult to identify and balance out, especially if the results of a technological deployment can only be observed and assessed with a significant time lag. This creates significant scope for public buyers to seek to co-create solutions through negotiations within the procurement process, or to leave some issues to co-decision at contract execution phase. While seeking to increase flexibility and to leverage the technical expertise of the tech provider, such approaches also generate significant risks of commercial determination of the regulatory contract.
This issue of the definitional difficulties and the incommensurability of some or most of the regulatory goals also relates to the difficulty of establishing minimum technical requirements as award constraints to avoid entering into contracts that would fall short of the required regulatory standards. For example, to require that no contract is awarded unless the tender reaches a specific threshold in the technical evaluation in relation to all or selected requirements (e.g., explainability). While the imposition of minimum technical requirements is permissible under procurement rules, it is difficult to design a mechanism to quantify or objectify the evaluation of some of the desired technological attributes, which will necessarily require a complex assessment.[122] In conducting such assessment the public buyer cannot have unrestricted freedom of choice, but it also needs to avoid being captured by tenderers with superior technical skills. This will require clarifying at the outset the criteria and thresholds that would justify rejecting a tender or not entering into a contract at all (e.g., if no adequate solutions are offered). This could become a significant sticking point, especially because the public buyer is generally unable to pre-determine one and only one way for tenderers to justify compliance with the required (minimum) technical specifications—as it would run against the pro-competitive logic of public procurement, which results in the public buyer being bound to accept compliance by equivalence to avoid technical determinism (and the ensuing risk of favouritism). Engaging in analysis of compliance by equivalence and, more generally, technical dialogue also opens the process up to commercial determination.
In those regards, it should be stressed that it may be easier to use technical specifications as regulatory tools in relation to some goals (e.g., those geared towards avoiding technological lock-in, as it is relatively simpler to establish whether the source code is open access) than others (e.g., trying to establish if the solution likely to be developed will be ‘sufficiently’ explainable). Designing technical specifications to capture whether a digital technology is ‘ethical’ or ‘trustworthy’ seems particularly challenging,[123] as is ensuring the fairness of its deployment—which is bound to be highly contextual. These are meta-attributes or characteristics that refer to a rather broad set of principles in the design of the technology, but also of its specific deployment, and tend to proceduralise the taking into account of relevant considerations (e.g., which impact will the deployment have on the population affected?).
Additionally, in some respects, the extent to which a technological deployment will be ethical or trustworthy is out of the hands of the tech provider (e.g., may depend on decisions of the entity adopting the technology, e.g., on how it is used). And, in some other respects, it depends on specific decisions and choices made during contract implementation that may be difficult to anticipate, or in which the tech provider may deviate from any commitments made at tender stage (e.g., where such deviation is non-observable or unlikely to be sanctioned). This could make it impossible to verify at the point of the tender whether the end result will or not meet the relevant requirements. Or, in other words, whether the technological deployment will be ethical or trustworthy (as demanded by the technical specifications). This issue relates to a more general problem of verification of forward-looking requirements in procurement procedures, and can create an incentive to push some of the regulatory requirements that cannot be easily described or that have a strong (future-looking) procedural component to contract design and implementation, in particular as contract performance clauses. Once again, this generates a significant risk of commercial determination (as well as exacerbating the issues arising from decentred interactions discussed above).
These and other risks of commercial determination are particularly relevant where tech providers have the upper hand due to bargaining power arising from their market position or superior technical skills. In the current context, this must be presumed to be the situation in most cases of AI procurement.[124]
C. Recapitulation: Regulatory Tunnelling and Commercial Determination as Major Obstacles
By focusing on the likely content of the regulatory contract, this section has shown how the public buyer can hardly be presumed to be the rule-maker in AI procurement. In the absence of legislative requirements that can be enforced through contract, the public buyer needs to determine those requirements for the purposes of selecting tech providers and choosing specific AI solutions in line with general principles and goals of AI regulation. Doing so is a taxing challenge and public buyers can be expected to (pragmatically) rely on industry-led standards—to which the government is giving broader credence in the context of its pro-innovation approach to AI regulation. The adoption of such standards as tender and contract requirements generate (a risk of) regulatory tunnelling.
The risk of a pro-industry bias in the standards is further exacerbated by the risks of commercial determination that arise from the difficulties in fully specifying the technological attributes required to meet (some) principles and goals of AI regulation. As those attributes are left to co-determination with the tech provider (e.g., in the context of negotiations, or decisions during contract execution in relation to contract compliance clauses), or susceptible to self-assessment by the tech provider, the risk of commercial determination increases. Given the dearth of available (practical) guidance and the growing gap in digital skills in the public sector, these risks of regulatory tunnelling and commercial determination raise important questions on the effectiveness of the strategy of AI regulation by contract that compound the institutional issues discussed in Section 3, especially in the context of decentred interactions.
5. Sketching an Alternative Regulatory Strategy
The analysis so far has shown that two of the foundational presumptions underpinning ‘regulation by contract’ do not necessarily or easily hold in the digital context. Therefore, ‘regulating by contract’ public sector AI use is bound to be an ineffective strategy. Continuing with the pretence that public buyers can ‘confidently and responsibly procure AI technologies for the benefit of citizens’ is thus unjustified and can generate individual harms and broader negative social effects as the public sector ramps up AI adoption and accumulates a potentially significant stock of AI deployments in a wide variety of areas of public sector activity.
The analysis so far has highlighted the existence of an institutional deficit in the process of public sector digitalisation, and AI adoption in particular. An alternative approach to institutional design is required.[125] Setting aside the contentious issue whether there should be a general AI regulator, a network of sectoral regulators, or a mixed approach,[126] the first part of this section will concentrate on key features of an independent regulator of the use of digital technologies by the public sector. This will be discussed around a notional ‘AI in the Public Sector Authority’ (AIPSA) that could be integrated into a broader network of sectoral regulators, or which role could be absorbed by a broader AI regulator with compatible institutional features, for instance, through a dedicated public sector unit (Section A). The analysis has also shown how there are pervasive risks of regulatory tunnelling, policy capture, and commercial determination of the process of public sector digitalisation. Therefore, one of the main roles for AIPSA would be to control the process of standardisation and to neutralise such risks by ensuring that the public interest is protected in the way the general goals of digital regulation are operationalised and embedded into such standards and related practices[127]—which would need to be controlled through a system of permission for public sector AI use (Section B).
A. Creating External Oversight: A Notional ‘AI in the Public Sector Authority’
The adoption of digital technologies in the process of public sector digitalisation creates regulatory challenges that require external oversight—in particular, to discipline the behaviour of the public sector entity seeking to use the AI, as well as that of the tech provider. In my view, such external oversight role needs to be assigned to a new regulator to avoid the regulatory inertia/drift that would arise from expanding the regulatory scope of an existing regulator (e.g., the Information Commissioner’s Office), as well as to avoid the regulatory fragmentation that would arise from entrusting such oversight to a network of ‘sufficiently’ close regulators depending on the area of public sector activity being digitalised/automated.
My proposal is to create AIPSA,[128] which would be an independent authority tasked with regulating the adoption and use of digital technologies by the public sector, whether through in-house development or procurement from tech providers.[129] AIPSA would also absorb regulatory functions in cognate areas, such as the governance of public sector data, and integrate work in areas such as cybersecurity. It would also serve a coordinating function with the data protection authority. AIPSA’s statutory function would consist in the promotion of the overarching goals of digital regulation discussed above, as well as in ensuring compliance with mandatory requirements applicable to the digitalisation of the public sector. The specific institutional design of AIPSA would require further contextual analysis—which exceeds the possibilities of this article—but it is submitted that it should seek to achieve three main goals: first, to establish the remit of public sector digitalisation as a distinct regulatory field to avoid regulatory fragmentation; second, to ensure independence from government and from industry in order to promote the public interest; and, third, to focus efforts to build public sector digital capabilities.
i. AIPSA as a Way to Avoid Regulatory Fragmentation
One of the primary design features for AIPSA should be regulatory coherence. AIPSA’s remit should be broad, both in terms of the digital technologies which use is subject to oversight, and the aspects of digital regulation for which it has competence. Public sector use of AI would clearly be a core issue, as would the governance of public sector-held data required for the development and operation of such AI. However, AIPSA should be able to exercise functions in relation to systemic issues, such as technological debt, intellectual debt, interoperability, or systemic risks. This would require extending its remit to the adoption of all digital technologies, although AIPSA’s functions in relation to specific technologies would need to be differential. Relatedly, AIPSA may have coordination or collaboration tasks in relation to aspects of digital regulation for which there is a specialist regulator or body, such as cybersecurity or personal data protection. AIPSA should not duplicate the specific activities of those specialist regulators or bodies, but it should have overarching responsibility for the oversight of the relevant issues as they relate to public sector digitalisation.
ii. AIPSA as a Way to Promote the Public Interest
Another of the primary design features for AIPSA should be to put the promotion of the public interest first—especially through the concretisation and operationalisation of the primary goals of digital regulation seeking to ensure that the public sector only adopts digital technologies that are legal, fair, explainable, transparent, interoperable, or (cyber) secure—and thus create trust in the process of digitalisation and in the new model of digital public governance.[130] The main driver for the creation of AIPSA would be twofold. First, AIPSA would be tasked with preventing the public sector from deploying technological solutions that breach fundamental rights, individual rights, or digital regulation goals or principles. Second, and on equal footing, AIPSA would be tasked with avoiding current risks of regulatory capture and commercial determination, to rebalance the current trend of public sector digitalisation that is overly reliant on private sector-led technological development and the associated emergence of technical (regulatory) standards.
For AIPSA to be able to play this role, it should be designed with a strong emphasis on two features:[131] independence and digital capability.[132] AIPSA would need to be designed in a way that ensured both political and industry independence.[133] While industry independence could to some extent be strengthened by boosting the regulator’s capabilities (below), ensuring political independence would require a careful assessment of the composition and conditions of appointment of the members at the top of its leadership, as well as the post-appointment regime of incompatibilities. A proposal for AIPSA to have a non-removable single person top management structure (e.g., a Director) seems slightly problematic from that perspective,[134] as it could be easier to target political pressure to an individual than a committee. It is submitted that it is preferable to have a small board of regulators (e.g., with five members) with different expertise,[135] and with staggered renewals. It would also be necessary to extend regulatory safeguards of AIPSA’s independence to the entire organisation, much as with entities tasked with audit or control of public sector activity. A particular issue worthy of careful consideration would be the applicable requirements on post-role employment and appointments across the organisation. As would the issue of funding of AIPSA and its financial independence. However, addressing such issues in detail exceeds the possibilities of this article.
AIPSA’s independence would require developing mechanisms of accountability. In addition to a system of Parliamentary appointments (or confirmation thereof), AIPSA’s accountability would need to involve transparency of decision-making, both in relation to strategies and workplans, and in relation with specific decisions—with the latter being subject to judicial review.
iii. AIPSA as a Way to Boost Public Sector Digital Capability
AIPSA’s expertise and, in particular, its digital capability should be another of its primary design features. The importance of digital capabilities to effectively exercise a digital regulation role cannot be overemphasised. It is not only important in relation to the active aspects of the regulatory role—such as control of standard setting or permissioning or licencing of digital technology use, as discussed below—but also in relation to the passive aspects of the regulatory role and, in particular, in relation to reactive engagement with industry. High levels of digital capability would be essential to allow AIPSA to effectively scrutinise claims from those that sought to influence its operation and decision-making,[136] as well as reduce AIPSA’s dependence on industry-provided information.[137] AIPSA’s design should thus include elements of centres of expertise or excellence and incorporate strong components of continuous professional development for its workforce, as well as a dedicated unit on horizon scanning and market intelligence.[138]
B. Mandatory Requirements for Public Sector Digitalisation
Regulating the adoption of digital technologies by the public sector necessitates establishing the substantive requirements that such technology must meet, as well as the governance requirements needed to ensure its proper use. Substantive regulation of the technology will depend on standard setting, which is thus one of the primary functions to be assigned to a digital technology regulator,[139] such as AIPSA. It is also important to establish the degree of bindingness of such standards and other governance requirements. In that regard, regulators can act either as soft-touch guidance providers, or as hard-edged regulators.[140] The primary role of a hard-edged regulator in this context would relate to the pre-approval of technological deployments by the public sector. It is submitted that AIPSA would need to act as a hard-edged regulator, as its creation would otherwise be indistinguishable from current soft law approaches to the generation of guidance for the acquisition of digital technologies by the public sector[141]—which are insufficient. Crucially, though, AIPSA’s hard-edged regulatory powers would be primarily targeted at the public sector entities seeking to adopt digital technologies, with tech providers effectively opting-in to the (heightened) minimum standards and requirements as a condition for conducting business with the government in areas that can affect the public interest. AIPSA’s role in setting mandatory requirements for public sector digitalisation would be twofold: standard certification and deployment authorisation.[142]
i. Standard Certification
First, through an approval or certification mechanism, AIPSA would control the process of standardisation to neutralise risks of regulatory capture and commercial determination. Where no standards were susceptible of approval or certification, AIPSA would develop them.
A great variety of standards are quickly emerging,[143] but there is no mechanism to ensure that they reflect the attributes required of technology to be deployed by the public sector—and some of those attributes may be impossible to sufficiently operationalise in a generalisable way through standards. Crucially, the process of standard setting involves the need to concretise and operationalise abstract normative concepts such as transparency, fairness, and accountability, and thus requires answering ‘hard normative questions’[144] that go well beyond ‘mere’ technical judgement. Or, in other words, creating these standards requires complex value judgements deeply woven into technical decisions.[145] This heightens the risk of regulatory tunnelling around digital regulation, as compared to other areas of technical harmonisation.[146]
AIPSA would thus be responsible for neutralising such risks through the scrutiny of any such standards against the digital regulation goals it is tasked with promoting. Only standards certified or approved by AIPSA could be used in the development or procurement of digital technologies for use by the public sector.[147] This approach would allow AIPSA to avoid having to generate standards from scratch, as it could certify existing standards that met the (heightened) regulatory requirements of the public sector with any required additions or modifications—and AIPSA would generate modules to ‘top up’ the requirements in such standards, or modify some of their requirements to ensure compliance. Only where certifiable standards were not independently generated in a way that could be suitably added to or modified, would AIPSA need to create them. It can be expected that this would be a minority of cases as the primary issue that can be expected with emerging industry standards is not one of wholesale unsuitability, but one of insufficient prescriptiveness or inadequate consideration of aspects specific to the deployment of AI in a public sector context—compared to private sector AI use.
Additionally, given that standards will hardly be able to generate ‘self-executable’ criteria,[148] even in relation to certified or approved standards, AIPSA should issue binding guidance on the implementation of such standards where they allow for different approaches, or where compliance with the standard is context dependent or subject to certain tolerances.[149] This could be done through technical guides and reports. One of the advantages of this type of specific guidance is that it would reduce the need for public sector entities seeking to adopt digital technologies to formulate their own approaches (in a constant reinvention of the wheel that would also raise the technical barriers to accessing AI procurement contracts). To avoid excessive prescriptiveness, a set of exceptions could be foreseen in the context of the individualised pre-approvals of specific uses discussed in the next sub-section.
ii. Deployment Authorisation
Second, through a permissioning or licencing process, AIPSA would ensure that decisions on the adoption of digital technologies by the public sector are not driven by ‘policy irresistibility’,[150] that they are supported by clear governance structures and draw on sufficient resources, and that adherence to the goals of digital regulation is sustained throughout the implementation and use of digital technologies by the public sector and subject to proactive transparency requirements.
To ensure compliance with the goals of digital regulation and adequate digital governance, AIPSA would be tasked with the pre-approval of each specific use of digital technologies by the public sector—and any changes in the use or the technology would require renewal of the approval. Such permissioning or licencing would be separate from (and additional to) any ex ante control for the placing of a technology in the market,[151] and would be aimed at the public sector entity seeking to adopt a digital technology.[152] The purpose of this pre-approval would not only be to ensure technical compliance, but also that decisions on digital technology adoption by the public sector are not driven by ‘policy irresistibility’, are cognisant of the need for the relevant enablers and their implementation is thus feasible, have considered new and modified risks and their mitigation, are supported by clear governance structures and draw on sufficient resources, and that adherence to the goals of digital regulation is sustained throughout the implementation and use of digital technologies and subject to proactive transparency requirements.
This would require AIPSA to verify and approve the existence of an adequate justification for the need to adopt the digital technology and that the data, technological and governance infrastructure is suited to that technological deployment. This would be an opportunity for AIPSA to provide a ‘reality check’ on the likely viability and fitness for purpose of the envisaged technological deployment, as well as to enforce principles that are not (easily) susceptible of inclusion in technical standards, such as a putative principle of AI minimisation.[153] It would also allow for a precautionary verification of the risk assessment carried out by the public entity seeking to adopt the technology, which could lead to the imposition of amendments or conditions seeking to improve risk management—or, in extreme cases, lead to a prohibition where AIPSA identified risks that could not be adequately managed.[154] It would also allow AIPSA to ensure that the public sector entity has the digital capability required to understand, use, and control the technology in a sustainable and resilient manner. The process of permissioning would also be a prime gateway for proactive transparency, as all authorised uses would automatically be included in a register of digital technologies used by the public sector.
While AIPSA should have the in principle competence to subject all uses of digital technology to pre-deployment authorisation—and this creates the default rule of requiring permission, which embeds a precautionary approach[155]—the operation of this system would require modular design to avoid overburdening AIPSA. In that regard, two primary regulatory tools would need to be developed. First, a ‘sifting tool’ that allowed for the straightforward approval of technology uses that did not raise specific risks meriting closer scrutiny[156]—for example, because they are designed in compliance with certified standards and the public entity is able to demonstrate compliance with all additional governance requirements, or because they refer to a modification of the use or the technology that does not alter its overall assessment in a previous approval. The sifting tool would need to be structured around different elements of risk and governance and related thresholds to allow for such screening.[157] These could be refined as AIPSA gained a better understanding of specific uses of particular technologies in specific contexts, in the same way as mechanisms of block exemption are developed in other areas (notably, competition law). The sifting tool would however still have to be used in all cases to facilitate the automatic inclusion of the permissioned use in the register, as well as to give AIPSA visibility of all ‘block-exempted’ uses. It would also allow AIPSA to develop an effective strategy of random audits and inspections in relation to those, which would be the second required regulatory tool.[158]
6. Conclusion: Some Thoughts on the ‘Regulatory Hallucination’
This article has shown how the current approach to deploying ‘regulation by contract’ techniques to govern the procurement and use of AI by the public sector is bound to be ineffective. Contrary to the hopes placed by the UK’s National AI Strategy on this approach, under current conditions, we cannot trust that public buyers will ‘confidently and responsibly procure AI technologies for the benefit of citizens’. It is more likely that public sector AI adoption will be largely for the convenience of public sector entities and for the benefit of tech providers—with citizen interests and individual rights at risk of significant harm. The article has highlighted the shortcomings of this regulatory approach and stressed that continuing with the pretence that no additional regulation is required—that no additional guardrails and safeguards are required as the public sector accelerates AI adoption—is unjustified and will lead to generate individual harms and broader negative social effects as the public sector ramps up AI adoption and accumulates a potentially significant stock of AI deployments in a wide variety of areas of public sector activity.
Despite its prima facie plausibility and the high confidence with which the National AI Strategy and subsequent UK government policy have embraced the current ‘pro-innovation’ approach to AI procurement and public sector AI use, this is the wrong answer to the myriad challenges emerging from quickly accelerating public sector digitisation. The hopes for public buyers to ‘confidently and responsibly procure AI technologies for the benefit of citizens’ stem from a ‘regulatory hallucination’. To bring that regulatory hallucination to an end, the article has sketched the main elements of an alternative regulatory strategy that would, however, be in direct contrast with the UK Government’s pro-innovation approach in the March 2023 White Paper. The chances of such alternative regulatory approach being implemented are thus slim, at least under the current government.
Now, what does this tell us about the UK’s (true) commitment to AI regulation and its claims to global leadership in AI safety? In my view, the use of algorithms and AI by the public sector is the litmus test for a nation’s commitment to meaningful AI regulation, as it involves (self)restricting the possibility of relying on technology and automation to address some of the problems arising from years of austerity, combined with social change and worsening challenges—not least those resulting from geopolitical instability and climate change. It also involves renouncing some powerful tools for the implementation of policies that can be particularly damaging to minorities and marginalised communities—which can however generate votes under the current climate of polarisation and regressive social policies. The UK is, in my view, failing such a test. The ‘do nothing’ strategy underlying the approach to public sector AI use ‘regulation by contract’ is indicative of a willingness to let experimentation and innovation (and the hope for resulting economic growth) trump any meaningful consideration for the risks and harms to individual rights and collective interests. It is also indicative of a willingness to be seen to do something without actually doing it—as evidenced with the development of the Algorithmic Transparency Recording Standard[159]—which matches the deregulatory approach resulting from the UK’s ‘pro-innovation’ stance.[160] If the UK government thinks it can influence global developments without first actually putting its own house in order, its aspirations of ‘leading the conversation on AI safety’ will be undermined by the regulatory hallucination in which its domestic policymaking is trapped.
- Department for Digital, Culture, Media and Sport, ‘New ten-year plan to make the UK a global AI superpower’ (22 September 2021) https://www.gov.uk/government/news/new-ten-year-plan-to-make-britain-a-global-ai-superpower accessed 17 October 2023. ↑
- Department for Science, Innovation and Technology, ‘National AI Strategy – AI Action Plan’ (18 July 2022) https://www.gov.uk/government/publications/national-ai-strategy-ai-action-plan/national-ai-strategy-ai-action-plan accessed 17 October 2023. ↑
- Nathalie A Smuha, ‘From a “Race to AI” to a “Race to AI Regulation”: Regulatory Competition for Artificial Intelligence’ (2021) 13 Law, Innovation and Technology 57. For a current assessment, see, for example, Andrea Renda, ‘Global AI governance is currently like the Tokyo Shibuya crossing – scrambled’ (CEPS blog, 5 October 2023) https://www.ceps.eu/global-ai-governance-is-currently-like-the-tokyo-shibuya-crossing-scrambled/ accessed 8 October 2023. ↑
- OECD.AI, ‘OECD AI Principles’ (2019) https://oecd.ai/en/ai-principles accessed 17 November 2023. ↑
- Ministry of Foreign Affairs of Japan, ‘G7 Leaders’ Statement on the Hiroshima AI Process’ (30 October 2023) https://www.mofa.go.jp/ecm/ec/page5e_000076.html accessed 25 March 2024. ↑
- Council of Europe, Draft Framework Convention on Artificial Intelligence, Human Rights, Democracy and the Rule Of Law (14 March 2024) CAI(2023)28rev4. The text was leaked by Luca Bertuzzi on LinkedIn on 19 March 2024 https://tinyurl.com/4cv8cweb accessed 25 March 2024. On file with author. ↑
- Department for Science, Innovation and Technology, ‘AI Safety Summit: introduction’ (25 September 2023) https://www.gov.uk/government/publications/ai-safety-summit-introduction accessed 8 October 2023. ↑
- Department for Science, Innovation and Technology, ‘Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023’ (1 November 2023) https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit-1-2-november-2023 accessed 17 November 2023. ↑
- Department for Science, Innovation and Technology, ‘UK government sets out AI Safety Summit ambitions’ (4 September 2023) https://www.gov.uk/government/news/uk-government-sets-out-ai-safety-summit-ambitions accessed 17 October 2023. ↑
- Department for Science, Innovation and Technology, ‘UK unveils world leading approach to innovation in first artificial intelligence white paper to turbocharge growth’ (29 March 2023) https://www.gov.uk/government/news/uk-unveils-world-leading-approach-to-innovation-in-first-artificial-intelligence-white-paper-to-turbocharge-growth accessed 17 November 2023. ↑
- Department for Science, Innovation and Technology and Office for Artificial Intelligence, ‘AI regulation: a pro-innovation approach’ (CP 815, 2023) (the ‘AI White Paper’); which gave continuity to the previous Department for Digital, Culture, Media and Sport, ‘Establishing a pro-innovation approach to regulating AI. An overview of the UK’s emerging approach’ (CP 728, 2022). For a further confirmation of the intended regulatory strategy, see Department for Science, Innovation and Technology, ‘A pro-innovation approach to AI regulation: government response’ (CP 1019, 2024) (the ‘Pro-Innovation AI Government Response’). ↑
- Pro-Innovation AI Government Response (n 11) para 76. ↑
- This could be reconciled if the calls for international collaboration were such as to deflect from meaningful regulatory intervention, which may be the case given the way ‘AI safety’ has been defined by the government for the purposes of the summit. However, assessing this issue exceeds the possibilities of this article. ↑
- See, for example, Matt Davies and Michael Birtwistle, ‘Regulating AI in the UK. Strengthening the UK’s proposals for the benefit of people and society’ (Ada Lovelace Institute, 18 July 2023) https://www.adalovelaceinstitute.org/report/regulating-ai-in-the-uk/ accessed 8 October 2023. ↑
- Department for Science, Innovation and Technology, ‘Frontier AI: capabilities and risks – discussion paper’ (25 October 2023) at 9 https://assets.publishing.service.gov.uk/media/65395abae6c968000daa9b25/frontier-ai-capabilities-risks-report.pdf accessed 17 November 2023. ↑
- Department for Digital, Culture, Media and Sport, ‘National AI Strategy’ (CP 525, 2021). ↑
- Ibid 40. ↑
- Ibid 47, emphasis added. ↑
- Ibid 59. ↑
- Office for Artificial Intelligence, ‘A guide to using artificial intelligence in the public sector’ (2019) https://www.gov.uk/government/collections/a-guide-to-using-artificial-intelligence-in-the-public-sector accessed 8 October 2023. ↑
- Office for Artificial Intelligence, ‘Guidelines for AI Procurement’ (2020) https://www.gov.uk/government/publications/guidelines-for-ai-procurement accessed 8 October 2023. ↑
- Centre for Data Ethics and Innovation, ‘Review into bias in algorithmic decision-making’ (2020) https://www.gov.uk/government/publications/cdei-publishes-review-into-bias-in-algorithmic-decision-making accessed 8 October 2023. ↑
- Centre for Data Ethics and Innovation, ‘Algorithmic Transparency Recording Standard Hub’ (2023) https://www.gov.uk/government/collections/algorithmic-transparency-recording-standard-hub accessed 8 October 2023. See also Public Law Project, ‘Tracking Automated Government register’ (9 February 2023) https://publiclawproject.org.uk/resources/the-tracking-automated-government-register/ accessed 8 October 2023. ↑
- Ibid. ↑
- Pro-Innovation AI Government Response (n 11) para 44. The government indicates that it will ‘be making use of the ATRS [algorithmic transparency recording standard] a requirement for all government departments and plan to expand this across the broader public sector over time’. However, as the standard allows for completion without publication, or with heavily redacted publication, the level of transparency that would follow broader engagement with the ATRS remains unclear. ↑
- 2023 c. 54. ↑
- Procurement HL Bill (2022-23) 4. ↑
- Ibid, amendment 46, seeking to introduce principles for automated decision-making and data ethics. ↑
- For criticism, see Albert Sanchez-Graells, ‘AI regulation by contract: submission to UK Parliament’ (How to Crack a Nut, 14 December 2022) https://www.howtocrackanut.com/blog/ai-regulation-by-contract-stc-inquiry-submission accessed 8 October 2023. ↑
- National Audit Office,Digital transformation in government: addressing the barriers to efficiency(HC 2022-23, 1171) (hereafter NAO, ‘Digital transformation in government’). ↑
- Department for Science, Innovation and Technology, ‘UK Science and Technology Framework’ (2023) part 6 https://www.gov.uk/government/publications/uk-science-and-technology-framework accessed 8 October 2023. ↑
- House of Commons Science, Innovation and Technology Committee,The governance of artificial intelligence: interim report(HC 2022-23, 1769). ↑
- AI White Paper (n 11). ↑
- Pro-Innovation AI Government Response (n 11). ↑
- For criticism, amongst others, see Andrew Charlesworth and others, ‘Response to the UK’s March 2023 White Paper “A pro-innovation approach to AI regulation”’ (SSRN, 19 June 2023) https://ssrn.com/abstract=4477368 accessed 8 October 2023 (hereafter, Charlesworth and others, ‘AI White Paper response’). ↑
- See, for example, the Equality and Human Rights Commission’s response to the AI White Paper https://www.equalityhumanrights.com/our-work/advising-parliament-and-governments/department-science-innovation-and-technology-pro accessed 25 March 2024. ↑
- Data Protection and Digital Information (No. 2) HC Bill (2022-23) 265. ↑
- See, for example, the Information Commissioner’s response to the AI White Paper https://ico.org.uk/media/about-the-ico/consultation-responses/4024792/ico-response-ai-white-paper-20230304.pdf accessed 25 March 2024. ↑
- Alex Lawrence-Archer and Ravi Naik, ‘Effective protection against AI harms—Analysis’ (AWO, July 2023) https://www.awo.agency/blog/awo-analysis-shows-gaps-in-effective-protection-from-ai-harms/ accessed 8 October 2023. ↑
- For broader discussion of the current characteristics of this process, see Patrick Dunleavy and Helen Margetts, ‘Data science, artificial intelligence and the third wave of digital era governance’ (2023) Public Policy and Administration, advanced access https://doi.org/10.1177/09520767231198737 accessed 8 October 2023. See also Karen Yeung, ‘The New Public Analytics as an Emerging Paradigm in Public Sector Administration’ (2022) 27(2) Tilburg Law Review 1 (hereafter Yeung, ‘New Public Analytics’). ↑
- National Audit Office,Use of artificial intelligence in government(HC 2023-24, 612). ↑
- NAO, ‘Digital transformation in government’ (n 30) 9. ↑
- George Parker, ‘Jeremy Hunt tells ministers to quicken adoption of AI to boost economy’ (FT, 18 June 2023) https://www.ft.com/content/626b16e9-22b2-42fa-9e54-f1b1f249ce66 accessed 8 October 2023. ↑
- NHS England, Transformation Directorate, ‘The National Strategy for AI in Health and Social Care’ (undated) https://transform.england.nhs.uk/ai-lab/ai-lab-programmes/the-national-strategy-for-ai-in-health-and-social-care/ accessed 8 October 2023. ↑
- Sophia Waterfield, ‘The education secretary wants more AI in classrooms, but is the technology ready?’ (TechMonitor, 30 March 2023) https://techmonitor.ai/government-computing/ai-education-uk-gillian-keegan accessed 8 October 2023. ↑
- See, for example,Siemens Mobility Limited v High Speed Two (HS2) Limited[2023] EWHC 2768 (TCC) at [146]. ↑
- See, for example, Jennifer Cobbe, ‘Administrative law and the machines of government: judicial review of automated public-sector decision-making’ (2019) 39(4) Legal Studies 636–55. ↑
- See, for example, Bernard Keenan, ‘Automatic Facial Recognition and the Intensification of Police Surveillance’ (2021) 84(4) Modern Law Review 886–97. ↑
- Joe Purshouse and Liz Campbell, ‘Automated facial recognition and policing: aBridgetoo far?’ (2021) 42(2) Legal Studies 209–27. ↑
- Albert Sanchez-Graells, ‘Resh(AI)ping Good Administration: Addressing the Mass Effects of Public Sector Digitalisation’ (2024) 13 Laws 9 https://doi.org/10.3390/laws13010009 accessed 25 March 2024 (hereafter Sanchez-Graells, ‘Resh(AI)ping Good Administration’). ↑
- For analysis of the convergence towards the same light-touch regulatory approach, except for the ‘high-risk’ AI uses under the EU AI Act, see Albert Sanchez-Graells,Digital Technologies and Public Procurement. Gatekeeping and Experimentation in Digital Public Governance(OUP 2024) 15–24 and 225–37 (hereafter Sanchez-Graells, ‘Digital Technologies and Public Procurement’). For a critical update on the EU’s approach, see Albert Sanchez-Graells, ‘Final EU model contractual AI clauses available – some thoughts on regulatory tunnelling’ (How to Crack a Nut, 6 October 2023) https://www.howtocrackanut.com/blog/2023/10/6/final-eu-model-contractual-ai-clauses-and-regulatory-tunnelling accessed 8 October 2023. See also Albert Sanchez-Graells, ‘Public Procurement of Artificial Intelligence: recent developments and remaining challenges in EU law’ (2024) 2 LegalTech – Zeitschrift für die digitale Rechtsanwendung (LTZ) 1–10. ↑
- Albert Sanchez-Graells, ‘Some thoughts on the US’ Executive Order on the safe, secure, and trustworthy development and use of AI’ (How to Crack a Nut, 7 November 2023) https://www.howtocrackanut.com/blog/2023/11/7/some-thoughts-on-the-executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-ai accessed 17 November 2023. ↑
- Where the public sector AI use involves the processing of personal data, protections under the personal data regime will be relevant. However, there is significant controversy on the scope and level of prescriptiveness of, for example, the protection afforded in the context of solely automated data processing, which is also subject to legislative change in the UK. Tackling that issue in detail exceeds the possibilities of this article, while setting it aside does not detract from the analysis of the self-standing regulatory role that can be expected of procurement. For broader discussion of data protection compliance in procurement settings, see Kevin McGillivray,Government Cloud Procurement. Contracts, Data Protection, and the Quest for Compliance(Cambridge University Press 2021). ↑
- Above (n 11). ↑
- Ibid para 10. ↑
- There is a rich literature on regulation by contract, or government by contract, which are the two primary labels attached to this phenomenon in the UK. It should be noted that a significant part of this scholarship focuses on public law issues rather than the commercial determination of private behaviour. See, for example, Colin Turpin,Government Contracts(Penguin 1972) 244–59; Terence Daintith, ‘Regulation by Contract – the Third Prerogative’ (1979) 32(1) Current Legal Problems 41; Mark Freedland, ‘Government by Contract and Public Law’ [1994] Public Law 86; Hugh Collins,Regulating Contracts(OUP 1999) 303–20; Peter Vincent-Jones,The New Public Contracting: Regulation, Responsiveness, Relationality(OUP 2006) (hereafter Vincent-Jones, ‘New Public Contracting’); Anne CL Davies,The Public Law of Government Contracts(OUP 2008). For comparative considerations, see Jean-Bernard Auby, ‘Contracting out and “public values”: a theoretical and comparative approach’ in Susan Rose-Ackerman, Peter L Lindseth and Blake Emerson (eds),Comparative Administrative Law(2nd edn, Edward Elgar 2017) 552. ↑
- See, for example, Peter Trepte,Regulating Procurement. Understanding the Ends and Means of Public Procurement Regulation(OUP 2004) 13–4 (hereafter Trepte, ‘Regulating Procurement’). For detailed analysis, see Christopher McCrudden,Buying Social Justice. Equality, Government Procurement, & Legal Change(OUP 2007); Sue Arrowsmith and Peter Kunzlik (eds),Social and Environmental Policies in EC Procurement Law. New Directives and New Directions(CUP 2009); Roberto Caranta and Martin Trybus (eds),The Law of Green and Social Procurement in Europe(DJØF Publishing 2010); Albert Sanchez-Graells (ed),Smart Public Procurement and Labour Standards. Pushing the Discussion after RegioPost(Hart 2018). ↑
- Due to space constraints this will not be explored in detail. ↑
- For simplicity, the expression ‘private provider’ (or ‘tech provider’) is used throughout the article to identify the contractor selected through a procurement process and/or with which the public sector enters into a contract. It should be acknowledged that publicly-owned providers can also participate in public tenders and hold public contracts for the supply of technological solutions. However, given the triangular relationship between public sector user entity, tech provider, and public buyer identified throughout the article, the simplified use of ‘private provider’ is preferred for clarity. ↑
- The reality of that use of procurement for regulation by contract is, however, more complex than the stylised example used here. See Richard Craven, ‘Managing dissonance: Bureaucratic justice and public procurement’ (2023) 17(1) Regulation & Governance 215. ↑
- See, for example, Mark Townsend, ‘Home Office secretly backs facial recognition technology to curb shoplifting’ (The Observer, 29 July 2023) https://www.theguardian.com/technology/2023/jul/29/home-office-secretly-backs-facial-recognition-technology-to-curb-shoplifting accessed 8 October 2023. ↑
- See, for example, Mark Townsend, ‘Revealed: Home Office secretly lobbied for facial recognition “spy” company’ (The Observer, 2 September 2023) https://www.theguardian.com/technology/2023/sep/02/home-office-accused-of-secret-lobbying-for-facial-recognition-spy-company accessed 25 March 2024. ↑
- See, for example, Harry Davies, ‘UK spy agencies want to relax “burdensome” laws on AI data use’ (The Guardian, 1 August 2023) https://www.theguardian.com/technology/2023/aug/01/uk-intelligence-spy-agencies-relax-burdensome-laws-ai-data-bpds accessed 8 October 2023. ↑
- Procurement, procurement function, or public buyer are used interchangeably throughout the article to refer to the department, unit or entity in charge of designing and running a tender for a public contract, as well as such contract, for the provision of digital technology solutions. ↑
- The Crown Commercial Service manages centralised procurement vehicles available to public sector buyers and, in that process, sets general requirements for the selection of private providers and general conditions for, for example, the technologies to be provided. In the context of AI, see, for example, the dynamic purchasing system on AI (RM6200) https://www.crowncommercial.gov.uk/agreements/RM6200 accessed 25 March 2024. ↑
- See, for example, IEEE, ‘GET Program for AI Ethics and Governance Standards’ (undated) https://ieeexplore.ieee.org/browse/standards/get-program/page/series?id=93 accessed 8 October 2023; or ISO/IEC 23053:2022 Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML). ↑
- NAO, ‘Digital transformation in government’ (n 30). ↑
- Trepte, ‘Regulating Procurement’ (n 57) 70–111; Christopher R Yukins, ‘A Versatile Prism: Assessing Procurement Law Through the Principal-Agent Model’ (2010) 40(1) Public Contract Law Journal 63; Albert Sanchez-Graells,Public Procurement and the EU Competition Rules(2nd end, Hart 2015) 56–8. ↑
- Conceptually, this can be framed in terms of the theory of delegation. For a law and economics assessment, see Robert D Cooter and Michael D Gilbert,Public Law and Economics(OUP 2022) 265 ff. ↑
- The issue is parallel to the tensions arising within other forms of contractualised governance where there is a misalignment between central and devolved government units, when the latter is entrusted with the procurement or contracting out of public services. For discussion, see Peter Vincent-Jones, ‘The New Public Contracting: Public Versus Private Ordering?’ (2007) 14(2) Indiana Journal of Global Legal Studies 259, 269 (hereafter Vincent-Jones, ‘Public Versus Private Ordering?’). ↑
- It is possible that some of these decisions are conditioned by other (non-procurement) frameworks, for example, of technological assessment. However, this issue is not explored in detail because any constraints derived from those frameworks would be external to procurement. ↑
- Or, at the very least, requires a horizontal repositioning, if the gatekeeping function is meant to be collaborative. ↑
- This is due to a lack of a legal or (explicit) contractual framework. For general discussion of the use of contractual governance to regulate public–public relationships, see Vincent-Jones, ‘New Public Contracting’ (n 56) 141–66 and 321–3. It is possible that several different types of rules, ranging from, for example, general administrative law to budgetary law, could apply to the public–public relationship. However, such an analysis exceeds the possibilities of this article. ↑
- Michael D Cohen, James G March and Johan P. Olsen, ‘A Garbage Can Model of Organizational Choice’ (1972) 17 Administrative Science Quarterly 1 (hereafter Cohen et al, ‘Garbage Can’). ↑
- By ‘policy irresistibility’ I mean the almost irresistible attractiveness of ‘tech fixes’ to policy problems than can arise from hyped narratives on the potential uses of emerging and still unknown technologies. ↑
- Michael Mintrom, ‘Policy Entrepreneurs and Dynamic Change’ in M Ramesh and others (eds),Cambridge Elements in Public Policy(CUP 2020) http://www.cambridge.org/9781108461467 accessed 8 October 2023. ↑
- Andrea Saltelli et al, ‘Science, the endless frontier of regulatory capture’ (2022) 135 Futures 102860. ↑
- Cohen et al, ‘Garbage Can’ (n 74) 1. ↑
- Ibid. ↑
- Ibid. ↑
- Ibid. ↑
- The applicability of the model to EU policymaking generally supports this approach. Jeremy Richardson,Policy-making in the EU. Interests, ideas and garbage cans of primeval soup(Taylor & Francis 1996). ↑
- Cohen et al, ‘Garbage Can’ (n 74) 16. ↑
- B Guy Peters, ‘Governance: A Garbage Can Perspective’ (2002) Vienna Institute for Advanced Studies, Political Science Series Num 84, 13 https://aei.pitt.edu/347/1/wp_84.pdf accessed 8 October 2023 (hereafter Peters, ‘Garbage Can’). ↑
- Cary Coglianese and Daniel E Walters, ‘Agenda-Setting in The Regulatory State: Theory and Evidence’ (2016) 68 Administrative Law Review 93, 97. ↑
- John W Kingdon,Agendas, Alternatives and Public Policies(Harper Collins 1984) 21. ↑
- Peters, ‘Garbage Can’ (n 84) 13. Ryan Hagemann, Jennifer Huddleston Skees and Adam Thierer, ‘Soft Law for Hard Problems: The Governance of Emerging Technologies in an Uncertain Future’ (2018) 17(1) Colorado Technology Law Journal 37, 107–10. ↑
- This scenario can be further complicated where the procurement is outsourced to consultants, which would add an additional layer of principal-agent issues in the functioning and decision-making of the adopting public entity. If anything, such as structure could be expected to amplify the risks identified here. Therefore, this specific (sub)scenario does not require further assessment. ↑
- For example, through the setting up of framework agreements or dynamic purchasing systems for the procurement of AI solutions and related consultancy services. See, for example, Crown Commercial Service, Artificial Intelligence (AI) (Agreement RM6200) (2020) https://www.crowncommercial.gov.uk/agreements/RM6200 accessed 8 October 2023. ↑
- Moreover, there are additional governance challenges and competing (commercial) incentives that detract from the ability of the central purchasing body to act as an independent regulator; see Albert Sanchez-Graells, ‘Competition Implications of Procurement Digitalisation and the Procurement of Digital Technologies by Central Purchasing Bodies’ in Carina Risvig Hamer, Magdalena Socha and Kirsi-Maria Halonen (eds),Public Procurement. Centralisation and new trends(DJØF Publishing 2024) 225–58. Available at https://ssrn.com/abstract=4376037 accessed 25 March 2024. ↑
- See, for example, Hadrien Pouget, ‘The EU’s AI Act Is Barreling Toward AI Standards That Do Not Exist’ (Lawfare, 12 January 2023) https://www.lawfareblog.com/eus-ai-act-barreling-toward-ai-standards-do-not-exist accessed 8 October 2023 (hereafter Pouget, ‘Standards That Do Not Exist’). ↑
- In fact, where the same organisation (or individual/s) is tasked with carrying out the procurement and overseeing contract implementation, the anticipated conflict of goals resulting from tender and contract design ‘in the shadow’ of implementation concerns would further weaken the gatekeeping role, independently of the public/private interaction issue. ↑
- This raises further risks of commercial determination where such aspects cannot be determined at tender stage and depend on implementation decisions. ↑
- Not least due to risks of capture; see Trepte, ‘Regulating Procurement’ (n 57) 82–3; Yeung, ‘New Public Analytics’ (n 40) 29. ↑
- For discussion of the need to inculcate on the contractual agent elements of institutional morality in the discharge of public functions, thus presuming a clear contrast between the interests of principal and agent in most contratualised governance regimes, see, for example, Vincent-Jones, ‘Public Versus Private Ordering?’ (n 70) 273, and idem, ‘Citizen Redress in Public Contracting for Human Services’ (2005) 68 Modern Law Review 887, 918. ↑
- This shows that, for example, the use of contract performance clauses as a regulatory tool can be largely ineffective where the behaviour of the economic operator can be dictated or influenced by the adopting public entity, over which the public buyer has no control. This is evidence of the limitations of the use of procurement to ‘regulate by contract’, especially as the public–public interaction is concerned and in the absence of other regulatory mechanisms to bind the adopting public entity. ↑
- Which is, however, not necessarily the case where the public buyer faces a concentrated industry or providers with superior knowledge and skills. However, analysing this issue exceeds the possibilities of this article. ↑
- This evidences that the risk of ‘shirking’ takes a peculiar two-sided dimension, different from the standard in relational contract theory; see, for example, David Frydlinger and Oliver Hart, ‘Overcoming contractual incompleteness: the role of guiding principles’ (2023) Journal of Law, Economics, and Organization, forthcoming https://doi.org/10.1093/jleo/ewac027 accessed 25 March 2024. ↑
- I use ‘regulatory tunnelling’ to refer to a situation where a regulatory mechanism results in a displacement or reallocation of decision-making power away from the intended regulator (in the context of this article, the public buyer) and in favour of a third party or, more problematically, the intended regulatee (in this case, the tech provider or, more generally, the tech industry). For discussion of how parallel approaches in other areas of EU risk-based regulation entrench the power of economic operators at the detriment of public interest, see Marta Morvillo and Maria Weimer, ‘Who shapes the CJEU regulatory jurisprudence? On the epistemic power of economic actors and ways to counter it’ (2022) 1 European Law Open 510. See also Carolyn Abbot and Maria Lee, ‘Economic Actors in EU Environmental Law’ (2015) 34(1) Yearbook of European Law 26. ↑
- I use ‘commercial determination’ to refer to situations where the regulatory requirements have been set or heavily influenced by the tech providers in a way that is detrimental or simply pays lip service to the more general principles of AI regulation, that is, a situation where the tech provider has steered the regulatory contract to its own benefit or advantage and this is detrimental to the digital regulation role of procurement. This borrows from the literature on commercial determinants of health, which is too vast to detail here. ↑
- These principles are meant to be developed by relevant regulators within their respective remits. Although no regulator has direct responsibility for public sector AI adoption, some of the future guidance issued by, for example, the Information Commissioner’s Office may be of (limited) assistance. Generally, see Department for Science, Innovation and Technology, ‘Implementing the UK’s AI regulatory principles: initial guidance for regulators’ (6 February 2024) https://www.gov.uk/government/publications/implementing-the-uks-ai-regulatory-principles-initial-guidance-for-regulators accessed 25 March 2024. See Albert Sanchez-Graells, ‘Initial UK Guidance on Pro-Innovation AI Regulation: Much Ado About Nothing?’ (How to Crack a Nut, 27 February 2024) https://www.howtocrackanut.com/blog/2024/2/23/initial-guidance-on-pro-innovation-ai-regulation accessed 25 March 2024. ↑
- See Procurement Policy Notice 02/24 on ‘Improving Transparency of AI use in Procurement’ (25 March 2024) https://assets.publishing.service.gov.uk/media/66019771a6c0f7bb15ef9252/PPN_02_24_Improving_Transparency_of_AI_use_in_Procurement.pdf accessed 25 March 2024. For discussion, see Albert Sanchez-Graells, ‘Did you use AI to write this tender? What? Just asking! – Also, how will you use AI to deliver this contract?’ (How to Crack a Nut, 26 March 2024) https://www.howtocrackanut.com/blog/2024/3/26/improving-transparency-of-ai-use-in-procurement-ppn accessed 26 March 2024. ↑
- Above (n 21). ↑
- Central Digital and Data Office, ‘The Technology Code of Practice’ (21 July 2023) https://www.gov.uk/guidance/the-technology-code-of-practice accessed 17 October 2023. ↑
- Central Digital and Data Office, ‘Government Design Principles’ (10 September 2019) https://www.gov.uk/guidance/government-design-principles accessed 17 October 2023. ↑
- Central Digital and Data Office, ‘Data Ethics Framework’ (16 September 2020) https://www.gov.uk/government/publications/data-ethics-framework accessed 17 October 2023. ↑
- Guidelines for AI Procurement (n 21) consideration number 7 in the ‘top 10 considerations’, emphasis added. ↑
- Ibid, consideration number 8. ↑
- Ibid, consideration number 9. ↑
- Ibid, emphasis added. ↑
- There is also no indication on how sectoral regulators are meant to engage in the same exercise; above (n 101). ↑
- Sanchez-Graells, ‘Digital Technologies and Public Procurement’ (n 51) 25–40. More generally, in relation to the EU AI Act, which presents the same issues even where mandatory rules on AI procurement are foreseen, see Michael Veale and Frederik Zuiderveen Borgesius, ‘Demystifying the Draft EU Artificial Intelligence Act—Analysing the good, the bad, and the unclear elements of the proposed approach’ (2021) 22(4) Computer Law Review International 97 (hereafter Veale and Borgesius, Demystifying the Draft EU AI Act’). ↑
- Which are themselves not on statutory footing, at least for now. ↑
- AI White Paper (n 11) para 49. ↑
- Ibid, Annex A. ↑
- Above (n 101). ↑
- For discussion in the narrower context of numerical estimation, see Piotr Bystranowski and others, ‘Anchoring Effect in Legal Decision-Making: A Meta-Analysis’ (2021) 45(1) Law and Human Behavior 1–23. ↑
- Veale and Borgesius, Demystifying the Draft EU AI Act’ (n 112) 105 ff. Mehwish Ansari and Vidushi Marda, ‘AI Act—leaving oversight to the techies will not protect rights’ (euobserver, 5 May 2023) https://euobserver.com/opinion/156992 accessed 17 October 2023. ↑
- Hadrien Pouget, ‘What will the role of standards be in AI governance?’ (Ada Lovelace blog, 5 April 2023) https://www.adalovelaceinstitute.org/blog/role-of-standards-in-ai-governance/ accessed 17 October 2023 (hereafter Pouget, ‘The role of standards in AI governance’). ↑
- The debate is particularly complex concerning medical AI applications; see, for example, Jon Rueda et al, ‘“Just” accuracy? Procedural fairness demands explainability in AI-based medical resource allocations’ (2022) AI & Society https://link.springer.com/article/10.1007/s00146-022-01614-9 accessed 17 October 2023;cfrSabine N van der Veer et al, ‘Trading off accuracy and explainability in AI decision-making: findings from 2 citizens’ juries’ (2021) 28(10) Journal of the American Medical Informatics Association 2128. ↑
- Roel Dobbe, Thomas Krendl Gilbert and Yonatan Mintz, ‘Hard choices in artificial intelligence’ (2021) 300 Artificial Intelligence 103555. ↑
- This mirrors the difficulties in assessing quality in procurement, as some of the regulatory requirements will not be strictly ‘technical’ in the sense of susceptible of detailed description as other types of technical specifications. Along the same lines Yeung, ‘New Public Analytics’ (n 40) 27. ↑
- However, see, for example, the attempt by IEEE, ‘GET Program for AI Ethics and Governance Standards’ (n 66). ↑
- NAO, ‘Digital transformation in government’ (n 30). ↑
- Margot E Kaminski, ‘Regulating the Risks of AI’ (2023) 103(5) Boston University Law Review 102, 157 ff. The approach put forward in this article draws from earlier proposals by Andrew Tutt, ‘An FDA for Algorithms’ (2017) 69 Administrative Law Review 83 (hereafter, Tutt, ‘FDA for algorithms’); and Colin Gavaghan and others, ‘Government Use of Artificial Intelligence in New Zealand’ (2019) https://ourarchive.otago.ac.nz/bitstream/handle/10523/9372/NZLF%20report.pdf accessed 17 October 2023 (hereafter Gavaghan et al, ‘Government Use of AI’). For other approaches to licencing or tort-based liability regulation, see Gianclaudio Malgieri and Frank Pasquale, ‘Licensing high-risk artificial intelligence: Toward ex ante justification for a disruptive technology’ (2024) 52 Computer Law & Security Review 105899 (hereafter Malgieri and Pasquale, ‘Licensing high-risk AI’); and Matthew U Scherer, ‘Regulating Artificial Intelligence Systems: Risks, Challenges, Competencies and Strategies’ (2016) 29(2) Harvard Journal of Law & Technology 353. ↑
- For related discussion, see Matt O’Shaughnessy and Matt Sheenan, ‘Lessons From the World’s Two Experiments in AI Governance’ (Carnegie Endowment for International Peace, 14 February 2023) https://carnegieendowment.org/2023/02/14/lessons-from-world-s-two-experiments-in-ai-governance-pub-89035 accessed 17 October 2023. ↑
- Similarly, Gavaghan et al, ‘Government Use of AI’ (n 125) 76. This is to guard against both direct and more subtle influences on the setting of the relevant standards and benchmarks, as well as broader conceptualisations of what the ‘public interest’ means in this context; see Wendy Y Li, ‘Regulatory capture’s third face of power’ (2023) 21(2) Socio-Economic Review 1217 (hereafter, Regulatory capture’s third face of power’). ↑
- For additional details, see Sanchez-Graells, ‘Digital Technologies and Public Procurement’ (n 51) 103–19. ↑
- In the same terms, see Gavaghan et al, ‘Government Use of AI’ (n 125) 73 and 76. ↑
- The issue of how to identify and concretise ‘the public interest’ in this context exceeds the possibilities of this article. For the purposes of this discussion, the public interest is expected to be encapsulated in the overarching goals of digital regulation AIPSA is expected to enforce. For additional discussion, see Sanchez-Graells, ‘Digital Technologies and Public Procurement’ (n 51) 25–40. ↑
- The literature on regulatory agency design is too vast to attempt to summarise it here. The discussion simply aims to stress crucial issues for further consideration. ↑
- See Julia Black, ‘Constitutionalising Regulatory Governance Systems’ (2021) LSE Law, Society and Economy Working Papers 02/2021, 16 http://eprints.lse.ac.uk/id/eprint/113670 accessed 7 April 2023. ↑
- See, for example, OECD, ‘The Governance of Regulators. Creating a Culture of Independence’ (undated) https://www.oecd.org/gov/regulatory-policy/Culture-of-Independence-Eng-web.pdf accessed 7 April 2023. See also National Audit Office, ‘Good practice guidance. Principles of effective regulation’ (2021) https://www.nao.org.uk/wp-content/uploads/2021/05/Principles-of-effective-regulation-SOff-interactive-accessible.pdf accessed 7 April 2023. ↑
- However, this is the preferred structure in the US context; see Tutt, ‘FDA for algorithms’ (n 125) 118. ↑
- Along the same lines, Gavaghan et al, ‘Government Use of AI’ (n 125) 63, suggesting that ‘expertise in computer science, data analytics, law and ethics seem like obvious inclusions’. ↑
- This is particularly important in relation to long-term approaches to influencing regulatory activities; Li, ‘Regulatory capture’s third face of power’ (n 127). ↑
- This is particularly important to guard against epistemological power imbalances favouring industry influence; see Morvillo and Weimer, ‘On the epistemic power of economic actors’ (n 99). ↑
- AIPSA’s design should also explicitly address the main obstacles in recruiting and retaining digital professionals. However, addressing this issue exceeds the possibilities of this article. ↑
- Tutt, ‘FDA for algorithms’ (n 125) 107–9. ↑
- Ibid 109–11. ↑
- For example, through the Guidelines for AI Procurement (n 21). ↑
- AIPSA’s role would need to be supported by a system of sanctions addressed to both public sector entities and tech providers, for example, in relation to the use of unapproved technologies or non-compliance with certified standards. However, examining this issue in detail exceeds the possibilities of this article. ↑
- See AI Standards Hub https://aistandardshub.org/ accessed 17 October 2023. ↑
- Johann Laux, Sandra Wachter and Brent Mittelstadt, ‘Three Pathways for Standardisation and Ethical Disclosure by Default under the European Union Artificial Intelligence Act’ (SSRN, 20 February 2023) https://ssrn.com/abstract=4365079 accessed 17 October 2023. See also Roel Dobbe, Thomas Krendl Gilbert and Yonatan Mintz, ‘Hard choices in artificial intelligence’ (2021) 300 Artificial Intelligence 103555. ↑
- Pouget, ‘The role of standards in AI governance’ (n 119). ↑
- Where it is still problematic, though. See Mariolina Eliantonio and Caroline Cauffman (eds)The Legitimacy of Standardisation as a Regulatory Technique(Edward Elgar 2020); Rodrigo Vallejo, ‘The Private Administrative Law of Technical Standardization’ (2021) 40 Yearbook of European Law 172. ↑
- The need for regulators to scrutinise emerging standards to assess their fit with regulatory requirements is stressed in the UK’s ‘pro-innovation’ strategy; see Department for Science, Innovation and Technology and Office for Artificial Intelligence, ‘AI regulation: a pro-innovation approach’ (CP 815, 2023) Annex A. ↑
- Pouget, ‘Standards That Do Not Exist’ (n 91). ↑
- This would perhaps be the practical way to specify the heightened needs of the public sector while still in compliance with the maximum harmonisation nature of the EU AI Act. ↑
- Sanchez-Graells, ‘Digital Technologies and Public Procurement’ (n 51) 123 ff. ↑
- As generally proposed by Tutt, ‘FDA for algorithms’ (n 125) 111 and 116–17. Arguing for a similar licencing scheme for some types of AI used, see Malgieri and Pasquale, ‘Licensing high-risk AI’ (n 125). ↑
- The same proposal is put forward by Gavaghan et al, ‘Government Use of AI’ (n 125) 73. ↑
- Sanchez-Graells, ‘Digital Technologies and Public Procurement’ (n 51) 36 ff. ↑
- This pre-emptive orex anteapproach would be one of the main tools to minimise risks of mass harms. Sanchez-Graells, ‘Resh(AI)ping Good Administration’ (n 50). ↑
- This is consistent with the precautionary approach advanced here from a normative perspective. ↑
- This could be built on efforts at formalising impact assessments, such as the Privacy, Human Rights and Ethics Framework developed in New Zealand; Gavaghan et al, ‘Government Use of AI’ (n 125) 71. Another potential foundation on which to build would be the recently released AI RMF; National Institute for Standards and Technology, ‘Artificial Intelligence Risk Management Framework (AI RMF 1.0)’ (2023) https://www.nist.gov/itl/ai-risk-management-framework accessed 17 October 2023. ↑
- Granted, developing the sifting tool would not be simple and would require constant review. It should also be acknowledged that there would be a slowing down of the process of AI adoption while exemption based on classes of risk or other technical considerations could be safely developed. The same would apply to new digital technologies. However, once again from a normative perspective, it is submitted that the benefits of theex antepermissioned approach outweigh the downsides of the slower technological uptake—at least in the way required for the reversal of the current default rule that all technologies can be adopted unless there is a specific impediment. ↑
- Addressing the issue of third-party audit and AIPSA inspection exceeds the possibilities of this article. ↑
- Above (n 23); At the time of writing (26 March 2024) in addition to the initial six pilot projects, only on further disclosure has been recorded. ↑
- Charlesworth and others, ‘AI White Paper response’ (n 35). ↑