Contents

“Man has become a document like any other, with an identity that he no longer 'owns', over which he has little control (...) and whose commercial purpose he underestimates.”^*1

I. Introduction

What are the privacy implications of emerging supranational frameworks on artificial intelligence (hereinafter AI)²? Especially since the release of generative AI, there have been multiple calls to action for addressing the risks posed by the deployment and use of AI systems. In many instances, these calls were followed by non-binding initiatives and technology-specific frameworks intended to complement existing frameworks such as data-protection regulations.

Initiatives such as the European Union’s latest Artificial Intelligence Act (AI Act)³ and the Council of Europe’s Framework Convention on AI (Framework Convention) are most welcome. The other option, self-regulation by the tech sector, mostly through soft law, is not desirable due to the underlying economic interests driving their activities. Binding rules are preferable to soft law instruments, although the speed at which the sector is evolving requires that caution be exercised throughout any legislative processes taken to this end. Furthermore, many AI-driven activities know no border. Regulating technologies of such transnational nature requires concerted regulatory efforts at the global level.⁴ Hence, the ongoing “rush to AI regulation”⁵ is an opportunity to collectively address longstanding privacy issues in light of technological advances in the AI domain.

Among all possible uses of AI technology, surveillance activities, as the act of “watching, listening to, or recording of an individual’s activities”⁶, constitute major sources of privacy erosion. By definition, surveillance is antithetical to privacy. Regulating corporate surveillance and state surveillance would thus go a long way in order to mitigate many privacy risks stemming from AI technology. Moreover, United Nations members have expressly called “upon all Member States and, where applicable, other stakeholders to refrain from or cease the use of artificial intelligence systems that are impossible to operate in compliance with international human rights law or that pose undue risks to the enjoyment of human rights.”⁷ Yet, no other area encapsulates the complexities and challenges of AI technology as profoundly as the surveillance practices of the private and public sectors, since these may involve AI tools at different levels. Although the burgeoning regulatory landscape pertaining to AI reveals that legislators worldwide have identified the key challenges associated with the technology, privacy generally takes the backseat.

This paper seeks to assess how some privacy concerns raised by AI - and their underlaying causes - are actually being addressed in emerging supranational legal frameworks on AI, focussing on AI-driven surveillance. To this end, the paper starts by exploring how AI technology and digital surveillance practices intersect (II). In a second section, the paper offers an attempt to conceptualise digital privacy (III). The paper then analyses emerging AI regulations, highlighting the remaining gaps when it comes to mitigating AI-driven surveillance (IV). A few concluding remarks question the capacity of international human rights law to rescue privacy (V).

II. AI systems for digital surveillance

Public and private actors are increasingly resorting to AI in their surveillance apparatus.⁸ This part examines AI uses in the context of digital surveillance, starting with digital surveillance by the private sector, or “corporate surveillance” (1), before moving to digital surveillance by the public sector, or “state surveillance” (2).

1. AI in corporate surveillance

“Corporate surveillance” is a synonym of “surveillance capitalism”, a term famously coined by Harvard Professor emerita Shoshana Zuboff, which she defined as “the unilateral claiming of private human experience as free raw material for translation into behavioural data.”⁹

“Surveillance capitalism” thus refers to a paradigm where individuals’ behaviours are tracked, their desires inferred and anticipated based on the information collected from them, for the purpose of steering consumption habits. When describing the advent of surveillance capitalism, authors speak of the “commodification” of both data and attention. Corporate surveillance proceeds from the dehumanizing rationale that economic value can (and must) be attached respectively, to users’ data and attention.¹⁰ While the underlying logics of corporate surveillance were already present in the advertising industry,¹¹ the ubiquity of AI surveillance tools makes it a more concerning trend today.

To commit corporate surveillance, digital companies rely on vast amounts of information i.e. big data, which aggregates information from a variety of sources.¹² This data is then exploited by algorithms to derive new insights into users' personalities and routines. AI-driven behaviour prediction is a crucial component of surveillance capitalism. Its ability to translate raw data into behavioural data is precisely what makes AI technology so valuable in this context, because it allows companies to predict users’ behaviours with the highest degrees of accuracy. The ensuing practices are often justified on the grounds of more tailored advertising, richer service offerings, or the free enjoyment of certain services. In this context, “Privacy is now less a line in the sand beyond which transgression is not permitted, than a shifting space of negotiation where privacy is traded for products, better services or special deals.”¹³

The resulting data commodification paradigm has been criticized for leaving individuals with no meaningful ways to consent to data collection, lack of legal protection regarding the inferences made from the bulk data collected, and lack of information regarding the processing and the parties involved.¹⁴ At a more abstract level, corporate surveillance has been criticized for taking away users’ capacity for judgement.¹⁵ The level of conditioning achieved through extreme content personalisation results in users gradually losing the ability to ponder over choice. In the long term, these mechanisms are detrimental to privacy and individual autonomy.¹⁶

Of course, the convergence of big data and AI technology raised concerns before the emergence of generative AI.¹⁷ AI technology was at work in the mechanisms involved in corporate surveillance very early on, in the form of predictive technology embedded into home and on-device assistants to gather data, and profiling algorithms designed to provide actionable insight into the habits of an individual and thus enable decision-making.¹⁸ Today, technological advances in the AI domain enable marketers and data scientists to collect more information, to make sense of larger volumes of data, and to infer granular knowledge about users.¹⁹ When it comes to generative AI in particular, the technology is notably used to power virtual companions²⁰ or digital versions of deceased loved ones.²¹ These applications are controversial for many reasons, including from a privacy standpoint, as they may lead to the divulgation of very intimate data, thereby enabling higher levels of surveillance.

The wealth of data detained by the largest digital platforms makes them useful partners for governments. While the private sector may not always be aware of a state’s tapping their databases, the private sector sometimes willingly cooperates with public agencies in state surveillance, repurposing commercial databases to accommodate the security needs of governments. For instance, China’s state surveillance apparatus relies heavily on the private sector for the constitution of databases.²² In the famous NSA surveillance case, US Telecom company AT&T reportedly copied and transmitted the communications of its consumers to government authorities.²³ Similarly, in the facts leading to ECJ’s “BCD case”, bulk communications data (BCD) was collected by the Security and Intelligence Agencies from mobile network operator,²⁴ and US Supreme Court’s United States v. Miller case featured the communication of a bank’s client information to US government agencies.²⁵

2. AI in state surveillance

The term "surveillance state" is used to describe a model of governance relying on pervasive surveillance tools to collect and analyse information about citizens for the purpose of anticipating crime, securing public spaces, and, more broadly, maintaining national security. State surveillance may be conducted through either digital or analogue means, although the increase in international terrorism in the 2000s and the subsequent digitization of society led to a generalization of the recourse to digital surveillance techniques. A prominent illustration of the surveillance state model is the extensively-documented national surveillance apparatus of the National Security Agency (NSA).²⁶

Corporate surveillance and state surveillance share some similarities, the first being the mechanisms at play i.e. the massive collection and analysis of data, usually implicating AI solutions. Second, the imbalance of power that characterizes the relationship between individuals, states and private actors means that the former are usually left with few means to resist surveillance, let alone the coercive power it enables. Third, the two surveillances may have a negative impact not only on privacy, but also on other fundamental rights such as free speech. Finally, there are no pure surveillance capitalists nor pure surveillance states, but a handful of business and governance models involving varying degrees of privacy intrusion.

The end goals are however dissimilar between the two types of surveillance. Indeed, while corporate surveillance seeks economic advantage by steering positive behaviour, state surveillance seeks national stability by discouraging them. Of the two, state surveillance may appear more justifiable, which has notably led some authors warning against thinking of surveillance as a “malign plot hatched by evil powers.”²⁷ For instance, state surveillance was useful in the context of the spread of the coronavirus during the pandemic. Yet, as pointed by Solove, “Too much social control, however, can adversely impact freedom, creativity, and self-development.”²⁸ In the same vein, the independent high-level expert group on artificial intelligence appointed by the EU Commission emphasized the delicate process of striking a balance between the prevention of harm through surveillance practices and the protection of privacy and autonomy.²⁹

Much like the former, state surveillance is the subject of increasing attention because of the growing reliance of states on AI surveillance tools.³⁰ AI technology is at play in several mechanisms of state surveillance, where it can be used to perform various image processing tasks such as object and behaviour detection in order to predict scenarios, so-called “algorithmic surveillance.” Arguably more problematic, AI technology can also be leveraged to execute surveillance activities such as biometric identification, emotion recognition and biometric categorization for law enforcement.

The use of AI tools for the purpose of conducting state surveillance activities is problematic for a number of reasons. In 2023, the UN High-Level Advisory Body on Artificial Intelligence singled-out real-time biometric surveillance for law enforcement purpose as posing an “unacceptable risk, violating the right to privacy.”³¹ Alongside the serious risk of biased outputs,³² one main issue with the inclusion of AI technology into a state’s surveillance apparatus is its ability to infer large quantities of information about physical persons based on the captured images. The French data protection authority speaks of a trend towards generalized “analysis”, as opposed to the initial generalized surveillance.³³ Such analysis leads to what Lyon calls "anticipatory governance", where surveillance is “less concerned with the overall picture of a given individual as with 'premeditating and pinpointing potential dangers.'"³⁴ In this context, the likelihood of errors and misuse is significant.

Unfortunately, the level of data transparency exhibited by surveillance activities is often lacking, preventing many from fully grasping the true extent of personal information private companies and states can extract from a few data points, how their data weighs in surveillance outcomes, and more broadly, the impact surveillance activities may have on their private lives. In this context, privacy and by extension human autonomy and dignity, cannot be properly guaranteed.³⁵

III. Beyond data protection; making sense of privacy in the digital era

The right to privacy is considered “one of the foundations of a democratic society.”³⁶ This part offers a brief background to privacy, (1) before exploring the subset concept of digital privacy, (2) in order to explicate the nature of the legal harm resulting from surveillance activities.

1. Background to privacy

Given its prominent role in modern societies, the right to privacy is enshrined in many authoritative sources. At the international level, the right to privacy is enshrined in Art. 12 of the 1948 Universal Declaration of Human Rights³⁷ and Art. 17 of the 1966 International Covenant on Civil and Political Rights³⁸, both providing in identical terms; “No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.”³⁹ The right to privacy is also replicated in Art. 7 of the Charter of Fundamental Rights of the European Union⁴⁰, Art. 8 of the European Convention on Human Rights⁴¹, Art. 21 of the ASEAN Human Rights Declaration⁴², and Art. 11 of the American Declaration of the Rights and Duties of Man⁴³, among other important documents.

Despite its ubiquity, the right to privacy remains an elusive concept. In 2006 Solove observed “[P]rivacy is a concept in disarray. Nobody can articulate what it means”⁴⁴ Almost twenty years later, privacy remains “a complicated concept to review”⁴⁵ This is due to the fact that privacy is inherently a protean concept. Privacy applies both horizontally, in person-to-person settings, and vertically, in institutions-to-person settings. Each context brings different expectations towards the conduct of external parties.⁴⁶

In legal doctrine, privacy is apprehended simultaneously as a primary right susceptible of direct violation and as a source of more specific rights, the violation of which doubles as privacy infringement, such as with the right to protect reputation or the right to abortion.⁴⁷ The right to privacy is also an enabler of other rights and values. Hence, free speech, consumer protection and the right to public participation cannot exist without sufficient privacy guarantees.⁴⁸ But its intricate relationship with other rights and freedoms is not the sole source of difficulties when it comes to defining the concept of privacy.

Indeed, privacy and the protections stemming from it are in constant evolution. The social demand for privacy is itself in a state of flux, and varies based on societal, cultural and technological factors.⁴⁹ In addition, the right to privacy is not absolute because it admits exceptions that may differ from one domestic system to another. In fact, while, it would seem that most states are aware of the importance of safeguarding privacy, they do not necessarily approach it the same way. Take the example of the freedom of the press, which, until recently, was given primacy over privacy in the UK, while privacy had long prevailed over the freedom of the press in France.⁵⁰

Few legal instruments provide a clear definition of privacy. As remarked by the European Court of Human Rights (ECtHR) in Pretty v. The U.K; “[…] the concept of ‘private life’ is a broad term not susceptible to exhaustive definition.”⁵¹ Solove similarly states that “the term ‘privacy’ is an umbrella term, referring to a wide and disparate group of related things.”⁵² Be that as it may, the basic premises of privacy remain relatively discernible.

2. Digital privacy defined

Overall, privacy cases around the world have drawn from three theories of privacy; non-intrusion, self-determination, (or non-interference), and control over one’s information.⁵³ Each theory highlights one dimension of privacy: the physical, the decisional and the informational dimension, which complement each other in different ways.⁵⁴ Exploring pivotal case law on privacy provides valuable insights into its three dimensions.

In 1890 US lawyers Samuel D. Warren and Louis Brandeis famously referred to the right to privacy as the “right to be let alone”,⁵⁵ a rather rudimentary understanding of privacy mainly interpreted in the context of the relationship between the administration and individuals. The concept was directly drawn from the fourth US constitutional amendment, which recognizes the right “[...]to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, [...]”⁵⁶ The view was later criticized for being “both too broad and too narrow to count as a successful definition.”⁵⁷ Still, the definition evokes the physical dimension of privacy: a safeguard for the physical premises of a person, or physical privacy.

Privacy does not only relate to the physical premises of a person, but extends to intangible values, such as one’s ability to make choices and take decisions regarding intimate matters without interference, or decisional privacy.⁵⁸ Building on a longstanding jurisprudence, the US Supreme Court for example considered in the oft-cited 1973 Roe v. Wade case that the right to abortion was encompassed in the right to privacy.⁵⁹ Similarly, decisional privacy has been an important part of the jurisprudence of the European Court of Human Rights stemming from Art. 8 ECHR, notably in connection to family matters.⁶⁰ At present, it is receiving increasing intention in the context of corporate surveillance.⁶¹

The third dimension of privacy, informational privacy, applies to situations implicating personal information, for instance, when one’s reputation is damaged by a smear campaign or when one’s personal data are being harvested without consent. Informational privacy is defined as the ability to exercise control over one's personal information, including image, correspondence and personal data. Informational privacy is sometimes referred to as “informational self-determination” or “informational autonomy.”⁶² The informational dimension of privacy was, for example, in question in the 2017 Bărbulescu v. Romania case before the ECtHR, where the Court considered that instant message communications qualified as “correspondence” protected under Art. 8 ECHR, even when sent from the workplace. In Whalen v. Roe, the US Supreme Court makes explicit reference to a constitutional right to informational privacy.⁶³ Guarantees such as the protection of reputation and data protection principles including consent, control over the data, right to erasure, and rectification of information, relate to the informational dimension privacy. These principles are covered in most data protection laws.

In light of the above, it can be asserted that the right to privacy is a claim that extends to physical locations, the body, personal decisions, and digital as well as non-digital information, as long as these closely relate to elements of the personality or the life of the rights-holder.⁶⁴

Digital privacy, in particular, both relates to the second and third aspects of privacy. Specifically, it motivates expectations regarding the ways third parties should treat the digital components of the private sphere, and what they effectively do with them, as long as the end goals have repercussions on their autonomy, taking into account the invasive nature of the technologies at play. In that sense, digital privacy acts as a defence against attempts to encroach on individual autonomy involving the processing of private information.

It bears noting that not all privacy invasions are blatantly illegal. In modern digital societies, individuals are invited to surrender components of their private selves in an ongoing manner, to access services, use goods, and overall, to improve their quality of life. Nevertheless, to relinquish personal information should not be equated to a total abandonment of privacy. It is helpful to consider the current paradigm as a sort of constant bargaining state, where components of the private self are exchanged for things via digital platforms. While absolutist views of informational privacy are hardly tenable under the current paradigm⁶⁵, privacy still requires that guarantees pertaining to the security and confidentiality of the information thus collected, and in fine the degree of autonomy kept by the right-holder, are assured.⁶⁶ In other words, the bargain should be based on trust and therefore, conditional.⁶⁷ Yet, as shown earlier, the privacy-adverse mechanisms involved in modern surveillance activities hardly, if ever, satisfy these criteria.⁶⁸

When consent is not required, as is often the case with state surveillance, the validity of surveillance practices must be assessed from the perspective of individuals' expectations of privacy, which must be reasonable. That is to say, balanced with the objectives sought and the necessity of the practice under scrutiny. Consent-based data collection practices should themselves guarantee free and informed consent. However, the practice leading to corporate surveillance usually rely on suboptimal measures to ensure informed consent. Few individuals realize the amount of personal information collected by corporations, let alone what is inferred from these data, and which decisions are taken based on said inferences. Due to their opacity and irresistibility, corporate surveillance practices subvert the conditions of the bargain, thereby undermining the concept of consent.⁶⁹

Besides, the commodification of attention and data that pervades surveillance activities inherently contradicts human dignity. As noted in the explanation paper to the Convention 108+, “[h]uman dignity requires safeguards to be put in place when processing personal data, in order for individuals not to be treated as mere objects.”⁷⁰

IV. AI-driven surveillance in emerging AI laws

Are the mechanisms involved in surveillance – namely, the extensive accumulation and analysis of data, and the subsequent inferences drawn therefrom – adequately addressed in emerging legal regimes on AI? Leaving aside non-binding instruments, this section covers supranational regulatory initiatives pertaining to AI, identifying gaps in their approach to privacy (1), before expanding the discussion to other relevant regimes (2).

1. AI-driven surveillance in emerging international AI regulations

a) The AI Act

While considering relevant regulatory trends tailored to AI, one inevitably stumbles upon the recent EU’s AI Act. Inspired by product safety rules, the AI Act sets forth a comprehensive and horizontal framework for the regulation of AI systems in the Union. Much like the General Data Protection Regulation (GDPR)⁷¹ before it, the AI Act could become a benchmark for AI regulation, the so-called “Brussel effect”.

The AI Act is based on the prescriptions set forth by the high-level expert group on artificial intelligence, which emphasized privacy as a core requirement to achieve “trustworthy AI”, understood as an AI system that is “lawful,” “ethical,” and “robust.”⁷² The text lays down several obligations for providers and deployers of AI systems, which vary based on the degree of risk associated with the system and its use. Some prohibitions are targeted at AI systems deemed to pose “unacceptable risks.” Importantly, the AI Act applies to both the public and the private sector, as long as the entity in question acts as provider or deployer of AI systems.

The AI Act does not deal specifically with data protection, since the GDPR already covers this important aspect of digital privacy. However, several privacy-adverse practices are addressed. Art. 5 AI Act notably prohibits particularly intrusive systems, which could be used for social control, and yield disproportionate harm to human rights, such as certain social-scoring practices and AI systems that create or expand “facial recognition databases through untargeted scraping of facial images.”⁷³

A first prohibition that seems relevant to corporate surveillance and its mechanisms relates to the use of AI systems for manipulative and exploitative purposes. The AI Act indeed prohibits some AI-enabled manipulative and exploitative practices involving the voluntary distortion of behaviours in ways that would cause significant harm to a person or a group of persons.⁷⁴ Recital 29 of the AI Act, which provides some interpretative guidance on the matter, clarifies that the prohibition applies to the commercial context as well, but also notes that “common and legitimate commercial practices, for example in the field of advertising, that comply with the applicable law should not, in themselves, be regarded as constituting harmful manipulative AI-enabled practices.”⁷⁵

Beyond the unclarities introduced by the use of the adjectives “common and legitimate”, and although Recital 29 does not seem to evacuate completely the possibility that consumer manipulation practices akin to corporate surveillance fall into the scope of Art. 5 of the AI Act, the applicability of the relevant provisions is somewhat neutralised by a requirement of significant harm, or the likelihood thereof, which is hard to prove in the case of invasive advertising practices. How to measure the harm in the context of manipulative commercial practices remains unclear, although the Recital indicates that “unfair commercial practices leading to economic or financial harms to consumers are prohibited under all circumstances, irrespective of whether they are put in place through AI systems or otherwise.”⁷⁶

On this point, relevant provisions may also be found outside the AI Act, notably in Directive 2005/29/EC (UCPD),⁷⁷ which, in its modernized form, notably protects European consumers against “coercion and undue influence”, understood as the act of “exploiting a position of power in relation to the consumer so as to apply pressure, even without using or threatening to use physical force, in a way which significantly limits the consumer's ability to make an informed decision.”⁷⁸ Therefore, depending on their characteristics, targeted advertising practices could sometimes amount to undue influence, although this should also be appreciated in light of consumers’ own responsibilities.⁷⁹

The 2021 Guidance submitted by the European Commission on the interpretation and application of the UCPD offers a more in-depth analysis of the mechanisms involved in corporate surveillance. Data-driven practices, dark patterns and commercial practices of social media are notably addressed.⁸⁰ Interestingly, the guidelines acknowledge that the superior knowledge extracted at the data aggregation phase, the constant fine-tuning of commercial practices on consumers to learn more about their behaviour, as well as the opacity of the practices, may help to distinguish “highly persuasive advertising or sales techniques from, on the other hand, commercial practices that may be manipulative and, hence, unfair under consumer law.”⁸¹ As the “significant harm” requirement of the AI Act is not replicated in the UCPD, it could be that its rules are easier to trigger than the AI Act’s; although it is likely that the threshold for recognising undue influence in the commercial context remains high outside of clear instances of coercion, as over-inclusive criteria risk outlawing the majority of business practices.

Early 2025, in conjunction with the priority entry into force of the provisions on prohibited practices, the European commission published its Guidelines on prohibited artificial intelligence (AI) practices, as defined by the AI Act.⁸² The document notably covers the question of the scope of the prohibition enshrined in Art. 5 para. 1, distinguishing between lawful persuasion, which “operates within the bounds of transparency and respect for individual autonomy”, and manipulation, which involves “covert techniques undermining autonomy, leading individuals to make decisions they might not have otherwise made if they were fully aware of the influences at play.”⁸³ The guidelines adds that “Both the AI Act and the UCPD aim to proactively prevent consumer harm from AI-driven business practices that are manipulative, misleading, or aggressive”⁸⁴ and clarifies that the AI Act’s requirements are broader in scope than those of the UCPD in the sense that its provisions are not restricted to consumers and commercial harm.⁸⁵ But it transpires from the guidelines that the threshold set by the AI Act’s requirements remains high. Ultimately, assessments will be made on a case-by-case basis, taking into account an array of parameters including transparency, conformity with data protection law, the vulnerability of the target, and the objective and impact of a technic, but the significant harm requirement makes it so that insidious techniques deployed by corporations to keep customers engaged with their product mostly fall outside the scope of the regulation, AI or not.

The AI Act is arguably more informative when it comes to state surveillance. Indeed, AI-driven social-scoring practices, which may be integrated in a state’s surveillance apparatus⁸⁶ and lead to discriminatory and unjust decisions being taken to restrict the right of a person, are prohibited under the AI Act. The regulation is also concerned with surveillance practices involving biometric data in the form of biometric categorisation, real-time and post-remote biometric identification in publicly accessible spaces. The first two are in principle prohibited, while the third falls into the lower category of high-risk systems (Art. 26 para. 10 AI Act). This means that they are in principle authorised so long as some safeguards are in place. The AI Act also addresses profiling in the context of law enforcement. AI systems can be involved in profiling and decision-making processes, significantly contributing to the outcome, with potentially high impact on fundamental rights.⁸⁷ Art. 5 AI Act therefore prohibits, with exceptions, risks assessments and crime prediction when based solely on profiling.⁸⁸

Finally, additional privacy-related requirements are contained in the regime for high-risk AI systems. Art. 10 AI Act indeed contains provisions on data governance, which may have some relevance to surveillance activities. Notably, Art. 10 para. 5 AI Act provides for the possibility to process special categories of data in order to mitigate biases in the outputs of an AI system. In this case, the AI Act demands that adequate privacy enhancing measures are deployed to complicate reidentification. Overall, bias reduction measures contribute to avoid unjust surveillance outcomes that may impact decisional privacy.

Be that as it may, the AI Act has been criticised for its permissive posture on real-time and post remote biometric identification, and overall lack of operational guidance. Particularly, the fact that the text still allows real-time remote biometric identification in “exhaustively listed and narrowly defined situations, where the use is strictly necessary to achieve a substantial public interest, the importance of which outweighs the risks”⁸⁹ has been described by human rights advocates as providing a “‘blueprint’ for how to conduct biometric mass surveillance practices” rather than strong privacy safeguards.⁹⁰ Others have argued that the more lenient stance on post-remote biometric identification could be easily abused by authorities.⁹¹ One might also regret the fact that AI uses in the context of national defence are excluded from the scope of the regulation, thereby leaving the door open to misclassification and in fine abuse.

b) The framework convention on AI

The Framework Convention, adopted in May 2024 by the Council of Europe and opened to signature in September 2024, is the second most influential development in the field of international AI regulation to date. The text, which constitutes the first binding convention on AI with international reach, is intended to apply to “the activities within the lifecycle of artificial intelligence systems that have the potential to interfere with human rights, democracy and the rule of law” (Art. 3 para. 1 Framework Convention). On this point, the text appears to have a broader scope than the AI Act.

The Framework Convention is the result of the work of the Committee on Artificial Intelligence (CAI), based on preliminary research carried out by the Ad hoc Committee on Artificial Intelligence. Its drafting involved the 46 Member States of the Council as well as 11 observer States, including Japan and the United States, and 68 representatives of civil society. Like the AI Act, the Convention aims to promote human rights friendly AI, by adopting a risk limitation approach (Art. 1 lit. b). However, the Convention does not create new rights but sets out a number of general principles such as human dignity and personal autonomy (Art. 7), transparency and control (Art. 8) and equality and non-discrimination (Art. 10), which draw directly from the guiding principles issued by the OECD.

As a framework Convention, the text seeks first and foremost to lay the foundations for more far-reaching international regulations in the future. Consequently, the Framework Convention is less technically detailed than the majority of national and European frameworks on the subject. The drafters chose not to name any specific activity involving AI that would fall within the scope of the text, leaving considerable room for manoeuvre for States to achieve its aims.

The Framework Convention on AI was also subject of criticism. Its broad formulation does not forecast strong effectiveness, which has led to it being heavily criticised, notably by the European Data Protection Committee (EDP).⁹² Yet, the general wording of its provisions is a consequence of its openness, as the CAI seeks to bring together States with different legal traditions, particularly in terms of AI regulation, which requires significant concessions. A follow-up mechanism provided in the form of a “Conference of the Parties” grants the Convention some degree of adaptability. However, it appears unlikely that a framework specifically addressing AI-driven surveillance activities will later be developed under the Convention. The exception expounded in Art. 3 para. 2 and 4 Framework Convention makes it that AI systems used for surveillance activities could easily fall outside its scope if they are shown to relate “to the protection of national interests.” The case of corporate surveillance is similarly uncertain under the Convention, as States are free to decide whether national private actors should be bound by the provisions of the Convention.

Hence, the Convention’s initial contribution in limiting surveillance practices is very tenuous, even though many aspects of surveillance contradict the basic rationale laid down in the explanatory document to the Convention; “[A]ctivities within the lifecycle of artificial intelligence systems should not lead to the dehumanization of individuals, undermine their agency or reduce them to mere data points […].”⁹³

2. Guidance from non-AI specific regimes

AI systems rely on data to function. Their development and use therefore implicate data processing activities.⁹⁴ Therefore, a discussion on AI regulation mobilizes way more frameworks than technology-specific regimes. The AI Act also hints several times at the GDPR which provides data subjects with several rights that are directly relevant to this discussion.⁹⁵ The text notably recognises a right to object to data processing including profiling when for the purpose of direct marketing, (Art. 21 GDPR) a right not to be subject to fully automated decision-making producing legal effect (Art. 22 GDPR), as well as a right to information and transparency regarding the logic involved, the significance and the envisaged consequences of automated decision-making for the data subject, (Art. 13 to 15 GDPR). As per Art. 23 GDPR, these rights can be restricted, among other, for national security reasons. Additionally, principles such as purpose limitation and data minimization (Art. 5 para. 1 lit. (b)(c) GDPR) also impose checks on surveillance practices.

At this point, the GDPR has been extensively discussed in the literature. Authors have underlined its inadequacy when it comes to AI-enabled surveillance. Andrew and Baker for instance argue that the GDPR’s complacency toward anonymisation and pseudonymisation “incentivize the use, collection, and trade of behavioural and other forms of de-identified data”, thereby enabling surveillance practices.⁹⁶ In the same vein, Zarsky argues that the provisions contained in Art. 22 GDPR on fully automated decision-making could be easily sidestepped by a data controller.⁹⁷ He adds that Big Data capabilities challenge the distinction between the different categories of data contained in the GDPR, with the most sensitive data extrapolatable from regular information.⁹⁸ More generally, prominent commentators have argued that the “individual control” model, on which most data protection legislations are built, is doomed, because it fails to account for the power imbalance between companies, states and individuals.⁹⁹ Joseph A. Cannataci, former UN Special Rapporteur on the right to privacy also deplored the EU’s lack of competence in the field of national security, which impedes proper oversight of surveillance policies.¹⁰⁰ Finally, recent discussions in the realm of generative AI regulation have highlighted the GDPR's poor performance in capturing the particularities of generative AI systems.¹⁰¹

Much like the GDPR, the Convention 108+ modernizing the Convention 108 for the Protection of Individuals with regard to Automatic Processing of Personal Data covers diverse aspects of AI-driven surveillance activities. Adopted in 2018, the modernization Protocol for the Convention 108 provides broad guidelines for the international protection of data worldwide which integrates provisions directly targeted at AI systems.¹⁰² Unlike the first version, the amended Convention 108+ is fully applicable to the national security domain. It also applies to both the public and the private sector, which makes it a more impactful instrument than the Framework Convention when it comes to controlling corporate surveillance.

The Convention 108+ constitutes the only existing binding treaty on privacy and data protection in the digital context. The CoE’s Committee of Minister has made multiple references to the Convention 108+, among others, at the occasion of a non-binding declaration on risks arising from surveillance technologies¹⁰³, and a recommendation dealing with automatic processing of personal data in the context of profiling.¹⁰⁴ When it comes to data processing in the context of national security, the Convention 108+ requires a test of proportionality and necessity. The Convention takes up several principles enshrined in the 2014 International Principles on the Application of Human Rights to Communications Surveillance,¹⁰⁵ a document drafted by privacy experts aiming to provide state actors with precise guidelines regarding the conduct of surveillance activities.

So far, the Protocol modernizing Convention 108+ has been ratified by 33 States, the majority being European. The updated Convention will only enter into force once this number reaches 38. It is worth noting that the United States, which hosts the most powerful digital firms, was not a party to the original Convention 108+. Given the current priorities at the White House, it is unlikely that Convention 108+ will be ratified by the US government.

V. Conclusive remarks on advancing AI privacy discussions

Privacy should play a central role in the regulation of AI tools. Current legal development on the matter however show that this is not really the case and that non-AI-specific frameworks have weaknesses. It is therefore interesting to investigate whether human rights law, which by default applies to AI technology and its uses, is up to the task of filling the gaps left by more specific frameworks when it comes to mitigating AI-enabled privacy risks. After all, both the Framework convention and the Convention 108+ limit their exceptions on national security to the respect of international human rights law. Unfortunately, due to several theoretical and structural deficiencies, international human rights law might not provide sufficiently robust baseline protection to individuals whose privacy is infringed upon by AI-driven surveillance practices.

On an abstract level, the inherent fluidity of the concept of privacy makes it difficult to operationalize in practice. The absence of a clear definition for privacy and its subjective dimensions necessitate an ongoing evaluation of the numerous expectations stemming from it. Admittedly, privacy must be considered in context, and approached as a mutable concept. It must be able to satisfactorily respond to new challenges and mitigate harms to human dignity and autonomy while simultaneously allowing society to function. But privacy cannot be toned down on the basis that individuals are giving up so many of it nowadays. Human autonomy and dignity are invariable, and as such, should always guide assessments of privacy expectations.

At present, it is difficult for individuals to understand when their data is used for AI-training purposes, especially since the issue is relatively new, large databases already exist and so is a sense of resignation over the propriety of personal data.¹⁰⁶ While the constant bargaining taking place online is a source of privacy risks that the principle cannot eliminate entirely, societies cannot afford to allow countervailing considerations to prevail over privacy in the constant checks and balances imposed by ubiquitous computing environment.¹⁰⁷ This means, first and foremost, that data collection must be rationalized, in the sense of empowering the data subject to make free decisions regarding the amount of personal information that is relinquished in exchange for a service or goods. In situations where consent can be circumvented, it is imperative to ensure transparency with regard to the collection, use and the anticipated outcomes of such data processing operations.

Still, a blatant issue with privacy as enshrined in the various existing international documents is that it is centred around the individual and thus, struggles to accommodate collective needs related to data processing. If anything, surveillance activities are societal-scale undertakings. Profiling virtually concerns billions of users. At the individual level, the right to privacy may offer some degree of protection against data misuses, but it cannot address the full picture. Indeed, due to the situation of quasi-monopoly of a few companies, users do not really possess a negotiating power regarding the trade of data for services. This power imbalance, which favours the acceptance of companies’ terms and conditions, counteracts any claim of arbitrariness and, ultimately, limits the relevance of the right to privacy as a safeguard since data subjects more often than not enter into data transaction without a proper understanding of the implications.¹⁰⁸ Even when basic privacy requirements are satisfied, the content of the right to privacy becomes gradually shallower as more data is required to access common services, and more data-sensitives activities are integrated in everyday life interactions.

The very individualistic and consent-oriented understanding of privacy as enshrined in international instruments is therefore lacklustre. Further advance in the protection of users’ data against corporate surveillance will not come from the current approach, but from rebalancing the bargain between users and providers. As expressed by some authors; “A privacy and data protection framework that places the primary responsibility on individuals to manage their data across hundreds, even thousands, of digital relationships and channels fundamentally does not scale, and thus will not succeed in protecting individual privacy.”¹⁰⁹ The concern was recently echoed by Solove and Hartzog, who called for the application of a “societal structure” model of privacy regulation also embracing AI.¹¹⁰

Another noticeable impediment to the performance of a human rights framework in the present case relates to the fact that human rights law is principally state-centric. While states are directly bound by the human rights treaties to which they commit, in addition to certain customary human rights which are binding upon all states, private entities are not directly bound by international human rights law. This is why State participation in instruments such as the International Covenant on Civil and Political rights is crucial. Although modern developments in the realm of human rights law have recognized the human rights responsibilities of private actors,¹¹¹ these actors are only liable for human rights harm under national law. It is thus the primary responsibility of states to ensure that the right to privacy is respected within their borders.

In addition, business firms usually enjoy greater freedom when it comes to data practices under the right to the freedom to conduct business, which may be used as a defence against certain claims. As a result, attempts to circumscribe the data practices of the private sector have been paradoxically weaker than for state surveillance.

Although regulators can influence the fairness of the bargain, their action is limited for several reasons. First, it might be difficult for regulators, to assess what is necessary when providers offer a wide range of services requiring various data to function properly, such as location and browsing data. Second, regulators might feel pressure to avoid undermining innovation and development in the digital sector, especially when national firms are concerned. Third, the promise of better population control through the use of AI technology is inherently attractive for authorities, and effectively curtails the right to privacy.

Nevertheless, the balance of interests between individuals, corporations and states must be readjusted, and new approaches to privacy might be the key. This difficult endeavour can only stem from national or regional initiatives, since value decisions are beyond the scope of international human rights law. To this end, the current momentum around AI regulation should be exploited fully. The longstanding paradigm revolving around the commodification of personal information for the purpose of influencing behaviours, especially when the objectives sought are of economic nature, needs to be challenged.

Vitae

Dr. William Letrone is a CNRS postdoctoral researcher in cybersecurity and data protection law at Nantes University, France and a member of the IPoP project.

Dr. Tony Cabus is a postdoctoral researcher at the Walther-Schücking Institute for International Law in Kiel, Germany.