B Explanatory memorandum
by Ms Thórhildur Sunna Ævarsdóttir, rapporteur
1 Introduction
1. In 2021, the Committee of Ministers
of the Council of Europe decided to create an
ad
hoc Committee on artificial intelligence (CAI) and entrust
it with the preparation of an “appropriate legal instrument on the development,
design, and application of Artificial Intelligence (AI) systems
based on the Council of Europe’s standards on human rights, democracy
and the rule of law, and conducive to innovation, which can be composed
of a binding legal instrument of a transversal character, including
notably general common principles, as well as additional binding
or non-binding instruments to address the challenges relating to
the application of AI in specific sectors”.
Note The CAI is composed of representatives
of member States, the European Union, observer States to the Council
of Europe, other interested non-European States (Argentina, Australia, Costa
Rica, Israel, Peru and Uruguay), representatives of relevant Council
of Europe bodies and committees, other international organisations,
representatives of civil society and the private sector. It has
held 10 meetings between April 2022 and March 2024 to finalise a
draft
Framework Convention on Artificial Intelligence, Human Rights, Democracy
and the Rule of Law (“the draft Framework Convention” or “DFC”). The CAI
submitted the DFC to the Committee of Ministers on 15 March 2024
and the Committee of Ministers transmitted it to the Parliamentary
Assembly on 20 March 2024, inviting the Assembly to give an opinion
on the text as soon as possible. The Assembly committed itself to
give an opinion during the April 2024 part-session, in order to
allow for the adoption of the Framework Convention by the Committee
of Ministers in due time. These timings imply that the opinion needed
to be adopted by the Committee on Legal Affairs and Human Rights
(once seized by the Bureau of Assembly) and debated in the plenary
under urgent procedure. The committee appointed me rapporteur at
its meeting of 4 March 2024, subject to the finalisation of the
DFC and its transmission to the Assembly.
2. In my capacity as former Chairperson of the Assembly’s Sub-Committee
on Artificial Intelligence and Human Rights (2022-2023), I have
participated actively throughout the process of negotiation of the
DFC and attended several plenary meetings of the CAI on behalf of
the Assembly. Like other Council of Europe bodies and civil society
representatives, I was given the chance to submit written comments
and drafting proposals on the text and present them orally where
necessary.
Note However, I could only participate
in the plenary meetings, not in the drafting group sessions which
were restricted to potential Parties to the Framework Convention
and preceded the plenary meetings. I would like to stress that some
of the final compromises reached on the text (particularly on its
scope of application) were decided by the drafting group at the
very last stages in the process and were therefore submitted to
the CAI plenary with no time for meaningful analysis and debate.
3. This Framework Convention will be the first ever legally binding
international treaty on artificial intelligence. I am aware that
the preparation and negotiation of the DFC has been a complex task
and process, given the need for reconciling different positions
of States from different parts of the globe. The Convention’s added
value will also be its global reach, as it will bring together Parties
from different continents and with diverse legal traditions wishing
to regulate AI from the perspective of human rights, democracy and
the rule of law. Artificial intelligence is an area which raises
world-wide challenges that need to be addressed and regulated globally.
In the Reykjavik Declaration of May 2023, the heads of State and
government of the Council of Europe understood the importance of
this topic and committed to “ensuring a leading role for the Council
of Europe in developing standards in the digital era to safeguard
human rights online and offline, including by finalising, as a priority,
the Council of Europe’s Framework Convention on Artificial Intelligence”.
Note The Committee of Ministers now has the responsibility
to ensure that the final convention is fully in line with the principles
and values of the Council of Europe.
4. In my explanatory memorandum, I will start by outlining the
previous and current work of the PACE on AI (chapter 2). I will
then set out briefly the main features of the DFC (chapter 3). I
will present the criticisms on the DFC voiced by different stakeholders
(chapter 4). Finally, I will give my own assessment of the draft
text and suggest some amendments that could be proposed in the Assembly’s
opinion, in order to improve the final text of the Framework Convention
(chapter 5).
2 Previous and current work of the Assembly
on Artificial Intelligence
5. The Assembly’s previous work
on the topic of artificial intelligence shows that its position
has always been in favour of striking the right balance between
mitigating the risks posed by AI and making full use of the advantages
that it can offer. The Assembly strongly believes that there is
a need to create a cross-cutting regulatory framework for AI, with
specific principles based on the protection of human rights, democracy
and the rule of law.
6. In October 2020, the Assembly adopted a set of resolutions
and recommendations (based on 7 reports prepared by its different
Committees)
Note, examining
the opportunities and risks of AI for democracy, human rights and
the rule of law. The Assembly endorsed a set of key ethical principles
that should be respected when developing and implementing AI applications.
These principles, which were further elaborated in a common appendix
to the reports, are: transparency, including accessibility and explicability;
justice and fairness, including non-discrimination; human responsibility
for decisions, including liability and the availability of remedies;
safety and security; and privacy and data protection. However, the
Assembly considered that “self-regulatory ethical principles and
policies voluntarily introduced by private actors are not adequate
and sufficient tools to regulate AI, as they do not necessarily
lead to democratic oversight and accountability”. In each of the situations
examined in its reports, the Assembly concluded that legal regulation
was necessary in order to avoid or minimise the potential risks
to democracy, human rights and the rule of law. It therefore called
on the Committee of Ministers to support the elaboration of “a legally
binding instrument governing artificial intelligence, possibly in
the form of a convention” [that is] “based on a comprehensive approach,
deals with the whole life cycle of AI-based systems, is addressed
to all stakeholders, and includes mechanisms to ensure the implementation
of this instrument” (see paragraph 4 of
Recommendation 2181 (2020) “Need for democratic governance of artificial intelligence”).
Such a convention should be open to non-member States (see paragraph 11.1
of
Recommendation 2185
(2020) “Artificial intelligence in health care: medical, legal
and ethical challenges ahead” and paragraph 2 of
Recommendation 2186 (2020) “Artificial intelligence and labour markets: friend
or foe?”).
7. In its resolutions, the Assembly considered that private actors
should fall within the scope of the above-mentioned legally binding
instrument. For instance, in
Resolution
2341 (2020) “Need for democratic governance of artificial intelligence”,
the Assembly stated that such instrument should “contain provisions
to limit the risks of the use of AI-based technologies by State
and
private actors to control
people” (paragraph 14.5) and “contain safeguards to prevent the
threats to democratic order resulting from the concentration of data,
information, power and influence in the hands of a few major
private companies involved in developing and
providing AI-based technologies and services (…), as well as provisions
that the activity of such actors is subject to democratic oversight”
(paragraph 14.6). In the Assembly’s view, the legally binding instrument
on AI should indeed be addressed to
all
stakeholders (paragraph 4.2 of
Recommendation 2181 (2020).
8. In its
Resolution
2341 (2020) “Need for democratic governance of artificial intelligence”,
the Assembly considered that the advocated legally binding instrument
should ensure that AI-based technologies comply with the Council
of Europe’s standards (on human rights, democracy and the rule of
law), as well as with the key ethical principles mentioned above.
It should not only minimize the risk that AI is misused to damage democracy
(e.g. through interference with electoral processes and manipulation)
but also seek to maximise the possible positive impact of AI on
the functioning of democracy, including through improving government accountability,
the fight against corruption, transparency and making democracy
more direct. In order to ensure accountability, the legal framework
should provide for an independent and proactive oversight mechanism, involving
all relevant stakeholders, which would guarantee effective compliance
with its provisions.
9. In its
Resolution
2343 (2020) “Preventing discrimination caused by the use of artificial
intelligence”, the Assembly underlined that many uses of AI can
have a direct impact on equality of access to fundamental rights. They
can also cause or exacerbate discrimination, leading to denials
of access to rights that disproportionately affect certain groups
(e.g. women, minorities). The Assembly therefore called on member
States to draw up clear national legislation, standards and procedures
to ensure that AI-based systems “comply with the rights to equality
and non-discrimination wherever the enjoyment of these rights may
be affected by the use of such systems”. Governments should notify
parliaments before AI-based technologies are deployed by public authorities.
Member States should promote the inclusion of women, girls and minorities
in science and technology education paths, support research into
data bias and promote digital literacy. In its
Recommendation 2183 (2020), the Assembly called on the Committee of Ministers to
take into account the particularly serious potential impact of the
use of AI on the rights to equality and non-discrimination when assessing
the feasibility of an international legal framework for AI.
Note
10. In its
Resolution
2342 (2020) “Justice by algorithm – The role of artificial intelligence
in policing and criminal justice systems”, the Assembly called for
national legal frameworks to regulate the use of AI in police and
criminal justice work, based on the core ethical principles mentioned
above. It stressed that the use of AI in policing and criminal justice
systems risks having a particularly serious impact on human rights
if it is not properly regulated. Its use may be inconsistent with
the core ethical principles, including transparency, human responsibility,
justice and fairness. In this context, the Assembly recommended
inter alia that member States maintain
a register of all AI applications in use; ensure that there is a
sufficient legal basis for every AI application; conduct initial
and periodic human rights impact assessments of these applications;
establish effective and independent oversight mechanisms; and ensure
that there is effective judicial review. In its
Recommendation 2182 (2020), the Assembly also called on the Committee of Ministers
to take into account the particularly serious impact on human rights
of the use of AI in this context when assessing the feasibility
of a European legal framework.
11. In its
Recommendation
2185 (2020) “Artificial intelligence in health care: medical, legal
and ethical challenges ahead”, the Assembly underlined its support
for an AI convention with an emphasis on the human rights implications
of AI in general and on the right to health in particular. It recommended
the Committee of Ministers to encourage member States to build comprehensive
national strategies for AI use in healthcare, set up systems to
evaluate and authorise health-related AI applications and elaborate
a legal framework for clarifying the liability of stakeholders.
The Assembly also stressed the need to ensure “privacy, confidentiality and
cyber-safety of sensitive personal health data” (to avoid any sovereign
or commercial misuse of such data) and the informed consent of users
of AI-driven healthcare applications. Such applications should “not
replace human judgment completely” so that decisions in professional
healthcare “are always validated by adequately trained health professionals”.
12. In its
Resolution
2345 (2020) “Artificial intelligence and labour markets: friend
or foe?”, the Assembly alerted to the fact that “AI used unwisely
has the potential to disrupt the labour market, fragmenting professional lives
and exacerbating socio-economic inequalities”. The Assembly noted
that “the use of AI for recruitment and in situations impacting
workers’ rights should always be treated as “high risk” and hence
heightened regulatory requirements should apply”; it also highlighted
the importance of “substantive human oversight in the implementation
of AI technology that affects labour markets and individual social
rights” and the need to “guarantee that any use of surveillance
techniques at the workplace is subject to special precautions in
terms of consent and privacy protection”. Therefore, the Assembly
called on member States to adopt a series of measures in this regard,
such as drafting national strategies for responsible AI use, requiring
AI developers to always notify users whenever they are in contact
with AI applications, designing a regulatory framework that promotes
complementarity between AI applications and human work and ensures
proper human oversight, as well as ensuring that algorithms used
in the public sphere are understandable, transparent, ethical and
gender sensitive. The Assembly further advocated for introducing
“AI literacy” through digital education programmes for young people
and lifelong learning paths for all. In its
Recommendation 2186 (2020) on the same subject, the Assembly recommended that the
European legal instrument on AI cover the need for enhanced protection of
work-related social rights.
13. In its
Resolution
2344 (2020) “The brain-computer interface: new rights or new threats
to fundamental freedoms?”, the Assembly referred to “the huge potential
benefits of neurotechnology, especially in the medical field”, while
stressing the “unique and unprecedented threats to fundamental values
of human rights and dignity” that brain-computer interfaces (BCI)
may pose. BCI technology should be developed with respect for human
rights and dignity; it should be safe; it should not be used against
a subject’s will or in a way that prevents the subject from acting
freely and being responsible for their actions; and it should not
create a privileged status for its users. The Assembly therefore
called on member States to establish ethical and legal frameworks
for research, development and application of BCI technology and
to consider the establishment of new “neurorights” such as cognitive
liberty, mental privacy, mental integrity and psychological continuity. Finally,
in its
Recommendation
2184 (2020), it called on the Committee of Ministers to take into
account the impact on human rights of AI in connection with BCI
systems, when assessing the feasibility of a legal framework for
AI.
14. In its
Resolution
2346 (2020) “Legal aspects of ‘autonomous’ vehicles”, the Assembly
stressed that the circulation of semi-autonomous or autonomous vehicles
may create a “responsibility gap”, where the human in the vehicle
cannot be held liable for criminal acts or torts. New approaches
to apportioning criminal or civil liability or alternatives to such
liability may be required. It considered that since modern automated
driving systems (ADS) rely on AI, ethical and regulatory standards
applicable to AI in general should also be applied to its use in
autonomous vehicles. Member States should ensure that the relevant
regulation is in accordance with human rights and rule of law standards,
including respect for the right to life, privacy rights and legal certainty.
In its
Recommendation
2187 (2020), the Assembly recommended the Committee of Ministers
to take into account the particularly serious potential impact on
human rights of the use of AI in ADS when assessing the necessity
and feasibility of a legal framework for AI.
15. Most recently, in its
Resolution
2485 (2023) “Emergence of lethal autonomous weapons systems (LAWS)
and their necessary apprehension through European human rights law”,
the Assembly addressed the concerns regarding the emergence of LAWS
(weapon systems that can attack and select targets without human
intervention) The Assembly supported clear regulation of the development
and use of LAWS to ensure respect for international humanitarian
law and human rights, on the basis of human control. In this context,
it endorsed a two-tier approach proposed by a group of European
States Parties to the Convention on Certain Conventional Weapons
(CCW). Under this proposal, LAWS operating completely outside human
control should remain banned, while other lethal weapons that have
elements of autonomy should be subject to appropriate human control,
human responsibility and accountability, measures to mitigate risks
and appropriate guarantees. According to this resolution, the appropriate
forum to agree on the future regulation of LAWS is the Conference
of the States Parties to the CCW. Pending agreement on a binding
regulation (in the form of a protocol to the CCW), the Assembly
proposed the drawing up of a non-binding code of conduct.
16. In addition to its reports and resolutions, the Parliamentary
Assembly has organised or participated in several events focused
on AI.
Note The
Assembly’s President Theodoros Rousopoulos named AI as one of his main
priorities in his inaugural address to the Assembly at its first
part-session of 2024 and stated: “
AI
is to be welcomed and it must remain a tool to assist human capabilities
and not a substitute for human will and autonomy”. The
Assembly has a dedicated Sub-Committee on Artificial Intelligence
and Human Rights (within the Committee on Legal Affairs and Human
Rights), currently chaired by Damien Cottier (ALDE/Switzerland). During
my chairmanship, the Sub-Committee held a meeting and an exchange
of views on social media bots and the threat they pose to democratic
discourse and elections. The Assembly will continue to work on the challenges
and benefits of AI for human rights, democracy and the rule of law,
through new reports and resolutions on emerging or sectorial topics
Note and by following
closely and contributing where necessary to the standard-setting
work of the CAI and other relevant inter-governmental bodies.
Note It
would obviously also wish to participate as an observer in the future
follow-up mechanism set up by the Framework Convention (the Conference
of the Parties).
3 Main features of the Draft Framework
Convention
17. Chapter I of the DFC contains
general provisions such as the object and purpose of the framework convention
(Article 1), the definition of artificial intelligence systems for
the purposes of the convention (Article 2) and its scope of application
(Article 3). As explained in the
Draft
Explanatory Report (“DER”), “no provision of this Framework Convention
is intended to create new human rights obligations or undermine
the scope and content of the existing applicable protections, but
rather, by setting out various legally binding obligations contained
in its Chapters II to VI, to facilitate the effective implementation
of the applicable human rights obligations of each Party in the
context of the new challenges raised by artificial intelligence”.
As regards the scope, the most challenging and disputed provision
throughout the negotiations, Article 3.1 (a) obliges each Party
to “apply the Convention to the activities within the lifecycle
of artificial intelligence systems undertaken by public authorities,
or private parties acting on their behalf”. Article 3.1 (b) obliges
all Parties to “address risks and impacts arising from activities
within the lifecycle of artificial intelligence systems by private
actors to the extent not covered in subparagraph (a) in a manner
conforming with the object and purpose of the Convention”. As explained
in the DER, “addressing risks is not merely acknowledging those
risks, but requires the adoption or maintaining of appropriate legislative,
administrative or other measures to give effect to this provision
as well as co-operation between the Parties (…)”. Each Party is
obliged to specify in a declaration “how it intends to implement
this obligation, either by applying the principles and obligations
set forth in Chapters II to VI of the Framework Convention to activities
of private actors or by taking other appropriate measures”. Although
Parties may amend their declarations at any time, the DER states
that “for Parties that have chosen not to apply the principles and
the obligations of the FC in relation to activities of other private
actors, the Drafters expect the approaches of those Parties to develop
over time as their approaches to regulate the private sector evolve”. Article
3.2 excludes from the scope of the FC “activities within the lifecycle
of artificial intelligence systems related to the protection of
its national security interests, with the understanding that such
activities are conducted in a manner consistent with applicable
international law, including international human rights law obligations,
and with respect for its democratic institutions and processes”.
The DER clarifies in this respect that “all regular law enforcement
activities for the prevention, detection, investigation, and prosecution
of crimes, including threats to public security, also remain within
the scope of the FC if and insofar as the national security interests
of the Parties are not at stake”. Finally, Articles 3.3 and 3.4
exclude “research and development activities” under certain conditions
(“unless testing or similar activities are undertaken in such a way
that they have the potential to interfere with human rights, democracy
and the rule of law”) and “matters relating to national defence”.
18. Chapter II contains two provisions enshrining general obligations
on the protection of human rights (Article 4) and integrity of democratic
processes and respect for the rule of law (Article 5). While the
reference to human rights must be understood as meaning human rights
obligations under international treaties already binding on the
Parties (a list is provided in the DER) and domestic law protections,
the DER acknowledges that there is no commonly agreed definition
of “democratic institutions and processes” and gives some examples of
areas where AI may pose risks: the principle of separation of powers,
judicial independence, access to justice (these three also mentioned
in the text of Article 5.1), an effective system of checks and balances, political
pluralism and free and fair elections.
19. Chapter III lays down “general common principles that each
Party shall implement in regard to artificial intelligence systems”.
These include human dignity and individual autonomy (Article 7),
transparency and oversight (Article 8), accountability and responsibility
(Article 9), equality and non-discrimination (Article 10), privacy
and data protection (Article 11), reliability (Article 12), and
safe innovation (Article 13). Chapter IV regulates remedies and
procedural guarantees (Articles 14 and 15). Remedies for violations
of human rights resulting from the activities within the lifecycle
of AI systems must be accessible and effective. Effective procedural
guarantees shall be available where an AI system significantly impacts
on the enjoyment of human rights. Chapter V contains a unique and
dedicated provision on “Risk and impact management framework”, obliging
the Parties to “identify, assess, prevent and mitigate risks posed
by artificial intelligence systems by considering actual and potential
impacts to human rights, democracy and the rule of law” (Article
16.1). As explained in the DER, the purpose of this provision is
to ensure a uniform approach to risk and impact assessment, while
leaving a certain degree of flexibility in the measures the Parties
choose, which should be in any case “graduated and differentiated,
as appropriate” (Article 16.2). Within this framework, Article 16.4 states
that Parties “shall assess the need for a moratorium or ban or other
appropriate measures in respect of certain uses of AI systems where
it considers such uses are incompatible with the respect of human
rights, the functioning of democracy or the rule of law”. The determination
of what is “incompatible” in this context is made by each Party,
according to the DER.
20. Chapter VI on “Implementation of the Convention” contains
transversal or interpretative clauses: on non-discrimination (Article
17), rights of persons with disabilities and of children (Article
18), public consultation (Article 19), digital literacy and skills
(Article 20), safeguard for existing human rights (Article 21) and
wider protection (Article 22).
21. Chapter VII contains provisions on the “Follow-up mechanism
and co-operation”. Article 23 sets up a Conference of the Parties
as a follow-up mechanism. This Conference will be composed of representatives
of the Parties. It will have the power to identify problems in the
use and implementation of the Convention; make proposals for amendment
and formulate its opinion on any amendment proposals; express specific recommendations
on any question concerning its interpretation or application; facilitate
the exchange of information; facilitate the friendly settlement
of disputes; and facilitate co-operation with relevant stakeholders (Article
23.2 as interpreted in the DER). Although there is no required periodicity
of convocation, Article 23.2 states that Parties “shall consult
periodically” and Article 23.4 imposes an initial obligation for
the Conference of the Parties to adopt its rules of procedure “within
12 months of the entry into force”. Interestingly, Article 24 establishes
a reporting obligation to the Conference of the Parties “within
the first two years after becoming a Party and then periodically
thereafter”. This reporting obligation on States covers also the
activities each Party undertakes to give effect to the obligation
set out in Article 3.1 (b) (address risks from private actors).
Article 25 sets out provisions on international co-operation between
the Parties (mainly exchange of information). Article 26 requires
Parties to “establish or designate one or more effective mechanisms
to oversee compliance with the obligations in the Convention” (“effective
oversight mechanisms”). Parties may indeed establish new dedicated
mechanisms or structures or adapt or redefine the functions of existing
ones. Parties have an obligation to facilitate or promote co-operation
among the different designated mechanisms and between these and
existing domestic human rights structures (Article 26.3 and 4).
22. Chapter VIII “Final clauses” contains clauses which are similar
to those contained in other Council of Europe Conventions. It is
important to note that in accordance with Article 34, the only reservation
that a State may make in respect of the Convention is the one provided
for in Article 33.1 (so-called “federal clause”). No other reservations
are possible.
4 Criticisms voiced by different stakeholders
23. At the final stages of the
negotiations of the FC (March 2024), a joint open letter to States
and the EU was published by civil society organisations. The signatories
included citizens, academics, experts in digital technologies and
civil society organisations that had been observing the negotiations,
including the Conference of INGOs of the Council of Europe (CINGO)
Note. In the letter, the
signatories called on the negotiators “to cover equally the public
and private sectors” and “to unequivocally reject blanket exemptions
regarding national security and defence”. They stressed that leaving
the private sector out of the scope would result in “giving these
companies a blank check rather than effectively protecting human
rights, democracy and the rule of law”. As regards the blanket exemption
on national security and defence, they argued that “nothing justifies
the unconditional waiving of the safeguards set in international,
European and national law that usually apply in these fields”. This
letter joined similar initiatives that had been addressed to the
EU delegation
Note and the Biden Administration.
Note In the letter addressed to Secretary
of State Antony Blinken, it was stated as follows: “if the United
States Department of State is specifically concerned about the alignment
of the treaty with domestic law, then the solution is to take a
derogation or exception to those provisions that you believe may
be incompatible with US law. It is not to ask every other nation
at the table to sacrifice safeguards they are prepared to establish”.
24. The European Data Protection Supervisor (EDPS), in a statement
published on 11 March 2024 in view of the last plenary meeting of
the CAI, raised several concerns. The EDPS was concerned that “the
very high level of generality of the legal provisions of the draft
Framework Convention, together with their largely declarative nature,
would inevitably lead to divergent application of the Convention,
thus undermining legal certainty, and more generally its added value”.
As concerns its scope, recalling that the Committee of Ministers had
instructed the CAI to elaborate a “binding legal instrument of a
transversal character”, it stressed that “any limitation of the
scope of the future Framework Convention only to the activities
undertaken by public authorities or entities acting on their behalf
would contradict the overall policy objective of the Framework Convention”. The
EDPS further expressed concerns about “the absence of ‘red lines’
in the draft Framework Convention, which from the outset prohibit
AI applications posing unacceptable levels of risk”. In this regard,
“the latest drafts of the Framework Convention and its Explanatory
Report offer neither clear and unambiguous criteria, nor any specific
examples of prohibited uses”.
Note
25. The European Network of National Human Rights Institutions
(ENNHRI), which participated in the negotiations as an observer,
delivered the following statement of concern at the end of the final
plenary session of the CAI on 14 March 2024.
Note
“ENNHRI participated actively throughout the
negotiations as an observer. We have asked for opportunities to
engage in the negotiation process and have consistently contributed
with a view to a Convention embedding a human rights-based approach
to artificial intelligence. While obstacles were encountered along
the way, we have witnessed commendable goodwill to find solutions
for tackling these obstacles.
From a human rights perspective,
we are pleased to see the inclusion in the draft Convention of some possibility
to lodge a complaint, the attention to human dignity and individual
autonomy, and references to public consultation processes. These
formulations demonstrate a commitment to protect human rights, democracy
and rule of law.
Yet, we believe that for the
Convention to guarantee high standards of human rights protection,
certain aspects cannot be overlooked:
– First, the Convention should
have covered both the public and the private sector equally. This was recommended by different CoE instruments
and standards, including the Recommendation CM/Rec(2020)1 of the
Committee of Ministers to member States on the human rights impacts
of algorithmic systems and the Recommendation CM/Rec(2022)13 to
member States on the impacts of digital technologies on freedom
of expression. It is also in line with the commitment in the Reykjavik
Declaration "for the Council of Europe to play a leading role in
developing standards in the digital era to safeguard human rights
online and offline.” Since many AI systems are developed and deployed
by private entities, the private sector is a central actor in this
field, with critical impacts on how AI affects human rights, democracy,
and the rule of law. A Convention introducing a differentiated approach
for the private sector, including non-binding regulation, creates
an important protection gap.
– Secondly, the scope of the
Convention should equally include AI systems used for national
security, such as heightened surveillance,
data collection, and decision-making processes aimed at countering perceived
threats to national security, could present significant risks to
human rights, democracy and rule of law. They could affect freedom
of movement, of peaceful assembly and association and of expression
and the right to non-discrimination, participation and privacy,
among others. An exemption on national security from the scope of
the Convention leads to a gap in the protection of human rights. Consistent
with the European Convention on Human Rights (ECHR) and the International
Covenant on Civil and Political Rights (ICCPR), national security
could constitute a legitimate basis for restrictions, yet is subject
to the provisions of these Conventions.
– Thirdly, ENNHRI is concerned
that other essential elements to ensure a human rights-based
approach to AI are missing in the draft Convention. The failure to include elements, such as
for example clear and unambiguous criteria for bans, undermines
principles of accountability and transparency which are needed to
ensure effective human rights protection. The absence of sufficiently
robust and independent oversight mechanisms foreseen by the draft
Convention at the Council of Europe and domestic levels exacerbates
these concerns.
– Furthermore, imprecise
language dilutes commitments and raises concerns
about the enforceability and effectiveness
of the obligations outlined in the draft Convention. This includes
recurrent use of ‘seek to ensure’ or ‘where practicable’ and ‘where
appropriate’ in the draft Convention. The frequent references to
'in accordance with domestic law' conflict with the Convention's
task of ensuring common international legal standards.
ENNHRI urged the CAI on multiple
occasions to address these issues, with a view to ensure an effective CoE
Convention that protects human rights, democracy and rule of law
in the context of AI.
ENNHRI calls on the Committee
of Ministers to ensure effective human rights protection in the Convention,
thereby addressing our concerns”
26. The European Network of Equality Bodies (Equinet), which also
participated in the negotiations as an observer, publicly raised
similar concerns on 15 March 2024.
Note Equinet expressed the view that
it is important to clearly acknowledge the need for improvement
of certain aspects of the FC, not only for transparency and accountability
purposes, but also with a view to the implementation of the convention
in order to identify possible solutions for overcoming the identified
gaps.
27. The Council of Europe Commissioner for Human Rights, in a
statement published on 13 March 2024, hoped that the outcome of
the negotiations would be a “solid human rights treaty that effectively
addresses the adverse impacts of AI on individuals and society and
expands the predictability and dependability of the use of the AI
systems world-wide”. She further recommended as follows.
Note
“Given the speed and reach
of technological advances today, the Convention should include a
positive obligation for states to create a legal and regulatory
framework that effectively protects individuals from all human rights
violations, whether these are committed by public or private sector
actors. Any exemptions from the scope of the Convention should be
based on law and necessary and proportionate in a democratic society.
It is also essential for the Convention to establish clear criteria
for identifying and banning unacceptable risks to human rights,
democracy and the rule of law across states parties. Finally, it
should explicitly safeguard the right to an effective remedy, including
the right to human review of automated decisions, aligned with Article
13 of the European Convention on Human Rights.”
28. Jan Kleijssen, member of ALLAI (independent organisation dedicated
to foster responsible AI) advisory board and former Director of
Information Society and Action against Crime at the Council of Europe,
also made some public comments following the finalisation of the
DFC by the CAI.
Note
“While many have celebrated the agreement there
has also been much criticism on both procedure and final outcome
(…) I do agree with those who are unhappy with the way the negotiation
process developed, away from the envisaged multistakeholder model.
(…) it is no secret that the agreed draft falls short of the high
expectations based on the final report of the CAHAI, the pioneering
Committee that prepared the ground for the negotiations. Like many
others, I pushed hard for a strong instrument, especially with regard
to the private sector, national security issues and environmental
impact assessments. The future Conference of the Parties can, and
should, play a crucial role in the implementation and development
of the Convention. Similar Conventional Committees, such as the
TPD and the TCY have ensures that the respective Conventions on
Data Protection and Cybercrime became effective global benchmarks
– with State Parties on all continents.”
29. CINGO (on behalf of its President and Steering Committee on
AI) also submitted to the Assembly certain recommendations in view
of the preparation of its Opinion. They raised three points and
proposed concrete amendments. First, they strongly objected to the
blanket exemption for national security. National security should
rather be a legitimate ground for restrictions to the application
of the Framework Convention based on legality, necessity and proportionality.
Since the FC does not create new rights but only adapts them to
the AI context, the loophole of this Framework Convention for national
security alone cannot be justified. Secondly, health and environment
should be reintroduced in the list of principles (Chapter III) or
alternatively in the provision on risk and impact management framework
(Article 16). Thirdly, the DFC uses imprecise language throughout
the text which may lead to legal uncertainty and unenforceability.
CINGO therefore proposes to replace expressions such as “seek to
ensure” in all key provisions with “ensure” (for instance, Articles
5.1 and 15.2).
30. On 11 April 2024, I received a submission from Amnesty International
containing several recommendations and improvements to the DFC addressed
to the Assembly (as well as to the Committee of Ministers).
Note These include: drawing clear red
lines on AI-based practices that are incompatible with human rights,
such as systems used for public facial recognition, predictive policing,
etc; rejecting blanket exemptions for national security, defence,
and research and development; including in the scope all private
and public actors; ensuring a comprehensive rights-based framework;
ensuring effective transparency and accountability for AI developers
and deployers; ensuring effective human rights due diligence throughout
the AI lifecycle; ensuring effective redress and remedy for impacted
people and communities; establishing clear obligations to support
meaningful involvement of impacted communities, civil society organisations
and human rights experts; and establishing clear obligations to
ensure consistent and effective application of the Convention.
5 Assessment and possible improvements
5.1 Positive aspects of the Draft Framework
Convention
31. First of all, the Framework
Convention will be the first global treaty on AI based on standards
of human rights, democracy and the rule of law. The fact that it
is a convention open to non-member States that have participated
in the negotiations and other non-European countries which might
join from all over the world at a later stage is crucial. As I said
in my introduction, the challenges and risks posed by AI are transnational
and need to be tackled in a global instrument. For this reason,
I fully understand that the framework convention has been drafted
in such a way that it will facilitate ratification or accession
by States having different legal and political traditions, allowing
for a sufficient level of flexibility and margin of discretion in
its implementation. Furthermore, the fact that the first international
treaty regulating AI will be based on a human-rights approach and
the acquis of the Council
of Europe (human rights, democracy and the rule of law) must be
acknowledged as a very important achievement for the Council of
Europe and as a sign of its global leadership in setting standards
in emerging areas. We have seen this before with previous open Council
of Europe conventions such as the Budapest Convention (Cybercrime)
and the Data Protection Convention (CETS No. 108).
32. I am also pleased to see that most of the key ethical principles
endorsed by the Parliamentary Assembly in its 2020 reports are reflected
in one way or the other in the text of the DFC: transparency, including accessibility
and explicability (in Article 8); justice and fairness, including
non-discrimination (in Articles 10, 14-15 on remedies and procedural
guarantees, and 17); responsibility (in Article 9 on accountability
and responsibility); safety and security (in Article 12 on reliability)
and privacy (in Article 11). I also welcome the inclusion of the
overarching principle of human dignity and individual autonomy,
as the basis of all human rights.
Note The Assembly has already considered
that certain technologies such as brain-computer interfaces have
the potential to threaten human rights and more generally human
dignity, therefore stressing the importance of applying in this
context the principles of capacity, autonomy, human agency and responsibility.
Note
33. Another significant added value of this convention is that
is not only intended to protect human rights in the context of AI,
but also the two other core values of the Council of Europe, democracy
and the rule of law (see Article 5, as reflected in the title of
the convention itself). Artificial intelligence technologies have
a potential to disrupt the functioning of democratic institutions
and processes, for instance through interference in electoral processes,
misinformation and manipulation of public opinion. They can also
have an impact on the functioning of the rule of law, including
the independence and integrity of the judiciary and access to justice.
For instance, the use of AI by the police and criminal justice systems
raises serious issues with regard to the principles of human responsibility
for decision-making, justice and fairness.
Note While
there is no common definition of “democratic institutions and processes”
in the text of the Convention, the DER gives a non-exhaustive list
of principles that can be potentially threatened by AI: separation
of powers, political pluralism, judicial independence, free and
fair elections, rule of law, etc. In my opinion, the interpretation
of Article 5 should also be based on the interdependent notions
of democracy and rule of law as developed over the years by different Council
of Europe bodies, particularly the European Court of Human Rights
(notions of “democratic society” and “the principle of the rule
of law” underpinning the ECHR), the Venice Commission (the Rule
of Law Checklist)
Note or the recently formulated Reykjavik
Principles for Democracy endorsed by the Council of Europe’s Heads
of State and Government.
Note
34. Other very positive aspects of the FW are the explicit reference
to gender equality (Article 10) and women (Preamble) in the context
of non-discrimination, the obligation to take due account of the
vulnerabilities of the rights of persons with disabilities and children
(Article 18),
Note the obligation to conduct public
consultations on AI-related issues “in the light of social, economic,
legal, ethical, environmental and other relevant implications” (Article
19) and the obligation to promote digital literacy and skills (Article
20). In this regard, the Assembly has stressed that the use of AI
may lead to discriminatory results or disproportionately affect
certain groups, including women, and that women’s access to professions
in science and technology should be improved for the sake of diversity
and equality.
Note It has also called on member States
to promote digital or AI literacy in general and in specific contexts.
Note
35. With regard to the follow-up mechanism, I commend the introduction
– at the very advanced stages of the negotiation – of an obligation
for the Conference of the Parties to adopt its own rules of procedure
“within 12 months of the entry into force of the Convention” (Article
23.4). This will indeed facilitate the implementation of the reporting
obligation by the first States having ratified the FC, which will
need to present their first report within the first two years after
becoming a Party. The reporting obligation enshrined in Article
24 (as part of the compromise reached on the scope of application
of Article 3) will be key to the interpretation, implementation and
development of the Convention, including with regards to the private
sector. I would have preferred to see an explicit reference in Article
23.2 to the power to
review or
monitor the implementation of the
Convention by the Conference of the Parties, but in my understanding
the power to identify any problems in the effective use and implementation
(Article 23.2 (a)) and the power to make “specific recommendations
concerning the interpretation and application” of the Convention
(Article 23.2 (c)) give a sufficient legal basis for the review
or assessment needed for an effective reporting system.
Note The
DER clarifies that “although not legally binding in nature, these
recommendations may be seen as a joint expression of opinion by
the Parties on a given subject which should be taken into account
in good faith by the Parties in their application of the Framework Convention”.
36. I also find it interesting that the Conference of the Parties
will be tasked with facilitating co-operation with “relevant stakeholders”
(Article 23.2 (f)) and that the latter may also be involved in the
exchanges of information and co-operation envisaged between the
Parties “as/where appropriate” (Article 25.2 and 3). According to
the DER, “relevant stakeholders” should include NGOs, non-State
actors (academics, industry representatives) as well as other bodies
which can improve the effectiveness of the follow-up mechanism.
Although it would have been preferable to foresee in the text of
the Convention or at least in the DER that “relevant stakeholders”, including
other Council of Europe bodies, may be invited to attend as observers
the meetings of the Conference of the Parties, I trust that the
Assembly will be given the opportunity to participate in the exchanges
of information and engage in the co-operation envisaged.
5.2 Areas for improvement in the Draft
Framework Convention and proposed amendments
37. I agree with ENNHRI and civil
society organisations that some language and qualifiers used throughout the
text may raise difficulties with regard to the enforceability and
uniform application of the Convention. For instance, the use of
“seek to ensure” instead of “ensure” (for instance, Articles 5.1
and 15.2), “where appropriate” or “as appropriate” (e.g. Articles
16.2 (g) and 19), “where practicable” (Article 26.3 and 4) and the frequent
references to “domestic law/legal system” (e.g. Articles 6, 14.1,
15.1 and 18) may be seen as undermining legal certainty and foreseeability
in the interpretation of the Convention. As regards the use of expressions
such as “seek to ensure”, there is no reason for instance to treat
differently the general obligation to ensure the respect for human
rights (Article 4) and the general obligation to protect democratic
institutions and process (Article 5). Both obligations should be
read in the light of the object and purpose of the Convention which
is to “ensure that activities
within the lifecycle of AI systems are fully consistent with human
rights, democracy and the rule of law”. As concerns the multiple
references to “domestic law”, the DER states that “all references
to domestic law in this Framework Convention should be read as limited
to cases where domestic law provides for a higher standard of human
rights protection than applicable international law” (commentary to
Article 21). If this is applied coherently, it means that international
(human rights) obligations should always be the minimum standard
applicable from which domestic law provisions cannot derogate (see
also commentary to Article 4).
38. I regret that the DFC refers to some internationally established
rights as “principles” and not as “rights” (Chapter III). For example,
the right to non-discrimination and equality or the right to privacy
should have been formulated as positive rights of individuals. This
would not have implied creating new human rights, as negotiating
States are already bound by international treaties which recognise
them as individual rights. In my view, a better option would have
been to refer in the heading of Chapter III to “fundamental principles
and rights”.
Note The
fact that it is a Framework Convention rather than a Convention
should not make a difference, as we have seen with other Framework
Conventions that refer to individual “rights” when setting out the principles
and obligations for State Parties (see the Framework Convention
for the Protection of National Minorities). More generally, I believe
that the drafters have missed an opportunity to include new rights,
rights that could flow from already existing rights (human dignity,
personal autonomy, right to respect for private life) but adapted
or specifically tailored to an AI context. For instance, when examining
the brain-computer interface technology, the Assembly called on
member States to consider establishing new “neurorights”, such as cognitive
liberty, mental privacy and mental integrity.
Note The DFC could also have contained
specific provisions on how to maximise the positive impact of AI
for democratic processes, for instance improving government accountability
and facilitating democratic action and participation.
Note Although there are references
to the “benefits” (Preamble) or “positive effect” of AI (Article
25.2 concerning the exchange of information), there are no specific
obligations on how to use the potential of AI for improving democratic
processes and the enjoyment of human rights.
39. Finally, I tend to agree with most of the critical voices
mentioned above regarding the scope of application of the FC (Article
3), the most controversial issue of the agreed draft. I had submitted
my own written comments on this issue throughout the process, stressing
that the Assembly supports as a matter of principle the widest scope
of application possible and recalling that in its AI reports it
had considered that self-regulatory principles introduced by private
actors are not sufficient. A legally binding instrument should be
based on a comprehensive approach and be addressed to
all stakeholders, including private
actors. I therefore fully supported Option A of the version published
in December 2023, which did not exclude private actors from its scope,
in line with previous PACE resolutions and the CAHAI position
Note. However, given that some negotiating States
were not ready to accept a horizontal scope covering equally both
public and private entities, different options (B and C) introducing
a differentiated approach for the private sector (with an obligation
for all Parties to take appropriate steps for the realisation of
the Convention or to progressively address the risks in respect of
private entities) were also presented. In the version sent ahead
of the final plenary meeting, new options introducing opt-out clauses
for States wishing to exclude private entities were added. In one
of the proposals, the opt-out option was subject to strict conditions
as well as to an obligation to take appropriate steps to address the
risks and impacts arising from private actors. Although I preferred
option A, an opt-out clause along these lines would have been an
acceptable alternative, giving the flexibility for certain (a minority
of) States to exclude private actors through a declaration, while
keeping by default full applicability of the Convention to private actors.
40. The current version (introduced at the very last stages of
the negotiations within the drafting group with no meaningful discussion
at the plenary) is a system rather “à Ia carte”, where each Party
will be able to determine in a declaration how it intends to address
the risks and impacts arising from the use of AI by private actors
in a manner conforming with the object and purpose of the Convention.
This is far from ideal for legal certainty and predictability of
the obligations imposed by the Convention vis-à-vis private actors.
I am of course aware of the constraints of the final stages of the
negotiations and the enormous pressure to reach a compromise. I
think that the time pressure was not necessarily beneficial for
the quality of the final outcome reached. The process followed did
not allow for a timely and in-depth analysis and consultation with
all stakeholders, including civil society and other observers participating
in the CAI. This seems to be in contrast with previous Council of
Europe practices of drafting conventions.
41. Although I would have preferred other options and more consultation
on the scope public/private, I will not propose a particular amendment
at this stage. I understand that the difficulties in reaching the
final compromise make it rather unrealistic to expect a re-opening
of the negotiations with all the parties involved on this point.
Having said that, I would like to stress that the reporting obligation
introduced as part of the compromise in Article 24 must be seen
as a positive safeguard and as a compensation for the differentiated scope
public/private. Furthermore, I expect that all member States of
the Council of Europe that will ratify the FC will opt for the full
application of Chapters II to VI when they submit the relevant declaration
on private actors under Article 3. The Assembly should strongly
call on them to do so, to be fully coherent with the mandate given by
the Committee of Ministers to elaborate a convention of a transversal
character, the previous work of the CAHAI, as well as more generally
with Council of Europe standards, including Recommendation CM/Rec(2016)3
of the Committee of Ministers to member states on human rights and
business and the case law of the European Court of Human Rights
regarding States’ positive obligations vis-à-vis private actors.
The Conference of the Parties should also play its part and conduct
a proper assessment of how State Parties comply with Article 3.1(b),
including with respect to those States that decide to take “other
appropriate measures” as an alternative to applying the Convention
to private sector activities. I trust that a progressive interpretation
of this provision by the follow-up mechanism will foster advances
over time, through reporting requirements and peer pressure.
42. I will now set out the list of possible improvements and amendments
to the DFC. These are limited to the issues that I consider sufficiently
important to be included in the opinion of the Assembly and that
seem to me reasonable with regard to the overall structure, nature
and purpose of the draft Framework Convention:
1. Replace Article 3.2 with the
following text:
“Each
Party may restrict the application of the provisions of this Convention
if activities within the lifecycle of artificial intelligence systems
are necessary to protect its national security or national defence
interests and if such activities are conducted in a manner consistent
with applicable international law, including international human
rights law obligations, and with respect for its democratic institutions
and processes.”
2. Delete Article 3.4
While I could have been ready to accept a compromise on the
issue of public/private entities based on an opt-out clause subject
to conditions, I cannot accept a full exemption of national security
from the scope of the Convention. The Council of Europe does not
have limited competences in the area of national security.Note On the contrary, measures taken by
public authorities for the protection of national security must
be subject to the rule of law and the European Convention on Human
Rights. If authorities invoke national security grounds as a justification
for using AI tools, any interference with human rights resulting
from that use must be in accordance with the law, pursue a legitimate
aim (e.g. national security) and be necessary and proportionate in
a democratic society. The Protocol amending the Convention for the
Protection of Individuals with regard to Automatic Processing of
Personal Data (CETS No. 223, Convention 108 +) also applies to the
processing of data for national security purposes, while allowing
for a number of exceptions and restrictions. For non-member States,
the International Covenant on Civil and Political Rights similarly
allows for restrictions on grounds of national security. This is
why a blanket exemption seems to me completely unjustified and contrary
to existing human rights obligations.Note The
fact that Article 3.2 refers to international human rights obligations
outside the FC or respect for democratic institutions does not change
the nature of the exemption, which is furthermore not subject to
any reporting obligation under Article 24 (unlike the exemption
related to the private sector).
The same considerations apply to the blanket exclusion of
national defence from the scope of the Convention (Article 3.4).
While the Statute of the Council of Europe excludes matters relating
to national defence, interferences with human rights resulting from
military activities are not outside the scope of human rights treaties,
including the ECHR.Note The
Assembly has in fact adopted a report on the use of AI within the
area of defence and lethal force.Note The
use of AI in ongoing armed conflicts raises serious issues with
regards to international humanitarian law and human rights law.Note If the FC applies
mainly to public entities, the exemptions regarding AI uses for
national security and defence purposes risk narrowing even more
the scope of application and calling into question its transversal
nature.
I therefore propose to treat national security and national
defence in the same way and in a single Article.Note The suggested provision
would be partly based on Option C of the DFC version of December
2023, while adding “national defence” and “with respect for its
democratic institutions and processes” and removing “essential”
from “national security interests”. The ER could also add some clarifications
as to the test established by international human rights law for
restrictions based on national security grounds (legality, necessity,
proportionality).
3. In Article 5.1, add:
“free and fair elections”
after “democratic institutions and processes, including”
It is important to explicitly refer to “free and fair elections”
as one of the main democratic institutions and processes that may
be threatened by AI. It is already mentioned in the ER as well as
in the Reykjavik Principles for Democracy. The Court has also recognised
that the rights guaranteed by Article 3 of Protocol No. 1 to the ECHR
(right to free elections) are crucial to “establishing and maintaining
the foundations of an effective and meaningful democracy governed
by the rule of law”.Note
4. Insert a new Article under
Chapter III:
“Every
Party shall adopt or maintain measures to preserve health and the
environment in the context of activities within the lifecycle of
artificial intelligence systems, in line with applicable international
and domestic law.”
In previous versions of the DFC (January and December 2023)
there was a specific provision on the preservation of health and
the environment. Although in the current version there are some
references to health and environment in the Preamble (12) and to
environmental implications in Article 19 (public consultation),
a specific provision on health and environment should indeed be
laid down expressly as a general principle under Chapter III. This
principle could be qualified by a reference to “applicable international
and domestic law”. The Assembly already recommended in 2020 that
a convention on AI should also address the implications of AI on the
right to health, having in mind the development of AI-driven health
applications and medical devices.Note In line
with the principle of human responsibility, it was also stressed
that AI health applications should not replace human judgment completely
and that AI-enabled decisions in healthcare should always be validated
by health professionals (what some have called the right to a human
doctor). The right to health is already guaranteed by the European
Social Charter, the International Covenant on Economic, Social and
Cultural Rights and the EU Charter of Fundamental Rights, and indirectly
protected through the case law of the European Court of Human Rights
(under articles 2, 3 and 8 of the ECHR).Note
As regards the protection of the environment, AI technologies
and industries have in fact considerable impacts on natural resources
and energy. Although not yet fully recognised as an autonomous human
right at the Council of Europe, the heads of state and government
in Reykjavik 2023 noted the increased recognition of the right to
a clean, healthy and sustainable environment in international and
regional human rights instruments, national constitutions, legislations
and policies and decided to strengthen the work of the Council of
Europe in this field (the “Reykjavik process”).Note The European Court of Human Rights
has recently derived from Article 8 a right for individuals to effective
protection by the State authorities from the serious adverse effects
of climate change on their lives, health, well-being and quality
of life.Note The
Court had already held that Article 8 extends to adverse effects
arising from various sources of environmental harm and risk of harm. Member
States would send a wrong message if they omitted a reference to
the environment (as a general principle) in the first international
treaty negotiated after Reykjavik. The reference to “applicable
international and domestic law” would allow for a gradual recognition
of this principle/right in line with international and Council of
Europe normative developments, reflecting the evolution of the political
will of States.
5. Add in the text of Article
14.3 (c) or in the Explanatory Report a reference to:
“judicial authorities”
or “judicial review”
The DER states that the complaints may be lodged with the
oversight mechanism(s) referred to in Article 25. A reference to
“judicial authorities” or “judicial review” would also make sense,
with the understanding that this would apply only to the extent
required by each Party’s international and domestic law obligations
(in accordance with the logic of paragraph 1 of Article 14). While
it is true that Article 13 of the ECHR does not as such require
that the national authority be strictly a “judicial authority”,
Article 6 (right to an independent and impartial tribunal) and the
Court’s case law under different Articles of the Convention may
impose in certain contexts and circumstances the right of access
to a court or to effective judicial review. The Assembly had also recommended,
in the context of policing and criminal justice systems, that the
introduction, operation and use of AI applications be subject to
effective judicial review.Note
6. In Article 15.1, add a reference
to:
“human review”
The DER indicates that effective procedural guarantees in
the context of this provision should, for instance, include human
oversight, including ex ante and ex post review of the decision by
humans. A reference to “human review/oversight” appeared in previous
public versions of the text of the Convention (January and December
2023). Furthermore, the Assembly always stressed the importance
of the principle of human responsibility for decisions in its AI-related
reports, including in the context of policing and criminal justice systems,
healthcare, work, neurotechnology and LAWS. The CAHAI also advocated
for including in the future AI treaty a mandatory right to human
review of decisions taken or informed by an AI system in the public
sector, except where competing legitimate overriding grounds exclude
this.Note The reference to “human review”
is all the more important because Article 15 only applies “where
an artificial intelligence system significantly impacts upon
the enjoyment of human rights”. This should not be understood however
as implying that where an artificial intelligence system simply
produces effects on individual rights and interests (but does not
reach the threshold of significantly impacting on human rights)
human review is no longer needed.
7. Add in Articles 16.1, 16.2
(a), (e) and 16.3:
“and the preservation of the environment”
after “the rule of law”
The Preamble of the DFC underlines that the Convention is
intended to encourage the consideration of “wider risks and impacts
related to these technologies including, but not limited to, human
health and the environment, and socio-economic aspects including
employment and labor”. The DER also mentions in its commentary to Article
16 that risk and impact assessments “can, where appropriate, take
due account of the need to preserve a healthy and sustainable environment.”
For the sake of coherence, this should also be reflected in the
text of Article 16, particularly if the proposal to include a separate
provision on health and environment under Chapter III (see above
no. 4) is not retained. Environmental sustainability should be one
of the criteria for risk and impact assessment of AI systems.
8. Replace Article 16.4 with
the following text:
“Each
Party shall take such legislative or other measures as may be required
to put in place mechanisms for a moratorium or ban or limitations
in respect of certain uses of artificial intelligence systems where such
uses are considered incompatible with the respect of human rights,
the functioning of democracy or the rule of law.”
I understand the disappointment expressed by some civil society
organisations and other stakeholders concerning the absence of ‘red
lines’ in Article 16 which prohibit certain AI applications on the
basis of unacceptable levels of risk.Note In addition, the DER does not provide
any objective criteria nor any specific examples of prohibited uses.
I hope that this gap will be filled in the future by other legally
binding or non-binding instruments prepared by the CAI.Note The Conference of the Parties could
also contribute to the interpretation of this provision by giving
examples of prohibited uses. The national oversight mechanisms referred
to in Article 26 could also possibly propose lists of prohibited
or high-risk uses of AI or be consulted on such lists.
In any event, the current version of Article 16.4 still does
not seem satisfactory to me as it leaves too much leeway to the
Parties to the Convention. It only creates an obligation to “assess
the need” for a moratorium or ban or other measures and leaves the
determination of what is “incompatible” to each Party. I would therefore suggest
using a stronger and clearer wording (similar to previous versions
of the draft): an obligation to ‘take legislative or other measures”
to put in place “a moratorium or ban or limitations” where such
uses “are considered incompatible” with human rights. This implies
that practices that are considered incompatible with human rights,
democracy and the rule of law by domestic authorities or the Conference
of the Parties, including on the basis of other applicable legal
instruments, should be prohibited, be the object of a moratorium,
or be appropriately limited.
9. Insert a new Article under
Chapter VI:
“Each
Party shall take appropriate measures to ensure protection of whistleblowers
in relation to the activities within the lifecycle of artificial
intelligence systems which could adversely impact human rights, democracy
and the rule of law.”
This provision was included in previous versions of the DFC.
Now there is only a reference to whistleblowers in the DER concerning
Article 8 (transparency and oversight). The Assembly has been a
pioneer in the protection of whistleblowersNote and
would strongly support a specific provision in the Convention. Whistleblowers
may indeed report misconduct and breaches of law by AI private and
public actors and therefore contribute to the implementation of
general principles such as transparency, accountability and responsibility,
reliability and safe innovation. Furthermore, the CAHAI in 2021
also recommended including a specific provision on whistleblowers
in the future legally binding transversal instrument.
10. Add the following sentence
at the end of Article 26.2:
“The
functions and powers of such mechanisms shall include investigative
powers, the power to act upon complaints, periodic reporting, promotion,
public awareness and consultation on the effective implementation
of this Convention.”
The Assembly considers that effective and proactive oversight
mechanisms should have the necessary powers for overseeing the compliance
with the Convention, with the needed technical and legal expertise
for following the new developments in AI and evaluating its risks.Note They should be able
to deal with the complaints referred to in Article 14.2 c. They
should also prepare periodic reports, which could be used for the
reporting obligation to the Conference of the Parties under Article
24. They should also be consulted on proposals for any legislative
or other measures aimed at the implementation of the Convention,
for instance any possible bans or moratoria issued under Article
16.4. In case of multiple mechanisms, these functions could be shared
by the different existing mechanisms. If this proposal is not retained
in the text of the Convention, the DER could at least be amended
to give more guidance on the type of powers envisaged for national
oversight mechanisms.
11. Insert a new Article under
Chapter VII, after Article 26, entitled “Parliamentary involvement”:
“1.
National parliaments shall be invited to participate in the follow-up
and review of the measures taken for the implementation of this
Convention.
2. The Parliamentary Assembly
of the Council of Europe shall be invited to regularly take stock
of the implementation of this Convention.”
Given the Framework Convention’s specific focus on the impacts
of AI on the functioning of democracy, a more visible role for national
parliaments in the follow-up of the implementation of the measures
taken to implement the Convention would be fully justified. The
Assembly has already referred to the need to “ensure that the use of
AI-based technologies by public authorities is subject to adequate
parliamentary oversight and public scrutiny”.Note PACE could also
be invited to regularly follow the implementation of the Convention
by member States, given the importance of parliamentary debate on
the impacts of AI on democracy, human rights and the rule of law
and the Assembly’s role and continuous engagement on this subject.
The proposed new provision under Chapter VII draws inspiration from
a provision in the Istanbul Convention on preventing and combating
violence against women and domestic violence (Article 70).
6 Conclusions
43. The Draft Framework Convention
on AI, human rights, democracy and the rule of law should be seen
as an important achievement of the CAI and the Council of Europe.
Once adopted by the Committee of Ministers, it will become the first
ever international treaty on artificial intelligence. It is based
on the Council of Europe standards on human rights, democracy and
the rule of law, which are commonly shared by the non-European States
that have participated in the negotiations and will hopefully sign
and ratify the Framework Convention. Its added value will be its
global reach, bringing together States from all over the world wishing
to address the global challenges posed by AI from the perspective
of these values and with a view to mitigating or limiting its risks.
This means that the negotiation and drafting process has had to
accommodate diverse legal and political traditions and systems,
with the result that the draft text often contains very general
and abstract provisions that will need to be clarified and developed
through its interpretation and follow-up mechanisms. I also expect that
the Framework Convention will be supplemented by other binding or
non-binding instruments concerning the use of AI in specific sectors
(for instance, work-related and social rights, cultural rights,
or specific sectors of the public administration such as law enforcement,
justice, healthcare, migration and electoral administration) and
the risk and impact management methodology.
44. The most problematic aspect of the DFC is its restricted scope
of application, with a system “à la carte” with regard to private
actors, and with blanket exemptions on national security and defence
purposes. The final compromise on the public/private sectors should
not be used by member States of the Council of Europe to modulate
or dilute the application of the Convention to private actors operating
within their jurisdictions. The Assembly should strongly call on
all member States to fully apply the Convention provisions to private
actors when submitting their declarations under Article 3.1, in
line with the mandate given by the Committee of Ministers, the position
of the Assembly and the Council of Europe Commissioner for Human
Rights, as well as the requirements under the ECHR. The Conference
of the Parties to the Framework Convention should also be invited
to fully use its powers and reporting system to assess how Parties
apply the Convention to private actors and to provide guidance in
its interpretation and effective application.
45. The critical comments and proposals made by different stakeholders
deserve to be taken seriously. I have based my suggested improvements
and amendments to the DFC on some of these proposals, or on my previous
drafting proposals submitted to the CAI. All these amendments are
intended to strengthen the draft taking due account of the overall
structure and underlying logic of the agreed text. Some relate to
the national security and defence exemptions (1, 2). Others aim
to add explicit references or develop the content of certain provisions
with regard to: free and fair elections (3), health and environment
(4, 7), judicial review (5), human review (6), prohibition of AI
uses (8), whistleblowers (9), national oversight mechanisms (10)
and parliamentary involvement (11). They are all reflected in the
draft Opinion preceding this explanatory memorandum.
