Mervinskiy 485

TheEuropeanDataProtectionSupervisor(EDPS)isanindependentinstitutionoftheEU,responsible underArticle52(2)ofRegulation2018/1725‘Withrespecttotheprocessingofpersonaldata…for ensuringthatthefundamentalrightsandfreedomsofnaturalpersons,andinparticulartheirrightto dataprotection,arerespectedbyUnioninstitutionsandbodies’,andunderArticle52(3)‘…foradvising Unioninstitutionsandbodiesanddatasubjectsonallmattersconcerningtheprocessingofpersonal data’.

WojciechRafałWiewiórowskiwasappointedasSupervisoron5December2019foratermoffiveyears.

Under Article 42(1) ofRegulation2018/1725,theCommissionshall‘followingtheadoptionof proposalsforalegislativeact,ofrecommendationsorofproposalstotheCouncilpursuanttoArticle 218TFEUorwhenpreparingdelegatedactsorimplementingacts,consulttheEDPSwherethereisan impactontheprotectionofindividuals’rightsandfreedomswithregardtotheprocessingofpersonal data’.

ThisOpinionrelatestotheRecommendationforaCouncilDecisionauthorisingtheopeningof negotiationsonbehalfoftheEuropeanUnionforaCouncilofEuropeconventiononartificial intelligence,humanrights,democracyandtheruleoflaw.ThisOpiniondoesnotprecludeanyfuture additionalcommentsorrecommendationsbytheEDPS,inparticulariffurtherissuesareidentifiedor newinformationbecomesavailable.Furthermore,thisOpinioniswithoutprejudicetoanyfuture actionthatmaybetakenbytheEDPSintheexerciseofhispowerspursuanttoRegulation(EU) 2018/1725.ThisOpinionislimitedtotheprovisionsofthedraftProposalthatarerelevantfromadata protectionperspective.

ExecutiveSummary

On18August2022,theEuropeanCommissionissuedaRecommendationforaCouncilDecision authorisingtheopeningofnegotiationsonbehalfoftheEuropeanUnionforaCouncilofEurope conventiononartificialintelligence(AI),humanrights,democracyandtheruleoflaw(‘the convention’),pursuanttoArticle218TFEU.

Havingregardtothe‘transborder’natureofartificialintelligence,theEDPSwelcomesthegeneral objective,pursuedbytheCouncilofEurope,ofelaboratingthefirstlegallybindinginternational instrumentonartificialintelligence,basedontheCouncilofEurope’sstandardsonhumanrights, democracyandtheruleoflaw.Accordingly,theEDPSsupportstheopeningofnegotiationson behalfoftheUnionfortheconvention,andwelcomestheUnion’sroleinpromotingtrustworthy AIthatisconsistentwiththeUnion’svalues.

TheEDPStakesnoteofthefactthatthesubjectmatteroftheconventionwouldberegulatedin theEUbytheproposedAIAct,andacknowledgestheCommission’saimtoensurethatthe conventioniscompatiblewiththeproposedAIAct,takingintoaccountfuturedevelopmentsinthe legislativeprocess.However,theEDPSconsidersthattheconventionrepresentsanimportant opportunitytocomplementtheproposedAIActbystrengtheningtheprotectionof fundamentalrightsofallpersonsaffectedbyAIsystemsandthereforeadvocatesthatthe conventionprovidesclearandstrongsafeguardsforthepersonsaffectedbytheuseofAIsystems.

Inthelightoftheabove,theEDPSmakesfourmainrecommendationsonthenegotiating directives:

thegeneralobjectivesforthenegotiationoftheconventionshouldgivemoreprominencetothe safeguardsandrightstobeprovidedtotheindividualsandgroupsofindividualssubjecttoAI systems,inlinewiththeprimaryfocusandobjectivesoftheCouncilofEurope; anexplicitreferencetocomplianceoftheconventionwiththeexistingEUlegalframeworkon dataprotectionshouldbeincludedinaspecificdirective;

inlinewiththeriskbasedapproach,theobjectiveofimposingaprohibitiononAIsystemsposing unacceptablerisksshouldbeintroduced;

theconventionshouldpromotetheadoptionofadataprotectionbydesignandbydefault approachateverystepofAIsystems’lifecycle.

Additionally,theOpinionoffersfurtherrecommendationsoninclusionintheconventionof minimumproceduralsafeguards,aswellasminimumrequirementsfortransparency, explainabilityandauditability,complianceandcontrolmechanisms,oncrossbordercooperation betweencompetentauthoritiestobedesignatedbythepartiestotheconventionforthe supervisionofthesafeguardsandrightstobeprovidedinaccordancewiththeconvention.

THEEUROPEANDATAPROTECTIONSUPERVISOR,

HavingregardtotheTreatyontheFunctioningoftheEuropeanUnion,

HavingregardtoRegulation(EU)No2018/1725oftheEuropeanParliamentandoftheCouncilof 23October2018ontheprotectionofindividualswithregardtotheprocessingofpersonaldataby Unioninstitutions,bodies,officesandagenciesandonthefreemovementofsuchdata(‘EUDPR’)1 , andinparticularArticle42(1)thereof,

HASADOPTEDTHEFOLLOWINGOPINION:

1.Introduction

1.On18August2022,theEuropeanCommissionissuedaRecommendationforaCouncil DecisionauthorisingtheopeningofnegotiationsonbehalfoftheEuropeanUnionfora CouncilofEuropeconventiononartificialintelligence(AI),humanrights,democracyand theruleoflaw2,pursuanttoArticle218TFEU(‘theRecommendation’).

2.TheobjectiveoftheRecommendationistoauthorisetheopeningofnegotiationsonbehalf oftheUnionforafutureCouncilofEuropeconventiononAI,humanrights,democracyand theruleoflaw(‘theconvention’),toadoptnegotiatingdirectivesandtonominatethe CommissionastheUnionnegotiator3

3.Intheexplanatorymemorandum4,theCommissionstressesthatnegotiationsforthe conventionrelatetomattersfallingundertheexclusiveUnioncompetence,alsoduetothe significantoverlapbetweenthe‘zerodraft’oftheconventioncirculatedbytheCommittee onArtificialIntelligence(CAI)oftheCouncilofEurope,ontheonehand,andofthe CommissionproposalforaregulationonAI(‘theproposedAIAct’)5,ontheotherhand,in termsoftheirscopeandcontent6 .

4.Theexplanatorymemorandum7totheRecommendationhighlightsthatthe‘zerodraft’ proposestoincludethefollowingprovisions:

OJL295,21.11.2018,p.39.

COM(2022)414final

COM(2022)414final,page3.

COM(2022)414final,page5.

ProposalforaRegulationlayingdownharmonisedrulesonartificialintelligence(ArtificialIntelligenceAct)andamendingcertain Unionlegislativeacts,COM/2021/206final.

Seealsorecital(5)oftheRecommendation.

COM(2022)414final,pages2and3.

subjectsthatwouldapplytoallAIsystems,irrespectiveoftheirlevelofrisk;

methodology(tobesetoutlaterinanannextotheconvention);

finalprovisions,includingapossibilityforEUMemberStatestoapplyEUlawin theirmutualrelationsformatterscoveredbytheconventionandapossibilityfor theUniontoaccedetotheconvention

5.TheRecommendation,underrecitals(6)and(7),highlightsthattheconclusionofthe conventionmayaffectexistingandforeseeableUnionrules.ToprotecttheintegrityofUnion lawandensureconsistencybetweentherulesofinternationallawandUnionlaw,the CommissionshouldbeauthorisedtonegotiatetheconventiononbehalfoftheUnion.

6.ThepresentOpinionoftheEDPSisissuedinresponsetoaconsultationbytheCommission of18August2022,pursuanttoArticle42(1)ofEUDPR.TheEDPSwelcomesthereferenceto thisconsultationinRecital8oftheRecommendation.

2.Generalremarks

7.TheEDPSwelcomesthegeneralobjective,pursuedbytheCouncilofEurope,ofelaborating a “legallybindinginstrumentofatransversalnatureonartificialintelligence,basedonthe CouncilofEurope’sstandardsonhumanrights,democracyandtheruleoflaw”8.Indeed,the ‘transborder’natureofartificialintelligencedevelopment,deploymentanduse“mayaffect allhumanity”9 .Theconventionwillbeopentoparticipationnotonlybythe46Member StatesoftheCouncilofEurope,butalsobynonMemberStatesoftheorganisation.

8.Accordingly,theEDPSsupportstheopeningofnegotiationsonbehalfoftheUnionfora futureconventiononAI,andwelcomestheUnion’sroleinpromotingtrustworthyAIthatis consistentwiththeUnion’svalues,throughthefirstlegallybindinginternational instrumentonAI,basedonsharedvaluesandprinciples10,notablyonhumandignity, democracyandtheruleoflaw11 .

9.However,theEDPSnotesthatthefirstdirective(directive(5))regardingthegeneral objectivesforthenegotiations,startswithareferencetothecompatibilityofthefuture convention‘withEUsinglemarketlaw’,andnottothefundamentalrights.Similarly,thefirst directiveonthesubstanceofthenegotiations(directive(11))laysdown,asaimofthe negotiations,that“theprovisionsoftheconventionarefullycompatiblewithEUsingle market”.

10.TheEDPSnotesthatthismarketcentricapproachisalignedwithoneofthemainobjectives oftheproposedAIAct12,thesinglemarketdimensionoftheregulationofAIsystems.Inthis respect,theEDPSrecallstherecommendationsputforwardintheEDPSEDPBJoint Opinion5/2021(‘thejointOpinion’)13.Atthesametime,theremitoftheCouncilofEurope ismuchbroaderIndeed,theCommitteeonArtificialIntelligence(CAI),setupbythe CommitteeofMinistersoftheCouncilofEuropefortheperiod20222024,hasbeen instructedto“establishaninternationalnegotiationprocessandconductworktoelaboratean appropriatelegalframeworkonthedevelopment,designandapplicationofartificial intelligence, basedonCouncilofEurope’sstandardsofhumanrights,democracyand theruleoflaw, andconducivetoinnovation.”[emphasisadded]14 .

11.Againstthisbackground,theEDPSconsidersthattheconventionrepresentsanimportant opportunitytocomplementtheproposedAIActbystrengtheningtheprotectionof fundamentalrightsofallpersonsaffectedbyAIsystems.Accordingly,andinlinewiththe JointOpinionontheAIAct,theEDPSconsidersthatsafeguardingtherightsof individualsandgroupsofindividualssubjecttotheuseofAIsystemsshouldbe givengreaterprominenceamongthegeneralobjectivesforthenegotiationofthe convention.

12.TheEDPSunderlinesthatAIsystemscanbeusedinavarietyofcontextinwhichEUand nationallawprovidespecificsubstantiveandproceduralguaranteesaimingatsafeguarding fundamentalrightsandfreedomsotherthanprivacyandpersonaldataprotection.These includethepresumptionofinnocenceandtherighttofairtrial15,ortheprincipleofequal treatmentinemploymentandoccupation16 .

13.TheEDPSrecallsthatthosefundamentalrightsandrelevantinstrumentsofEUlawmust alsobetakenintoaccountwhenensuringconsistencyoftheconventionwithEUlaw17 .

Article1(a)oftheproposedAIActstatesthatthisRegulationlaysdown“harmonisedrulesfortheplacingonthemarket,theputting intoserviceandtheuseofartificialintelligencesystems(‘AIsystems’)intheUnion”.

EDPBEDPSJointOpinion5/2021ontheproposalforaRegulationoftheEuropeanParliamentandoftheCouncillayingdown harmonisedrulesonartificialintelligence(ArtificialIntelligenceAct),issuedon18June2021.

SeeCAI’stermsofreference

COM(2022)414final,page4.

COM(2022)414final,page4.

Seeforinstancetheprohibition,recommendedintheJointOpinion,ofAIsystemsAIsystemstobeusedbylawenforcement authoritiesforpredictingtheoccurrenceorreoccurrenceofanactualorpotentialcriminaloffencebasedonprofilingofnatural personsasreferredtoinArticle3(4)ofDirective(EU)2016/680orassessingpersonalitytraitsandcharacteristicsorpastcriminal behaviourofnaturalpersonsorgroups.Thisprohibitionwouldbeunderpinnedbybothprivacyanddataprotectionand presumptionofinnocenceandrighttofairtrialconsiderations.

3.Relationshipwithotherinstruments

3.1InterplaywithEUlaw,includingtheCharter

14.Negotiatingdirective(5)laysdowntheobjectiveofcompatibilityoftheconventionwithEU law,statingthattheUnionshouldaimtoachievethattheconvention“iscompatiblewith EUsinglemarketlawandotherareasofEUlaw,includingitsgeneralprinciplesofEUlawand thefundamentalrightsandfreedomsasenshrinedintheEUCharterofFundamentalRightsand implementedthroughsecondaryEUlegislation”.

15.TheEDPSrecommendsdeletingtheword“including”tobetterreflecttheinterplaybetween generalprinciplesandfundamentalrights,ontheonehand,andsecondarylaw(EUsingle marketlawandotherareasoflaw),ontheotherhand.Thesamerecommendationapplies todirective(11).

16.Inthisregard,giventhatdata,includingpersonaldata,“areinmanycasesthekeypremise forautonomousdecisionswhichwillinevitablyaffectindividuals’livesatvariouslevels”18the EDPSunderlinestheimportanceforthefutureconventiontofullyrespecttheEU acquis intheareaofpersonaldataprotection

17.Inaddition,theEDPSwelcomesdirective(13),whichspecifiesthattheconventionshould “innoway”underminethelevelofprotectionoffundamentalrightsandfreedomsandthe guaranteesprovidedinUnionlaw,includingtheindependenceofauthoritiessupervising fundamentalrightsinsofarasrequiredunderEUlawandrecallstheneedtointerpretsuch freedomsandguaranteesbroadly,assetoutinSection2above.

3.2InterplayoftheconventionwiththeproposedAIAct

18.TheEDPStakesnoteoftheoverlapofthescopeoftheconventionwiththescopeofthe futureAIActandsupportstheCommission’saimtoensurethattheconventionisconsistent withtheproposedAIAct,takingintoaccountfuturedevelopmentsinthelegislativeprocess, aslaiddowninDirectives(6)and(12).Atthesametime,theEDPSrecallsthat“alotofwork remainstobedoneuntiltheProposal(oftheAIAct)cangivebirthtoawellfunctioninglegal framework,efficientlysupplementingtheGDPRinprotectingbasichumanrightswhilefostering innovation”19.Againstthisbackground,theEDPSwouldwelcometheinclusioninthe conventionofprovisionsaimingatreinforcingtherightsofpersonsimpactedbytheuse ofAIsystemsthatwouldcomplementthefutureAIAct

19.Inparticular,theEDPSsuggeststhattheCommissionshouldaimatincludinginthe conventionamethodologyforassessingtherisksposedbyAIsystemsonfundamental

rights,enshrinedintheEuropeanConventiononHumanRights(‘ECHR’),providingfor clear,concreteandobjectivecriteriaforsuchhumanrightsimpactassessment(HRIA).

3.3Interplayoftheconventionwithexistingdataprotectionlegalframework

20.ApartfromageneralreferencetoConvention108+,themandateismissingaclear directivereferringtotherelationoftheconventiontodataprotectionlaw.Inthis regard,intheJointOpiniontheEDPSandEDPBobservedthatdata,includingpersonal data,“areinmanycasesthekeypremiseforautonomousdecisionswhichwillinevitablyaffect individuals’livesatvariouslevels”20

21.Hence,theEDPSrecommendsinsertingadirectiveexplicitlyreferringtoconsistencyof theconventionwiththeexistinglegalframeworkondataprotection.Indeed, consistencywithdataprotectionprinciplesandlawsshouldbeaprerequisiteonwhichthe conventionshouldbuildupon.

22.TheEDPSconsidersinparticularthatdirective(16),statingthattheconventionshould [emphasisadded]“avoidoverlapsandprovidemeaningfuladdedvalue comparedto otherrelevantinternationalorregionalconvention,inparticularintheareaofdataprotection”, doesnotprovidesufficientclarityinthisrespect.

4.Scopeoftheconvention

23.TheEDPSwelcomestheproposedscopeoftheconvention,whichshouldcoverbothpublic andprivateprovidersandusersofAIsystems21,inaccordancewithits‘transversal nature’22.Inthisregard,theEDPSnotesthattheproposedAIActwouldalsohavea horizontalnature,sinceitwouldbeapplicabletoprovidersandusersofAIsystemsaspublic orprivateentities.

24.TheEDPSconsidersthat,unlessjustifiedorrequiredunderEUprimaryorsecondarylaw, thesameapproachshouldbetakenhavingregardtotheuseofAIsystems,regardlessof whetherprovidersandusersofAIsystemsarepublicorprivateentities.Thiswouldallowa morecoherentimplementationoftheriskbasedapproach23

JointOpinion,paragraph4.

COM(2022)414final,page2.Seealsonegotiatingdirective(14).

COM(2022)414final,page2.

Regrettably,thereareregulatorydivergenceswithintheproposedAIActregardingtheobligationsandlimitationsuponpublic andprivatesectoractorsinrelationtocertainAIsystems(notably,manipulativeAI,socialscoringandbiometricAIsystems).In theirJointOpinion,theEDPSandtheEDPBhaverecommendedinparticularaban,[emphasisadded]“forbothpublicauthorities andprivateentities,onAIsystemscategorizingindividualsfrombiometricsintoclustersaccordingtoethnicity,gender,aswell aspoliticalorsexualorientation,orothergroundsfordiscriminationprohibitedunderArticle21oftheCharter”,JointOpinion, paragraph33.

25.Accordingtotheexplanatorymemorandum24,theconventionshouldinclude“additional measuresforthepublicsector”.TheEDPSwelcomesthisapproach,insofarasthesemeasures complementthesafeguardsforpersonsimpactedbytheuseofAI,totakeintoaccountthe role,tasks,responsibilitiesandrulestowhichbodiesentrustedwithmissionsofpublic interestaresubject.Inthisregard,theEDPSnotesthattheseadditionalmeasuresshould alsoapplytoprivateentitieswhenprovidingpublicoressentialservices.

26.Moreover,theEDPSrecallsthat“theuseofAIintheareaofpoliceandlawenforcement requiresareaspecific,precise,foreseeableandproportionaterulesthatneedtoconsiderthe interestsofthepersonsconcernedandtheeffectsonthefunctioningofademocraticsociety. ”25 Accordingly,theEDPSrecommendstakingintoconsiderationnotonlytheinterestsofthe lawenforcementandjudicialauthorities,asstatedindirective(20)oftheAnnex,butalso thespecificrisksassociatedwiththeuseofAIintheareaofcriminaljusticeandlaw enforcement.Tothisend,theEDPSrecommendsaddingadirectiverecallingthenecessity tostriketherightbalancebetweenthepublicinterestandtheinterestsofthepersons subjecttoAIsystems,andtoensurefullcompliancewiththefundamentalrightstoprivacy andtotheprotectionofpersonaldata,aswellaswiththeotherfundamentalrights,notably therighttopresumptionofinnocenceandtoafairtrial.Theserightsareofteninextricably linkedwiththefundamentalrightstoprivacyandtotheprotectionofpersonaldata26

27.TheEDPSalsowelcomestheinclusionintheconvention,amongthedefinitionstobe provided,ofthenotionof“AIsubject”and,inconnectionwiththisdefinition,he inclusionofproceduralsafeguardsandrightsfor“AIsubjects”(namely,persons affectedbytheuseofAIsystems,e.g.workersaffectedbytheuseofAIworkmanagement systems;naturalpersonsapplyingforaloanaffectedbytheuseofAIcreditworthiness systems;migrantsandasylumseekersaffectedbytheuseofAIforborderandmigration control,etc.).

28.IntheJointOpinion,theEDPSandEDPBdeploredtheabsenceofanyreferenceinthe proposedAIActtotheindividualaffectedbyAIsystems,andconsideredsuchanabsence asa‘blindspot’intheproposedAIAct.TheEDPSalsonotesthatinthedraftreportonthe AIAct27,thecorapporteursoftheEuropeanParliament(LIBECommittee)haveproposed theinsertionofnewarticlesontherighttolodgeacomplaintagainsttheprovidersorusers ofAIsystemsandtherighttoaneffectivejudicialremedyagainstanationalsupervisory authorityforthepersonsaffectedbytheuseofAIsystems,bothasindividualsandasgroup ofpersonsconcerned.

COM(2022)414final,page2

JointOpinion,paragraph27.

Onfundamentalrights“inextricablylinked”tothefundamentalrightstoprivacyandtotheprotectionofpersonaldata,seeEDPS Guidelinesonassessingtheproportionalityofmeasuresthatlimitthefundamentalrightstoprivacyandtotheprotectionof personaldata,issuedon19December2019,atpages21,24.

draftreportontheproposalforaregulationoftheEuropeanParliamentandoftheCouncilonharmonisedrulesonArtificial Intelligence(ArtificialIntelligenceAct)andamendingcertainUnionLegislativeActs,issuedon20April2022

AIsystems.Suchrightswouldcomplementbutbewithoutprejudicetotherights

5.Risk-basedapproachandAIsystemsposingunacceptable risks

31.TheEDPSwelcomesthereferenceindirective(14)totheriskbasedapproach,according towhichtheconventionshouldlaydownrulesthatareproportionate,effectiveandclear acrosstheAIvaluechain.SuchanapproachistheoneunderpinningtheproposedAIAct, too

32.IntheJointOpinion,theEDPSandEDPBalsoappreciatedthattheproposal“wouldapply toanyAIsystems,includingthosewhichdonotinvolvetheprocessingofpersonaldatabutcan stillhaveanimpactoninterestsorfundamentalrightsandfreedoms

33.AlsohavingregardtothispossiblescenariorelatedtotheuseofAIsystems,theEDPS recommendsthatdirective(14)specifythatsocietal/grouprisksposedbyAIsystems(the risksforgroupsofindividualsorthesocietyasawhole,e.g.collectiveeffectswitha particularrelevance,likegroupdiscriminationorexpressionofpoliticalopinionsinpublic spaces)mustalsobeassessedandmitigated

34.Moreover,theEDPSnotesthat,althoughtheexplanatorymemorandumrefersto‘AIsystems posing‘unacceptable’risks’32,thiskeyissueisnotreflectedinthedirectives.Therefore,the EDPSstronglyrecommendsincludinginthenegotiatingdirectivesthatcertainAI systems,posingunacceptablerisks,shouldbeprohibited,aswellastoprovidean indicationofsuchAIsystems.

recallsthatthefollowingAIsystemsshouldalsobeprohibited:

,bypublicauthoritiesorontheirbehalf,aswellasbyprivatecompanies33;

forwhichtheAIsystemisintendedtobeused.

Liguedesdroitshumains

andJudgmentoftheCourtofJustice

biometricidentificationofindividualsinpubliclyaccessiblespaces34;more specifically,thenegotiatingdirectiveshouldincludethat“theconventionprohibitsanyuse ofAIforautomatedrecognitionofhumanfeaturesinpubliclyaccessiblespacessuchas offacesbutalsoofgait,fingerprints,DNA,voice,keystrokesandotherbiometricor behavioralsignalsinanycontext”;35

AIsystemscategorizingindividualsfrombiometrics(forinstance,fromfaceorvoice recognition)intoclustersaccordingtoethnicity,gender,aswellaspoliticalorsexual orientation,orothergroundsfordiscriminationprohibitedunderArticle21ofthe Charter(biometriccategorizationsystems)36;

AIsystemswhosescientificvalidityisnotprovenorwhichareindirectconflict withessentialvaluesoftheEU(e.g.,thepolygraph)37; AIsystemsintendedtobeusedbylawenforcementauthorities38formakingindividual riskassessmentsofnaturalpersonsinordertoassesstheriskofanaturalpersonfor offendingofreoffendingcriminaloffences39,orforpredictingtheoccurrenceor reoccurrenceofanactualorpotentialcriminaloffencebasedonprofilingofanaturalperson oronassessingpersonalitytraitsandcharacteristicsorpastcriminalbehavior40;

AIsystemsinferring‘emotions’ofnaturalpersons(socalledemotioncategorization systems),exceptforwellspecifiedusecases,namelyforhealthorresearchpurposes(e.g., apatientwhereemotionrecognitionisimportant)withappropriatesafeguardsinplaceand subjecttoalldataprotectionconditionsandlimitsincludingpurposelimitation

.

36.AccordingtotheEDPSandtheEDPB,thesetypesofpracticewouldnotmeetthenecessity andproportionalityrequirements,ormayevenaffecttheessenceoftherightto humandignity.Thus,theycouldbeconsideredunacceptableinterferenceswith

JointOpinion,paragraph30:“TheuseofAIsystemsmightpresentseriousproportionalityproblems,sinceitmightinvolvethe processingofdataofanindiscriminateanddisproportionatenumberofdatasubjectsfortheidentificationofonlyafewindividuals(e.g., passengersinairportsandtrainstations).Thefrictionlessnatureofremotebiometricidentificationsystemsalsopresentstransparency problemsandissuesrelatedtothelegalbasisfortheprocessingundertheEUlaw(theLED,theGDPR,theEUDPRandotherapplicable law).Theproblemregardingthewaytoproperlyinformindividualsaboutthisprocessingisstillunsolvedaswellastheeffectiveand timelyexerciseoftherightsofindividuals.Thesameappliestoitsirreversible,severeeffectonthepopulations’(reasonable)expectation ofbeinganonymousinpublicspaces,resultinginadirectnegativeeffectontheexerciseoffreedomofexpression,ofassembly,of associationaswellasfreedomofmovement

SeeJointOpinion,paragraph32.

SeeJointOpinion,paragraph33.

SeeJointOpinion,paragraph33.

SeeJointOpinion,paragraph34

SeeAnnexIIItotheproposedAIAct,atpoint6.(a):“AIsystemsintendedtobeusedbylawenforcementauthoritiesformaking individualriskassessmentsofnaturalpersonsinordertoassesstheriskofanaturalpersonforoffendingorreoffendingortherisk forpotentialvictimsofcriminaloffences”

SeeAnnexIIItotheproposedAIAct,atpoint6.(e):“AIsystemsintendedtobeusedbylawenforcementauthoritiesforpredicting theoccurrenceorreoccurrenceofanactualorpotentialcriminaloffencebasedonprofilingofnaturalpersonsasreferredtoin Article3(4)ofDirective(EU)2016/680orassessingpersonalitytraitsandcharacteristicsorpastcriminalbehaviourofnatural personsorgroups

JointOpinion,paragraph8. Inthissense,seealsoDraftreportontheproposalforaregulationoftheEuropeanParliamentandoftheCouncilonharmonised rulesonArtificialIntelligence(ArtificialIntelligenceAct)andamendingcertainUnionLegislativeActs,issuedon20April2022.

JointOpinion,paragraph35.

fundamentalrightsbytheCJEUandtheEuropeanCourtofHumanRightsThe UnionshouldthereforeaimtoachievethattheseAIsystemsareprohibited

6.DesignanddevelopmentofAIsystems

37.TheEDPS,inlinewiththerecommendationmadeintheJointOpinion43,recommends includinganegotiatingdirectiveaccordingtowhichtheconventionshouldpromotethe adoptionofadataprotectionbydesignandbydefaultapproachateverystepofAI systems’lifecycle,allowingtheeffectiveimplementationofdataprotectionprinciplesby meansofstateofthearttechnologies.

38.TheEDPSnotesthatdirective(17)concernsdifferentaspectsrelatedtotheimplementation oftherulesapplicabletothedesign,developmentandapplicationofAIsystems.TheEDPS isawarethatthenegotiatingdirectivesbytheirnaturecannotbetooprescriptive.However, theEDPSconsidersthatdirective(17)shouldbemorespecific.

39.Directive(17)referstoappropriate ex ante and ex post complianceandcontrol mechanisms.Inthisregard,theJointOpinionwelcomedthatAIsystemsposingahigh riskmustbesubjecttoapriorconformityassessmentbeforetheycanbeplacedonthe marketorotherwiseputintooperationintheEU.However,theEDPSandEDPBalso advocatedadaptingtheconformityassessmenttotheeffectthatanexantethirdparty conformityassessmentmustbecarriedoutforhighriskAI44.Similarly,theEDPS recommendsaclearinclusioninthedirectivesofthisrequirement(namely,thirdparty assessment,asopposedtoselfassessmentbytheprovideroftheAIsystem),takinginto accountthehighrisksforthepersonsaffectedbytheuseofAIsystems(the‘AIsubjects’).

42OntheAIsystemsthatshouldbeprohibited,seealsoEDPBStatementontheDigitalServicesandDataStrategy,adoptedon18 November2021,specifying,atpage2:“TheproposalfortheAIRwouldallowfortheuseofAIsystemscategorizingindividualsfrom biometrics(suchasfacialrecognition)accordingtoethnicity,gender,aswellaspoliticalorsexualorientation,orotherprohibitedgrounds ofdiscrimination,orAIsystemswhosescientificvalidityisnotprovenorwhichareindirectconflictwithessentialvaluesoftheEU.The EDPBconsidersthatsuchsystemsshouldbeprohibitedintheEUandcallsonthecolegislatorstoincludesuchabanintheAIR. Furthermore,theEDPBconsidersthattheuseofAItoinferemotionsofanaturalpersonishighlyundesirableandshouldbeprohibited, exceptforcertainwellspecifiedusecases,namelyforhealthorresearchpurposes,subjecttoappropriatesafeguards,conditionsand limits.Inthesamevein,giventhesignificantadverseeffectforindividuals’fundamentalrightsandfreedoms,theEDPBreiteratesthat theAIRshouldincludeabanonanyuseofAIforanautomatedrecognitionofhumanfeaturesinpubliclyaccessiblespacessuchasof facesbutalsoofgait,fingerprints,DNA,voice,keystrokesandotherbiometricorbehavioralsignalsinanycontext.TheproposedAIR currentlyallowsfortheuseofrealtimeremotebiometricidentificationsystemsinpubliclyaccessiblespacesforthepurposeoflaw enforcementincertaincases.TheEDPBwelcomestherecentlyadoptedEPResolutionwherethesignificantrisksarehighlighted”

JointOpinion,paragraph8.

SeealsoEDPSOpinion11/2021ontheProposalforaDirectiveonconsumercredits,issuedon26August2021,atparagraph55:“[] theEDPSrecallstheneedforintegrationoftherequirementsunderdataprotectionlaw(forinstance,dataminimisation,privacyby designandbydefault)intherequirementsundertheArtificialIntelligenceAct,inparticularinthecontextofthecertificationoftheAI creditworthinesssystem.Theintegrationofthisrequirementwouldbecruciallybeneficialtoindividuals’rights,bothasdatasubjectand consumer.

JointOpinion,paragraph37.

SeealsoEDPSOpinion11/2021ontheProposalforaDirectiveonconsumercredits,issuedon26August2021,atparagraph54:

TheEDPSalsorecommendsprovidingforexanteverificationofthecreditworthinessAIsystem,includingverificationofcompliance withtheProposal’srequirements,withtheinvolvementofthecompetentauthorityhavingspecificexpertiseonconsumerloans establishedpursuanttoArticle41oftheProposal.”

newconformityassessmentprocedurewheneverasignificantchangeoccurs45 .

41.Directive(17)alsoreferstocertificationmechanisms.Theirroleshouldberecognisedby theconvention.However,theobjectandlegaleffectsofthesecertificationsshouldbebetter specified.Inparticular,inordertoensurethatthecertificationisimplementedinaway thatiscompatiblewiththefundamentalrightsandfreedomsasenshrinedintheECHR andtheCharterasimplementedthroughsecondaryEUlegislation46,suchcertifications shouldbeconsistentwiththerequirementsundertheapplicableEUandMemberStates laws.47

42.Concerningtheroleofstandardsreferredtoindirective(17),theEDPSrecommends specifyingthattechnicalstandards,ontheonehand,canhaveapositiveimpacton harmonizationofproductsandservices;ontheotherhand,theirroleistoprovide technicalspecificationsofrules(clearandlegallybindingobligationsforthedesign, developmentandapplicationofAIsystems)alreadyestablishedbylaw48 .

43.Technicalstandardsshouldindeedbeusedforthespecificationofrequirements(for instancesafetyandqualityrequirements,withregardtoreliability,robustness, performanceandfunctionalsafety)establishedbythecorrespondingUnionlegislation.The EDPSconsidersthatthemandatefornegotiationshouldacknowledgetheroleaswellas theconditionsandlimitsoftechnicalstandardisationofAIsystems,referringtothe possibleadoptionoftechnicalstandardstoallowharmonisedimplementationofthe requirementsalreadysetoutatlegislativelevel.

44.Thisremarkisparticularlyimportantinlightofcomplexsystems,suchasAIsystems, whosedesign,developmentandapplicationconcerndifferentservices,relatedtoareas whicharealreadysubjecttospecificEUsecondarylegislation

Includingsignificantchangesofthethreatsscenario,havingregardtoexternalrisks,seeJointOpinion,paragraph40.

SeeDirective(5)oftheRecommendation.

JointOpinion,paragraph76.

(emphasisadded):“[..]Inlightoftheabove,theEDPSrecallstherecommendationsmadeintheJointEDPBEDPSOpinion toinclude data protection requirements, as well requirements stemming from sectoral legislation,inthiscaseconsumercredit, applicableUnionlegislation under the requirements for declaration of conformity oftheAIsystem.Intheabsenceofthis inclusion,theloanapplicant’sconsumeranddataprotectionrightsmightinpracticebejeopardisedbythe(highrisk)creditworthiness AIsystem

SeeRegulation(EU)No1025/2012oftheEuropeanParliamentandoftheCouncilof25October2012onEuropeanstandardisation, amendingCouncilDirectives89/686/EECand93/15/EECandDirectives94/9/EC,94/25/EC,95/16/EC,97/23/EC,98/34/EC, 2004/22/EC,2007/23/EC,2009/23/ECand2009/105/ECoftheEuropeanParliamentandoftheCouncilandrepealingCouncil Decision87/95/EECandDecisionNo1673/2006/ECoftheEuropeanParliamentandoftheCouncil,OJL316,14.11.2012,p.12, Article10(6),specifyingthattherequirementstobecoveredbystandardsaresetoutinthecorrespondingUnionharmonisation legislation;Article15(1)(b),referringtoconformityofthestandardstothecorrespondingUnionlegislationandpolicies.

49SeeCOM(2022)414final,pages3and4:,referringtosecondaryEUlegislationapplicabletoAIsystemsdependingontheservice forwhichtheAIsystemisintendedtobeused.

7.SupervisionofAIsystems

45.TheEDPSwelcomesthatdirective(17)and,morespecifically,directive(21),referto effectivesupervisionbycompetentauthoritiesDirective(21)specifiesthatthe conventionshouldprovideforcooperationmechanismsthatalloweffective enforcement.

46.Indeed,duetotheheterogeneityofareastowhichtheAIsystemsrefer(rangingfrom workandemploymenttofinancialservices,educationandhealthcare,administrationof justice,fraudprevention,etc.),thereisaneedforstructuredandinstitutionalised cooperationbetweendifferentcompetentauthorities(inparticular,betweenthedata protectionauthoritiesandthecompetentsectoralauthorities).

47.Moreover,theEDPSrecommendsincludingadirectiveaccordingtowhichtheconvention shouldprovidethatcompetentsupervisoryauthoritiesmustbegrantedadequate investigatoryandenforcementpowersTheseauthoritiesshouldbeempoweredin particulartohaveaccesstoanyrelevantdocuments,informationanddatanecessaryto openandconductinvestigationsandtomonitorcompliance,aswellastorequireaccess to,andexplanationrelatingto,databases,algorithmsandsourcecodes.

48.AsdiscussedinSection2above,thedevelopment,deploymentanduseofAIsystemsoften hasacross-bordernature.Thus,theEDPSrecommendsaddinganegotiatingdirective aimingatensuringthattheconventionfacilitatesandencouragescrossborder cooperationbetweencompetentauthorities.

8.Conclusions

49.Inlightoftheabove,theEDPSmakesthefollowingrecommendations:

(1)togivemoreprominencetotheobjectiveof“ensuringahighlevelofprotectionofhumanrights andpreservationofEuropeanvalues”,inlinewiththenatureandmandateoftheCouncilof Europe.

(2)todeletetheword“including”after“theEUsinglemarketlawandotherareasoflaw”in directives(5)and(11),tobetterreflectinterplaybetweengeneralprinciplesandfundamental rights,ontheonehand,andsecondarylaw(EUsinglemarketlawandotherareasoflaw),on theotherhand.

(3)toaddaspecificdirectiverecallingthenecessitytostriketherightbalancebetweenthepublic interestandtheinterestsofthepersonssubjecttoAIsystems,toensurefullcompliancewith therightstoprivacyandtotheprotectionofpersonaldata,aswellaswithotherfundamental rightsatstake,notablytherighttopresumptionofinnocenceandtoafairtrial,therightto goodadministrationandtheprincipleofnondiscrimination

(4)tospecifyinadirectivethattheconventionshouldprovidecertainminimumprocedural safeguardsandrightsforthepersonsaffectedbytheuseoftheAIsystems.

(5)tospecifyinadirectivethattheconventionshouldprovideforminimumrequirementson transparency,explainabilityandauditabilityofAIsystems

(6)toincludeindirective(14)thespecificationthatsocietal/grouprisksposedbyAIsystemsmust alsobeassessedandmitigated.

(7)tospecifyinthenegotiatingdirectivesthatcertainAIsystems,posingunacceptablerisks,should beprohibited,aswellastoprovideanindicativelistofsuchAIsystems.

(8)toincludeanegotiatingdirectiveaccordingtowhichtheconventionshouldpromotethe adoptionofadataprotectionbydesignandbydefaultapproachateverystepofAIsystems’ lifecycle.

(9)tospecifythecontentofdirective(17)asfollows:

anexantethirdpartyconformityassessmentmustbecarriedoutforhighriskAI;

thehighriskAIsystemsshouldbesubjecttoanewconformityassessmentprocedure wheneverasignificantchangeoccurs;

specifytheobjectandthelegaleffectofcertifications;

specifythattechnicalstandards,ontheonehand,canhaveapositiveimpacton harmonizationofproductsandservices;ontheotherhand,theirroleistoprovidetechnical specificationsofrulesalreadyestablishedbylaw.

(10)toincludeadirectiveaccordingtowhichtheconventionshouldprovidethatcompetent supervisoryauthoritiesmustbegrantedadequateinvestigatoryandenforcementpowers.

(11)toaddanegotiatingdirectiveaimingatensuringthattheconventionfacilitatesandencourages crossbordercooperationbetweencompetentauthorities.