3 minute read
4.5.6 Detection of solicitation of children (‘grooming’
that their real-world performance could be significantly lower than what is expected, leading to less accuracy and a higher percentage of ‘false positives’ .
64. Thirdly, performance indicators should be considered in the specific context of use of the relevant detection tools and provide an exhaustive insight into the behavior of the detection tools. When using artificial intelligence algorithms on images or text, it is well documented that bias and discrimination can occur due to the lack of representativeness of certain population groups in the data used to train the algorithm. These biases should be identified, measured and reduced to an acceptable level in order for the detection systems to be truly beneficial to society as a whole.
Advertisement
65. Although a study of the technologies used for detection has been carried out,
58 the EDPB and EDPS consider that further analysis is necessary in order to assess the reliability of the existing tools. This analysis should rely on exhaustive performance indicators and assess the impact of potential errors in real life conditions for all data subjects concerned by the Proposal.
66. As noted above, the EDPB and EDPS have serious doubts as to what extent the procedural safeguards provided for in Article 7(6) of the Proposal are sufficient to compensate these risks. Moreover, as indicated earlier, they note that the Proposal uses rather abstract and vague terms to describe the acceptable amount of risk (e.g. ‘appreciable extent’).
67. The EDPB and EDPS are concerned that these broad and vague notions will result in a lack of legal certainty and will also provoke strong divergences in the concrete implementation of the Proposal across the Union, depending on the interpretations that will be given to notions such as ‘likelihood’ and ‘appreciable extent’ by judicial or other independent administrative authorities in the Member States. This is worrying also in light of the fact that the provisions on detection orders will constitute ‘limitations’ to the principle of confidentiality laid down in Article 5 of the e-Privacy Directive. Thus, their clarity and foreseeability need to be improved in the proposed Regulation.
68. The EDPB and EDPS observe that the proposed measures concerning detection of solicitation of children (‘grooming’), entailing automated analysis of speech or text, are likely to constitute the most significant interference on the users’ rights to private and family life, including the confidentiality of communications, and to protection of personal data.
69. While the detection of known and even new CSAM can be limited in scope to the analysis of images and videos, grooming detection would extend by definition to all text-based (and possibly audio) communications that fall within the scope of a detection order. As a result, the intensity of the interference with the confidentiality of communications concerned is much greater.
70. The EDPB and EDPS consider that de facto general and indiscriminate automated analysis of textbased communications transmitted through interpersonal communications services with a view of identifying potential solicitation of children does not respect the requirements of necessity and proportionality. Even if the technology used is limited to the use of indicators, the EDPB and EDPS consider that the deployment of such general and indiscriminate analysis is excessive and may even affect the very core of the fundamental right to privacy enshrined in Article 7 of the Charter.
71. As already stated, the lack of substantive safeguards in the context of the measures for detection of solicitation of children cannot be compensated solely by procedural safeguards. Moreover, the
58 Impact Assessment Report, pp. 279 et seq.
