Combatting Online Harms Through Innovation
A. Avoiding over-reliance AI detection tools for the harms discussed here are blunt instruments. 230 For several reasons, their use can result in false positives and false negatives. One can adjust variables to catch more or less of a given type of content, but trade-offs are inevitable. For example, blocking more content that might incite extremist violence (e.g., via detection of certain terms or imagery) can result in also blocking members of victimized communities from discussing how to address such violence. This fact explains in part why each specified harm needs individual consideration; the trade-offs we may be willing to accept may differ for each one. 231 But what the public is willing to accept may not matter if only those developing and deploying these tools get to decide what types and levels of failure are tolerable, whether and how to assess risks and impacts, and what information is disclosed. Built-in imprecision Many of the AI systems built to detect particular kinds of content are “trained” to work by researchers who have fed it a set of examples that they have classified in various ways. 232 These datasets and classifications allow the system to predict whether a new example fits a given classification. For example, researchers might use a database of animal images in which some are labeled as “cats” and others as “not cats.” Then the researchers may feed in new images and ask the system to decide which ones are “cats.” For the system to work well, the dataset must be sufficiently big, accurate, and representative, so that no types of cats are excluded and no other animals are misbranded as feline. But the AI doesn’t actually understand what a “cat” is. It’s just trying to do some math. So, if the cats in the dataset include only cats with pointy ears, the system may not identify ones whose ears fold down. And if the system is trained to identify “cats” only by pointy ears and whiskers, then rabbits and foxes may be shocked to learn that they
Despite marketing pitches that trumpet the use of AI, some of these tools may not be AI at all and may not even be all that automated, relying instead on something as simple as spreadsheets or on the insertion of an interface that masks underlying human labor. 231 See, e.g., United Nations, supra note 127 at 43; Nafia Chowdhury, Automated Content Moderation: A Primer, Stanford Cyber Policy Center (Mar. 19, 2022), https://cyber fsi.stanford.edu/news/automated-content-moderationprimer; Samidh Chakrabarti, Twitter Post (Oct. 3, 2021) (“This is where the rubber hits the road. What is the acceptable tradeoff between benign and harmful posts? To prevent X harmful posts from going viral, would you be willing to prevent Y benign posts from going viral? No easy answers.”), https://twitter.com/samidh/status/1444544160518733824. 232 This work is not all done by scientists. Some big technology companies use low-paid microworkers, sometimes refugees in other parts of the world, to help with the huge amount of data training needed for these systems to work. See Karen Hao and Andrea Paola Hernández, How the AI industry profits from catastrophe, MIT Tech. Rev. (Apr. 20, 2022), https://www.technologyreview.com/2022/04/20/1050392; Julian Posada, Family Units, Logic (Dec. 25, 2021), https://logicmag.io/beacons/family-units/; Phil Jones, Refugees help power machine learning advances at Microsoft, Facebook, and Amazon, Rest of World (Sep. 22, 2021), https://restofworld.org/2021/refugees-machinelearning-big-tech/. 230
FEDERAL TRADE COMMISSION • FTC.GOV
41
