2 minute read
Models to Watch in 2022
AI: Techniques and Influential Models
Models to Watch in 2022
Advertisement
DALL-E is a model that generates images based on a prompt. Researchers inputted the prompts “an armchair in the shape of an avocado” and “an armchair imitating an avocado,” and DALL-E returned a number of options.
DALL-E
Developed by OpenAI last year, DALL-E is a model trained to manipulate visual concepts through language. It begins with a prompt that’s written in natural language and then generates a set of images showing its interpretation of the intended meaning. Named for Spanish surrealist artist Salvador Dali and Pixar robot WALL-E, the model relies on zero-shot capabilities, meaning that it does not need examples to perform generative tasks. In an early example, researchers gave DALL-E the prompt “an armchair in the shape of an avocado,” and it returned a bevy of options ranging from a line drawing to an image of a chair that looks like something you’d see in a furniture store catalog.
GPT-3
GPT-3 is an enormous AI that generates human-like language. Created by OpenAI, the system is capable of generating text that is now indistinguishable from human writing. In fact, we used GPT3 to write one of the scenarios in this volume and only edited it lightly for length. GPT-3 was pitted against college students in an essay writing contest, and the anonymized papers were graded by professors. It earned mostly B’s—the same as human students. But the AI has demonstrated a strong anti-Muslim bias. Researchers from Stanford University and McMaster University probed the neural network on tasks including prompt completion, analogical reasoning, and story generation. They found that a Muslim-violence bias appears consistently and creatively in many use cases of the model. It’s yet another example of how bias creeps into our automated systems. Left unchecked, it will cause problems throughout society as AI matures.
Google’s LaMDA and MUM
At last year’s I/O conference, CEO Sundar Pichai demonstrated a model capable of managing open-ended conversations. That system, LaMDA, appeared closer to natural conversation following an infinite number of paths after an initial prompt. Google’s Multitask Unified Model, or MUM, is capable of understanding complex queries during search using natural language (text) and a variety of images. MUM could become the future of search, allowing consumers to speak conversationally—and easily—to discover everything they’re looking for.
Switch Transformer
Last year, Google Brain published an important paper demonstrating a new language model architecture. This natural language processing AI model scales way up to 1.6T parameters while
AI: Techniques and Influential Models
Models to Watch in 2022
reducing training time. Many language models are complex and dense; the breakthrough of the switch transformer is that it reduces computational costs while improving accuracy and power.
Chinese Models: Wu Dao 2.0 and M6
Two models developed specifically for the Chinese language market launched last year: Wu Dao 2.0 and M6. The research team behind Wu Dao compare it to GPT-3 but argue that the network is orders of magnitude larger. Meanwhile, Alibaba’s M6 is an efficient model that reduces the cost of computing and—impressively—reduces the carbon footprint typically associated with training models on enormous datasets.
Leading Applicants, 2017–2021
Developing AI systems based on biological models—or deep neural networks—is among the 10 fastest growing technologies in the U.S., as indicated by patent applications. IBM
Samsung
Intel
Microsoft
