STATE OF AI REPORT 2021.
The most widely read and trusted analysis of key developments in AI.
Explore the full report
Transformers spread beyond language
The fourth report followed transformers into vision, biology, and deployed products. Protein-structure prediction advanced rapidly, AI improved an electricity-grid forecast, and language models entered developer tools. The same report questioned whether evaluation, academic resources, and long-term alignment research were keeping pace.
Published on October 12, 2021, this edition captures what we knew then. The findings and predictions below retain that historical perspective.
Read the original Air Street Press launch essay and website launch post, or explore the 2021 report by topic.
Let’s dive into the key stories.
Transformers become a general-purpose architecture
The 2021 report documented a broadening of transformers beyond language. Images could be treated as sequences of patches, while paired text and images supported reusable representations. Strong benchmark performance still left important weaknesses, including a tendency to reproduce falsehoods.
DreamerV2 learned a compact model of an Atari environment from pixels, then used that model to learn behavior. The report highlighted strong performance on a 55-task Atari benchmark using a single GPU. The important shift was where learning occurred: the agent could explore possible behavior inside a learned representation of the environment, reducing its dependence on expensive interactions with the game.
Chip supply exposes industrial dependencies
Compute capacity depended on more than the design of an AI accelerator. The 2021 report examined the equipment and manufacturing supply chains behind chips, highlighting ASML’s position in extreme-ultraviolet lithography and growing delays in semiconductor supply.
A company could have a use for more processors and still wait months for delivery. The report’s supply-chain discussion placed AI demand within a wider semiconductor shortage. Capability improvements therefore depended partly on the timing and concentration of physical manufacturing capacity, not only on research progress.
More countries train AI researchers, but resources remain uneven
The 2021 report showed a widening geography of AI education and hiring. Growth in talent did not guarantee equal access to the resources needed for research. University labs continued to face constraints in compute budgets and staffing.
A growing pool of researchers did not ensure equal access to experimental resources. The report’s examples of small academic compute budgets illustrated a practical constraint on what teams could attempt. Because the examples came from an informal poll, they were evidence of the problem rather than a representative funding census.
AI becomes part of operational infrastructure
The 2021 report included concrete deployments alongside research advances. A transformer-based forecast supported the UK electricity grid, and language models began appearing in developer workflows. The strongest examples measured an operational improvement or identified the specific work a model performed.
A Moorfields Eye Hospital team built a computer-vision system to detect and monitor dry age-related macular degeneration. One model predicted disease progression, while another measured features of the disease in optical coherence tomography scans. The report described development data from 200 patients and validation on 110 patients. That defined the evidence behind this particular clinical example.
Protein structure prediction takes a major step forward
Protein shape helps explain biological function, but measuring structures experimentally is difficult. The 2021 report described AlphaFold 2 as a major advance in predicting structures and followed how related ideas quickly spread to academic systems.
RoseTTAFold showed how related ideas could be developed by another research group after AlphaFold 2’s CASP14 result. Its joint treatment of sequence, pairwise relationships, and three-dimensional coordinates illustrated the technical direction. The significance was broader research access, not a claim that every biological problem had become tractable.
Governments begin regulating AI systems directly
The 2021 report described a shift toward rules aimed at AI systems themselves. The EU proposed a risk-based AI Act, while military deployment made questions of autonomy and responsibility more immediate. This was the policy landscape at publication, before later legislation and events.
The report’s account of the EU proposal grouped systems by risk and application. That made use context central to the discussion. Readers should treat the page as a record of the 2021 proposal, rather than use its categories or examples as a guide to legal obligations today.
Alignment research is small relative to capability development
The 2021 report asked whether enough research effort was going toward ensuring that increasingly capable systems behaved as intended. It separated long-term alignment from the broader field of near-term AI safety and documented a limited number of dedicated researchers at selected organizations.
The alignment count focused on a selected set of organizations and a particular long-term research concern. It could not describe the whole safety field. Even with that limitation, the report used the small scale of the effort to ask how resources were divided between improving capabilities and understanding their consequences.
The 2021 forecasts, in their original form
The forecasts followed transformers, scientific discovery, semiconductor consolidation, and the formation of new research companies. These are historical forecasts made at publication, not new predictions or retrospective claims of success.
The predictions ranged from more efficient image models to a semiconductor acquisition, ASML’s valuation, and the formation of a new research company. That breadth reflected a view that AI progress depended on organizational and industrial changes as well as benchmark gains.
Three of the forecasts published in 2021:
- Transformers replace recurrent networks to learn world models with which RL agents surpass human performance in large and rich game environments.
- ASML’s market cap reaches $500B.
- Anthropic publishes on the level of GPT, Dota, AlphaGo to establish itself as a third pole of AGI research.
The report is authored by Nathan Benaich and Ian Hogarth
For our writing throughout the year, subscribe to Air Street Press.
Cite the report
Benaich, N. and Hogarth, I. (2021). State of AI Report 2021. https://www.stateof.ai/2021