The article “Levels of AGI: Operationalizing Progress on the Path to AGI,” authored by Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, and Shane Legg, addresses the classification and assessment of Artificial General Intelligence (AGI) and its precursors. The paper proposes a framework defining levels of AGI capabilities, generality, and autonomy, discussing the rapid advancement of machine learning models and the evolution of AGI from philosophical debate to a matter of practical significance. The framework’s objective is to provide a common language for model comparison, risk assessment, and progress measurement towards AGI.
The article tackles questions such as:
- What does AGI mean in current research?
- How can progress towards AGI be measured and evaluated?
- How do current systems align with the concept of AGI?
In their approach, the authors outline six fundamental principles for an operational definition of AGI, focusing on capabilities, generality, cognitive and metacognitive tasks, and the development path towards AGI. They introduce the “Levels of AGI” ontology, which considers various performance levels (from “Emerging” to “Superhuman”) alongside the degree of generality (either “Narrow” or “General”).
The paper emphasizes that AGI is not necessarily synonymous with autonomy, discussing how progress in AGI levels unlocks but does not determine new levels of autonomy. This distinction allows for more nuanced insights into the risks associated with advanced AI systems and highlights the importance of research and development in human-AI interaction.
The six principles proposed in the article “Levels of AGI: Operationalizing Progress on the Path to AGI” for the definition and assessment of AGI are:
- Focusing on capabilities, not processes: This principle stresses that AGI assessment should be based on the abilities and outcomes an AGI system can achieve, rather than the specific mechanisms or processes it employs.
- Emphasizing generality and performance: AGI should be evaluated concerning its generality (i.e., the ability to handle a wide range of tasks) and performance (i.e., how well it performs these tasks).
- Focusing on cognitive and metacognitive tasks: Emphasis should be placed on tasks requiring cognitive and metacognitive abilities, such as problem-solving and learning, rather than physical tasks.
- Focusing on potential rather than deployment: AGI assessment should concentrate on what the system could potentially achieve, rather than just its current applications.
- Ecological validity for benchmarking tasks: Tasks and benchmarks for AGI assessment should be ecologically valid, reflecting realistic scenarios to test AGI’s applicability in real-world contexts.
- Focusing on the path to AGI rather than a single endpoint: Instead of concentrating on achieving a final AGI status, the focus should be on the various stages and progress on the path to AGI.
These principles provide a foundation for a more nuanced understanding and precise assessment of AGI by offering a broader perspective on the capabilities and potential of AGI systems.
Within the article, the authors also differentiate the various AI levels:
Level 0: No AI – This includes non-cognitive tools such as calculator software or compilers.
Level 1: Emerging – Simple rule-based systems or early AI models that perform better than untrained humans in specific areas.
Level 2: Competent – AI systems that are at least as competent as the average skilled adult, for example, toxicity detectors or digital assistants like Siri or Alexa.
Level 3: Expert – AI that is as performant as an expert in at least 90% of cases, such as grammar checkers or specific generative image models.
Level 4: Virtuoso – AI systems that outperform experts in most cases, such as Deep Blue in chess.
Level 5: Superhuman – AI that surpasses human performance in all aspects, like AlphaFold in protein structure determination.
Of interest in the article is also the assessment in the table on page 6. The table clarifies that certain AGI levels in the “General” category, such as Competent AGI, Expert AGI, and Virtuoso AGI, have not yet been achieved at the time of publication, and that the ultimate stage of AGI, Artificial Superintelligence (ASI), has also not yet been realized.
The text is available under the following source citation:
Morris, M. R., Sohl-Dickstein, J., Fiedel, N., Warkentin, T., Dafoe, A., Faust, A., Farabet, C., & Legg, S. (2023). Levels of AGI: Operationalizing Progress on the Path to AGI. [PDF]. Available at https://arxiv.org/pdf/2311.02462.pdf