OpenAI has announced the formation of a new team called “Superalignment.” The team, comprised of top-tier machine learning researchers and engineers, wants to create a scalable learning approach to safely control artificial intelligence systems with superintelligence that transcends human capabilities.
According to OpenAI, superintelligent AI has the potential to be the most significant technology ever produced, capable of tackling a wide range of global challenges. However, the organization recognizes the enormous hazards connected with uncontrolled superintelligence, which may lead to the annihilation of mankind.
The lack of a navigational or control mechanism for superintelligent AI poses a significant challenge. Present AI models rely on reinforcement learning and human feedback, with humans supervising the AI’s actions. OpenAI acknowledges the limitations of this approach, stressing the need for a more robust solution.
To address this critical issue, the Superalignment team, led by renowned researchers Ilya Sutskever and Jan Leike, is dedicated to achieving scientific and technological breakthroughs that ensure secure control over AI systems.
The team’s audacious goal is to create the first automated alignment researchers with human-level capabilities. They intend to employ an enormous amount of computing power to iteratively align superintelligence and overcome the core technical challenges involved in just four years.
OpenAI is convinced that humans will never be able to reliably oversee AI systems that surpass our intelligence. Therefore, the company has established the Superalignment team, granting them access to 20% of OpenAI’s computing power, which has been exclusively reserved for this purpose.
OpenAI acknowledges that superintelligence does not necessarily have malicious intentions but can potentially harm. The organization remains optimistic about the future development of superintelligent AI, expecting the Singularity to arrive within the next decade. However, they also recognize the possibility of failing to navigate and control it effectively.
While research priorities may evolve over time, the Superalignment team remains committed to addressing the core challenges. OpenAI pledges to broadly share the fruits of this effort, emphasizing the importance of collaboration and knowledge dissemination. As OpenAI launches the Superalignment team, the organization seeks talented individuals to join their ranks as Research Engineers, Research Scientists, and Research Managers.
