Superintelligent AI refers to machines that surpass human intelligence in reasoning, problem-solving, and creativity. While it promises breakthroughs in science, medicine, and technology, many scientists and experts warn about potential risks, including loss of control, ethical dilemmas, and societal disruption. Watch this video to understand what superintelligent AI is, why it matters, and why experts are concerned.

 Superintelligent AI: What Is Superintelligent AI and Why Scientists Are Worried About I

<p>Artificial intelligence leader OpenAI's co-founder Ilya Sutskever warned that the superintelligence of the technology needs to be controlled so as to prevent the extinction of the human race.<br />
<br />
"Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction," wrote Ilya Sutskever and head of alignment Jan Leike in a blog post, stating that they believe such advancements could arrive as soon as this decade.<br />
<br />
They said that new institutions for governance will be required for managing such risks and solving the superintelligence alignment's problem like ensuring AI systems, which are smarter than humans, continue to "follow human intent."</p>

<p><strong>ALSO READ |&nbsp;</strong><a href="https://www.wionews.com/science/this-ai-tool-can-help-doctors-treat-brain-tumours-quickly-and-accurately-study-finds-613368">This AI tool can help doctors treat brain tumours quickly and accurately, study finds</a></p>

<p>"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs,” the blog post read.<br />
<br />
They stated that in order to solve these issues, within a period of four years, they have been leading a new team and dedicating 20 per cent of the computing power, which has been secured till now, for this effort.<br />
<br />
"While this is an incredibly ambitious goal and we’re not guaranteed to succeed, we are optimistic that a focused, concerted effort can solve this problem," they stated.</p>

<h2><strong>Aligning superintelligent AI with human intent</strong></h2>

<p>The new team, apart from working towards improvising current OpenAI models like ChatGPT and mitigating risks, is also focused on the challenges of machine learning which involves aligning superintelligent AI systems with human intent.<br />
<br />
The team's goal is to devise a roughly human-level automated alignment researcher, with the use of vast amounts of computing to scale it and "iteratively align superintelligence."<br />
<br />
So as to do it, OpenAI will be developing a scalable training method, as well as validating the model achieved as part of its result and then a stress test of its alignment pipeline will be carried out.</p>

<p><strong>WATCH |&nbsp;Will AI run the world better than humans?</strong></p>

<div class="videoWrapper trancodeVideo"><iframe allowfullscreen="" autoplay="1" class="includeVideo" frameborder="0" height="350" src="https://www.wionews.com/videos/will-ai-run-the-world-better-than-humans-613263/embed" style="width:100%;max-width:560px;" width="560"></iframe></div>

<p><br />
“As we make progress on this, our AI systems can take over more and more of our alignment work and ultimately conceive, implement, study and develop better alignment techniques than we have now,” read a previous blog post written by Leike and colleagues John Schulman and Jeffrey Wu.<br />
<br />
“They will work together with humans to ensure that their own successors are more aligned with humans. . . . Human researchers will focus more and more of their effort on reviewing alignment research done by AI systems instead of generating this research by themselves,” they added.&nbsp;</p>

<main>
<section>
<p>You can&nbsp;<strong>now write for wionews.com</strong>&nbsp;and be a part of the community. Share your stories and opinions with us&nbsp;<strong><a href="https://www.wionews.com/content-contribution-registration-form">here</a>.</strong><br />
<br />
&nbsp;</p>

<p><strong>WATCH WION LIVE HERE</strong></p>

<p><iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="350" src="https://www.wionews.com/live-tv/embed?autoplay=1&amp;mute=1" width="100%"></iframe></p>
</section>
</main>


Artificial intelligence leader OpenAI's co-founder Ilya Sutskever warned that the superintelligence of the technology needs to be controlled so as to prevent the extinction of the human race.

Artificial intelligence leader OpenAI's co-founder Ilya Sutskever warned that the superintelligence of the technology needs to be controlled so as to prevent the extinction of the human race. "Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction," wrote Ilya Sutskever and head of alignment Jan Leike in a blog post, stating that they believe such advancements could arrive as soon as this decade. They said that new institutions for governance will be required for managing such risks and solving the superintelligence alignment's problem like ensuring AI systems, which are smarter than humans, continue to "follow human intent." ALSO READ | This AI tool can help doctors treat brain tumours quickly and accurately, study finds "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs,” the blog post read. They stated that in order to solve these issues, within a period of four years, they have been leading a new team and dedicating 20 per cent of the computing power, which has been secured till now, for this effort. "While this is an incredibly ambitious goal and we’re not guaranteed to succeed, we are optimistic that a focused, concerted effort can solve this problem," they stated. Aligning superintelligent AI with human intent The new team, apart from working towards improvising current OpenAI models like ChatGPT and mitigating risks, is also focused on the challenges of machine learning which involves aligning superintelligent AI systems with human intent. The team's goal is to devise a roughly human-level automated alignment researcher, with the use of vast amounts of computing to scale it and "iteratively align superintelligence." So as to do it, OpenAI will be developing a scalable training method, as well as validating the model achieved as part of its result and then a stress test of its alignment pipeline will be carried out. WATCH | Will AI run the world better than humans? “As we make progress on this, our AI systems can take over more and more of our alignment work and ultimately conceive, implement, study and develop better alignment techniques than we have now,” read a previous blog post written by Leike and colleagues John Schulman and Jeffrey Wu. “They will work together with humans to ensure that their own successors are more aligned with humans. . . . Human researchers will focus more and more of their effort on reviewing alignment research done by AI systems instead of generating this research by themselves,” they added. You can now write for wionews.com and be a part of the community. Share your stories and opinions with us here. WATCH WION LIVE HERE

OpenAI creating new team to control ‘superintelligent’ AI, prevent human extinction

superintelligent-ai

Superintelligent Ai, News on Superintelligent Ai, stories on Superintelligent Ai, articles on Superintelligent Ai, Superintelligent Ai stories at Wion, Superintelligent Ai news, Superintelligent Ai latest news

superintelligent ai

Superintelligent AI: What Is Superintelligent AI and Why Scientists Are Worried About I

OpenAI creating new team to control ‘superintelligent’ AI, prevent human extinction