The danger of advanced artificial intelligence controlling its own feedback

Wed, 9 Nov 2022 04:07:21 +1100

Andrew Pam <xanni [at] glasswings.com.au>

Andrew Pam
<https://theconversation.com/the-danger-of-advanced-artificial-intelligence-controlling-its-own-feedback-190445>

"How would an artificial intelligence (AI) decide what to do? One common
approach in AI research is called “reinforcement learning”.

Reinforcement learning gives the software a “reward” defined in some way, and
lets the software figure out how to maximise the reward. This approach has
produced some excellent results, such as building software agents that defeat
humans at games like chess and Go, or creating new designs for nuclear fusion
reactors.

However, we might want to hold off on making reinforcement learning agents too
flexible and effective.

As we argue in a new paper in AI Magazine, deploying a sufficiently advanced
reinforcement learning agent would likely be incompatible with the continued
survival of humanity."

Cheers,
       *** Xanni ***
--
mailto:xanni@xanadu.net               Andrew Pam
http://xanadu.com.au/                 Chief Scientist, Xanadu
https://glasswings.com.au/            Partner, Glass Wings
https://sericyb.com.au/               Manager, Serious Cybernetics

Comment via email

Home E-Mail Sponsors Index Search About Us