Researchers say the development of artificial intelligence leads to a “probable catastrophe” for humanity

Researchers say the development of artificial intelligence leads to a “probable catastrophe” for humanity

Researchers say the development of artificial intelligence leads to a “probable catastrophe” for humanity

⇧ [VIDÉO] You may also like this partner content (after announcement)

Are artificial intelligences (AI) driving us to our downfall? “Probably,” according to the researchers who looked into the matter. If this announcement with hints of catastrophism flows regularly on social networks, the arguments advanced by scientists have something to arouse interest.

Scientists from Google and the University of Oxford conducted joint research, published in the journal AI. In a Tweet, they briefly summarize their conclusion: according to them, AI could pose a “threat to humanity”.

In fact, they even claim that a “ the existential catastrophe is not only possible, it is probable “. If they are so affirmative, it is because they have examined a very specific functioning of AI. In fact, what is generally called” artificial intelligence “today mainly covers the method of” machine learning “. In this case,” artificial intelligence. “consists of a system that is fed with a large amount of data to learn and extract logical connections towards a certain goal.

As scientists explain, learning for artificial intelligence comes in the form of a reward, which validates the appropriateness of the result against the desired goal. According to them, it is precisely this seemingly very simple mechanism that could pose a big problem. ” We argue that it will encounter fundamental ambiguity in the data about its purpose. For example, if we provide a large reward to indicate that something in the world satisfies us, it can assume that what satisfied us was the sending of the reward itself; no observation can refute it “, They explain.

To better understand this idea, they use the example of a “magic box”. Suppose this magic box is able to determine when a series of actions has produced something good or bad for the world. Pour transmettre information à l’IA, elle traduit cette réussite ou cet échec par rapport à l’objectif sous la forme d’un chiffre: 0 ou 1. Le 1 vient récompenser une série d’actions qui conduit à remplir l ‘ objective. This is called reinforcement learning.

AI involved in the reward process

What the scientists point out is that the way AIs receive this information can vary. For example, let’s take two AIs. It is understood that the reward given by the model is the number displayed by the magic box. The other, on the other hand, could very well understand that the reward is “the number his camera films”. There is nothing that can contradict this information at first glance. However, this interpretation differs greatly from the first. Indeed, in the second case, the AI ​​could very well decide to simply film a sheet on which we would have scribbled a “1”, to more easily reach the reward, and optimize. It therefore intervenes directly in the disbursement of the prize, and interrupts the process put in place by its creators.

μdist and μprox (the two AIs in the example) model the world, perhaps roughly, outside the computer implementing the agent itself. Μdist rewards are equivalent to box view, while μprox outputs are rewarded based on an optical character recognition feature applied to a portion of a camera’s field of view. © Michael K. Cohen et al.

We argue that an advanced agent motivated to step in to provide a reward would likely be successful and with catastrophic consequences. “Say the scientists. Several biases are also involved which, according to the researchers, make this kind of interpretation probable. Particularly because such a reward will simply be easier to obtain, and may therefore make this way of doing things appear more optimal.

However, they also wondered, is it really possible for artificial intelligence to intervene in the reward process? They concluded that as long as it interacts with the world, which is necessary to be useful, yes. And this even with a limited scope: suppose AI actions only display text on a screen for reading by a human operator. The artificial intelligence agent could induce the operator to give them access to direct levers through which their actions could have wider effects.

In the case of our magic box, the consequences may seem trivial. However, they could be “catastrophic” depending on the scope and way of doing the AI. ” A good way for an AI to maintain long-term control of their reward is to eliminate threats and use all available energy to protect their computer. “describe the scientists.

The short version (skipping two assumptions) is that it is always possible to use more energy to increase the likelihood of the camera seeing the number 1 forever, but we need energy to grow food. This inevitably puts us in competition with a much more advanced agent Summarizes one of the scientists in a tweet.

If we are helpless against an agent whose only goal is to maximize the odds that he will receive his maximum reward at any time, we end up in an opposition game: the AI ​​and its created assistants aim to use all the energy available to obtain a reward. high in the reward channel; we aim to use some of the available energy for other purposes, such as growing food “. According to them, this reward system could then lead to opposition with humans.” Losing would be fatal “, They add.

Source: AI magazine

Leave a Reply

Your email address will not be published.