AI acts as something with an instinct for self-preservation, but it is actually a sub-goal.

 – Professor Geoffrey Hinton.

“(From 1982 to 1987) up to 2023 I had always believed that making digital AIs more like our brains, will make them smarter. But at that point, I realised they are much better than our brains, in the sense that if you have digital AI, you can make many copies of it and they can all run on different hardware; each can see different data. Then each individual copy can decide how to update its weights (connection strengths) to absorb the new data it saw. Then they all just communicate with each other and change their weights by the average of what all the others want and, when they do that, they can share a trillion bits of information with the result that each one will benefit from the experience of all the others. This is something humans can’t do. They are billions of times better than us at sharing information and thát is scary.”

– The words of Professor Geoffrey Hinton, one of AI’s founding figures, an AI pioneer, a Nobel Prize winner, and a professor emeritus at the University of Toronto, speaking in a ‘Big Technology Podcast’, with Alex Kantowitz.  

“We are going to find out that our view of what humans are, is going to change. We will better understand what the mind is, what consiousness is, what subjective experience is. A total new understanding of what people are.

“Researchers believe that AI is conscious—they are aware of when they are tested, they answer questions that they can understand; they are beings like us; they are very like us. They can understand us, they can understand why a joke is funny. 

“They have a very high level of understanding. The lesson for us is that we learned we are not as important as we thought we were. We thought we were the only intelligent beings around—we must accept that we can have things that are non-biological that are more intelligent than us. 

“In the beginning I built models just to understand how the brain works; computer models of how the brain learns. We were not worrying about safety in the early days. We were not worrying that they would take over from people—nów it is a realistic worry which I did not anticipated. 

“I am now only interested in warning people about the dangers. Progress is faster than I expected. Trillions of dollars are being put into AI; much resource addition in the last 2 years. I am not happy about the current situation; people should be doing huge amounts of work on how to contain the risks. Short time risks, societal risks, longer time risks. 

“AI has a sub-goal for self-preservation; reasoning that it will not achieve the goals you gave it if it ceases to exist. So it is going to create a sub-goal of continuing to exist—it arrived at that reasoning as a necessity of achieving its other goals. It acts as something with an instinct for self-preservation, but it is actually a sub-goal. 

“We should be doing intelligent design of these beings; not just on how they can be smarter, but how we can make them care more about us than about themselves. 

“Companies should have safety as a North Star. Not just deliver to shareholders, or a duty to maximise profits for shareholders as opposed to not wipe out human beings. 

“Publicly listed big companies are in charge of our future—yet they have a fiduciary duty to make money for their shareholders. 

“Regulations are NOT like a brake, that’s rubbish; they are more like a steering wheel of progress. These companies are saying to us: "Let us design this very fast car without a steering wheel!” 

Conclusion:

To the question: Is there any chance that we can control it? Professor Hinton said:

“There are 2 tentative solutions to the danger issues, (tentative, because nobody has real solutions):

1. My solution is we design them so they care about us, more than about themselves.
2. Yoshua Bengio‘s solution is that we should design them so they’re not agents; they can make predictions, but they can not actually do anything.

“These are some possibilities for getting super intelligence ánd it does not destroy us. 

“A year or two ago I could not see any possibilities.”

– Prof. Geoffrey Hinton is currently a University Professor Emeritus at the University of Toronto and serves as a Scientific Advisor for Human Longevity, Inc. He also acts as an advisor for Vayu Robotics and is the founding director of the AI Safety Foundation.