The idea that artificial intelligence could become a threat to humanity has been discussed for decades. Recent developments in this field, such as ChatGPT, have only increased the fears. But how realistic is the threat and can we control super-intelligent AI?
Scientists have done the math and concluded that we almost certainly cannot control high-level computer superintelligence. The problem is that controlling such an AI would require creating a simulation that we can analyze. But if we are unable to understand that simulation, it is impossible to create one.
Rules such as “do no harm to humans” cannot be established if we do not understand the scenarios the AI can offer. Once the computer system is operating at a level above the capabilities of our programmers, we can no longer set limits.
This problem is related to the stopping problem formulated by Alan Turing in 1936. The problem is to know whether a computer program will come to a conclusion and an answer or just stall trying to find one. Although we can know this for some specific programs, it is logically impossible to find a way to know this for every potential program that could ever be written.
This brings us back to an AI that, in a super-intelligent state, could hold all possible computer programs in its memory at the same time. Any program written to stop AI harming people and destroying the world may or may not come to a conclusion (and stop) – we mathematically cannot be absolutely sure either way, which means that it cannot be contained.
Researchers say that an alternative to teaching AI some ethics and telling it not to destroy the world is to limit the superintelligence. For example, it could be cut off from parts of the Internet or certain networks. However, the 2021 study rejects this idea, suggesting it would limit AI’s capabilities.
If we are going to move forward with artificial intelligence, we may not even know when a superintelligence beyond our control will emerge. This means that we need to start asking serious questions about which direction we are headed.
