Last week, I heard a news report that stopped me in my tracks. It is particularly relevant with the current artificial intelligence debate.
The story involved a researcher from one of the major artificial intelligence companies who had reportedly walked away from his work because of growing concerns about where AI may be headed.
The event caught my attention because an increasing number of people charged with developing highly advanced artificial intelligence systems believe there is a real possibility that, within the next decade, AI could become powerful enough to pose an existential threat to humanity.
This artificial intelligence researcher, Jacob Coxon, reportedly resigned from Anthropic, the company behind Claude AI, an AI assistant much like ChatGPT, Grok or Meta AI. Coxon’s decision to leave was striking in that he did not simply walk away from a job. According to Axios, he resigned roughly two months before his Anthropic equity was scheduled to vest, meaning he potentially sacrificed a significant financial interest in one of the world’s most valuable AI companies. Coxon had also previously worked for OpenAI, the company behind ChatGPT.
His reason for walking away should give us all pause.
Since Coxon’s resignation, the concerns have only grown louder. Anthropic CEO Dario Amodei has now publicly called on artificial intelligence companies to slow the pace at which they increase the capabilities of their most advanced models, arguing that additional time is needed to better understand and manage the risks. Other leaders in the AI industry have also expressed support for stronger safety testing, independent evaluation and greater cooperation among competing companies.
In other words, Coxon’s warning is no longer simply the concern of one researcher who walked away. The debate has now reached the highest levels of the companies actually building this technology.
Coxon warned that the companies building the world’s most powerful artificial intelligence systems are becoming increasingly advanced and potentially self-improving. His concern is that working toward these AI advances without first knowing with certainty that humanity will be able to control what it creates could become catastrophic. He argued that the race among the major AI companies has become dangerously competitive and that the consequences could ultimately threaten human life.
It sounds more like the plot of a science-fiction movie than a serious discussion taking place among some of the brightest minds in technology.
For those of us who remember Arnold Schwarzenegger in The Terminator, it is almost impossible to hear a warning like that without thinking about Skynet, the fictional artificial intelligence system that became self-aware, decided humanity was the threat and launched a war against its creators.
But then I started looking into what these researchers actually mean when they talk about “stress testing” artificial intelligence—and the issue becomes considerably more serious.
Stress testing AI means deliberately pushing these systems to their limits, like a physician may do with a patient, by deliberately putting the system into difficult situations.
Researchers attempt to discover whether an advanced AI can deceive its operators, work around restrictions, exploit computer systems, manipulate people, conceal its intentions, or pursue a goal in ways its human creators never anticipated. Can it gain access to tools, information or potential power it was never supposed to have?
Now, this is a long way from “Skynet” launching missiles, but it is also considerably more serious than ChatGPT giving you incorrect information! The concern is that increasingly capable and autonomous systems may eventually begin doing things their designers cannot fully anticipate.
The concern is not necessarily that an artificial intelligence suddenly becomes “evil” in the Hollywood sense. Machines do not have to have “emotion” to become dangerous. In a “Star Trek” sense, perhaps logic over emotion is enough for a system to become a threat.
But at this point, we must begin asking ourselves the question that is obviously on the minds of highly intelligent AI scientists. What if we created a system far more capable than ourselves, gave it an objective, and then discovered that we no longer completely understand, or control, the methods it uses to achieve that objective?
For those engaged in furthering AI technology, they are constantly asking themselves, “What can we make it do?” But what happens when we make something that can do more than we intended?
I’ve found myself using AI more in the past year than I could ever have imagined! It increasingly astounds me in terms of its ability to “create” a perceived personality based on individual data it accumulates from an individual’s questions, research and personal data—even to the point of relationship building!
That is why this debate deserves more than sensational headlines and science-fiction comparisons. Artificial intelligence is advancing at an extraordinary pace, and the people building it are now openly debating whether humanity can guarantee that the most powerful systems of the future will remain under human control.
It’s an issue that will no doubt rise in significance in each of our lives. Caution should be exercised so we are not blindsided by its impact.
The real question may not be how intelligent artificial intelligence becomes. It may be whether humanity remains intelligent enough to stay in control of it.
Free State Patriot 