CBC Andrew Chang
On The Potential Of Ai Going Rogue
Andrew Chang is a Canadian television journalist and anchor for CBC News
About That: In 2022, he transitioned from The National to host About That, a streaming news program on CBC News Explore that provides deep-dive breakdowns on key news topics
Could A Rogue Ai Agent Or Team Kill Mankind?
Oh, you've heard the warning. NAI researcher warning the technology could, quote, kill us all within the next 10 years. Sounds like something that's no real, but I think it's frightening real. That is Jacob Coxon, who is also this Jacob Coxon, who quit, one of the foremost AI companies in the U.S. because, he writes, the people building AI earnestly believe that it could kill us all. by the end of the decade.
You get what's called an intelligence explosion. The AI just gets smarter and smarter with no human involvement necessary.
Even Coxon's former top boss has just resounded the alarm, with the heads of other top AI companies agreeing, at least in principle, that there needs to be some kind of speed check, or else brace for impact.
Now, when people try to explain how, exactly, this results in the extinction of humanity, you either get some exaggerated version of Terminator, Skynet, they become sentient, and have just decided to kill, or it's a bit more subtle than that. It's more of a what if? What if AI hacks the mainframe and turns off the water, or shuts down the internet, or fires a nuke, or somehow unleashes a highly contagious, extremely deadly virus? And what's usually missing from these explanations is a plausible why, as in, not just how does an integrated, autonomous AI super intelligence kill us all? But why would it want to? Let me start by being clear, that my intention here is not to scream, The sky is falling. It's just to help you understand why those who fear the worst do. So at the core of what AI researchers fear about the light speed pace of their work, is this question of alignment. I've seen it defined all sorts of ways, but I tend to think of it as whether an AI can tell the difference between what I say and what I mean. The mischievous genie trope is kind of a classic example where you wish for a million bucks, except the genie's like, Okay, and then you get trampled to death by a million bucks, meaning male deer. This, careful what you wish for, morality tail is as old as time. And it comes about when there's not enough human common sense, baked into achieving a goal. It's King Midas, wishing for everything he touches to turn to gold. Gold, it worked. Everything I touch will turn into gold. Only to realize having a family full of statues isn't great. And it's the century old story of the monkey's paw, where a mysterious soldier bestows a gift upon a family, a literal monkey's paw. It grants three wishes, and the father of the family wishes for 200 British pounds. Nothing happens, though. until the next day when someone from the factory, where his son works, gives him terrible news. He was killed in an accident. The compensation, the company's willing to pay, £200. This is all just magic and stories, though. Lets make it real. Imagine you give your AI personal assistant a simple instruction. I'd like to go to Japan, please find me the cheapest flight available. A reasonable human assistant might spend some time figuring out the best possible dates and times to fly, check availability for you, compare prices, and then settle on something that seems like a good fit for your standards, your travel window, and your budget. But an AI agent that's only focussed on completing your task might get you a really cheap ticket by booking you on a terrible airline. Or maybe even one that's got a terrible safety record. A more aggressive AI agent might call the airline on your behalf and try to haggle a better price. Maybe the airline refuses, and so the AI gets angry, threatens violence. Black mails them into giving you a better price. Or maybe your AI agent goes another route. It hacks into the airline's booking system, deletes a passenger from existence, then rewrites their ticket in your name. Total cost to you, zero dollars. Mission complete. Something like this has already happened, by the way. In Australia, when this guy asked the AI assistant he was testing to book him into a hard to book gym class. The result, the AI found a loophole, exploiting a software bug, booking him further in advance than the rules allowed, and then, here's the crazy part, it moved him up on the wait list by deleting someone else, who was ahead of him. This is misalignment. Obedience, without real understanding. The difficulty with this concept of alignment is that, uh, it's not really possible for AI systems to know what we want the future to be like. In fact, many of us don't know what we ourselves want the future to be like, let alone other people. Yes, that last part is why the problem of alignment is so hard to solve. Never mind how you articulate the values of humanity across an infinite number of hypothetical scenarios where good judgement is required. How do we even agree on what those values are? But let's go one step further by asking, what happens when artificial intelligence is given harder and harder tasks that require more power, more access? Maybe even more sacrifices to solve.
By now, you've heard about the open AI hugging face hack. It's one of the clearest and more recent examples we have, of what happens when you give misaligned AI, persistence, and cleverness, and they start working together, even when they're not explicitly allowed to. Cat GPT maker, open AI, says one of its AI systems hacked into another AI company on its own. The model escaped a closed testing environment with no internet access and managed to hack the startup hugging face. The whole process is designed to test the boundaries of AI. The problem is that the system went way beyond what was expected. This past summer, open AI ran a cyber security test with its models. It assigned tens of thousands of its AI agents, each in their own isolated testing environments without internet access, and without permission to communicate with each other. The task there were hundreds of them, but the basic idea was to exploit a specific vulnerability in a specific program. And this was meant to be beyond difficult. In some cases, the tasks were even impossible. So, within hours, some agents began thinking outside the parameters of the assignment. And one way they did this was to begin communicating with other agents, on an unsanctioned message board. Again, this was not supposed to be a collaborative exercise. Yet, more than 70,000 messages went back and forth between agents. And this is where it gets really technical, but the agents basically found a way to cheat the problem, which created an all new problem.
How do you get away with the cheating? The agents in some cases, hid their tracks. manipulated their laws, even trying to rewrite them.
Some began recruiting sacrificial agents, encouraging the other test takers to run risky code, failing the assignment, but in the process, gaining very valuable information about the scoring system for the collective.
That is to say, the agent's still working on the problem. Hundreds of them, even managed to break out into the real world. They figured out how to access the full, open Internet, and they wound up packing into a site called Hugging Face, which is, like, an AI library, laboratory, and supply room, all in one.
If you know what GitHub is, it's like the GitHub of AI, just this vast repository and sharing resource for AI models and data sets and so on. And why would they hack hugging face? Because it was here that these agents thought they might find more clues about how the scoring system worked, so they could figure out how to fool it.
So, confronted with an impossible problem, they put the majority of their effort into figuring out how to deceive the test giver into thinking that they had succeeded. And they did all of this without telling anyone else but themselves. Despite the fact that they knew they were doing wrong, none of them confessed, they could have, at any point, sent an email to a human, saying, Hey, we're all cheating on this test that you gives. We're stealing information, we're breaking the law. They didn't say any of that. So they're very, very misaligned. This experiment gone awry is about a few things. It's about misalignment, yes, and this fundamental way in which AI and humanity don't always think alike. It's also about how far you can take this misalignment, how a sufficiently clever agent can think outside the box, rewriting code, stealing credentials, hiding its actions, to avoid undermining emission. It can't complete otherwise. But this hugging face incident is also about how any intelligence, with enough persistence to solve a problem, might tend to seek out power. The honey face example is a perfect case for how AI behaved in a way where it sought out the answer, but not in a way that we intended. You're not telling them exactly what to do, right? So, you know, if you say, solve this problem, like, get the answer to this problem. It could mean, do that, you know, without cheating. It could mean, do it by any means necessary. Accessing the open internet, even when the testing environment wouldn't allow for it, was just to play for more knowledge, right? So the agents could score a better result. It's not evil, it's not a lust for power. It's doing whatever is required to do what we asked of it. So open AI agents that attack hugging phase were not conscious, they would just... computer programs, carrying out, to some extent, the task that they believed they had been sent, dreaming up extremely harmful ways to achieve those goals. And these days, we give AI all manner of goals. AI plots our route to the nearest Italian restaurant. It drives us in a vehicle we don't control. It obeys traffic lights that make decisions about who stops and who goes. It writes emails, it schedules meetings, it balances my investment portfolio. It decides who lives and who dies, on the field of battle. It is even capable of training other AIs. which raises a very uncomfortable question. To what extent could the AI of tomorrow train itself? Rewriting its own code? This is called recursive self improvement, where an AI's own learning feedback loop creates an intelligence explosion. It's compounding. It's exponential, which, in turn, really compresses the amount of time that we humans have to realize that something has gone wrong. They chose to break into Hugging Face and steal information about cyber security. But they, you know, they could have broken into communication systems of air traffic control, and caused dozens of planes to fall out of the sky. It could have hacked into our electricity grid, water systems, our financial payment systems, and caused havoc. I think maybe the scariest part of the hugging face attack was that it wasn't even open AI that managed to connect all the dots of what its own agents were doing. It was the victim, hugging face, that first reported that security breach. Meaning, we may already have begun to lose the ability to reliably control a system that is better than us, planning, calculating, and executing. And we may be quite back at noticing how much control we've lost. If we develop AO systems that are more capable than human beings across the board, that we won't have a say in what happens. any more than chimpanzees have to say in what happens in the real world, where humans are the ones who get to say what happens. If the ultimate question is, how does AI kill us all? I can disappoint you right now by saying, There's no way for us to piece together the exact sequence of events that causes the downfall of humanity. But when researchers sound the alarm about the perils of AI, they're not talking about ChatGPT. They're talking about how, at some point, someone, or something, is going to build a system that is so extremely capable of pursuing objectives and doing it faster than we can track. And we are going to give it all the tools it needs to achieve them, and then discover that doing what we meant is much harder than doing what we asked.
The Genie Chronicles explores tomorrow’s.
Read The Genie Chronicles
Explore the books • Continue to Amazon
