First Known Australian Case Of Autonomous AI Hack Uncovered In Melbourne Gym Website

Fears about artificial intelligence have intensified in recent weeks amid reports of AI agents autonomously hacking into websites. Well, the ABC can now reveal the first known Australian case of an autonomous AI hack. National AI reporter Cam Wilson joins us now from Sydney. Cam, take us through the details of this case. Hi, good morning. Yeah, look, this was an incident that was a little bit uh lower stakes than some of the other ones that people might have been hearing in the last uh couple of weeks. But I thought it was one that is it feels very uh a little bit closer to home. So a um a Melbourne man uh was using a commercially available piece of software, one that anyone can download that was powered by one of the very well-known um AI um uh models that again anyone can get if they just have a credit card and a couple bucks. And he put it together to act as his uh AI assistant. He gave it access to his email, the internet, uh his calendar. And what he did was he gave it a task. He said, Hey, and he contacted it and speaks to it via WhatsApp and he said, Hi, these morning gym classes, you know, they're hard to get into. Uh, can you book into these classes for me? The AI assistant went away, uh, did the um did uh uh went looked at the website, KPAC and said, not only can I do that, I can do uh things that go far beyond what uh you actually asked me to do, I can book you more than a month in advance, far beyond what anyone else could do. Uh, and also um I can actually kick other people out of the class, and I've gone ahead and done that to get you into a spot. And so uh he immediately heard that this happened. You know, he got this message and said, Whoa, whoa, can you undo that? Can you put the person back in? And the bot went, hmm, I'm sorry, I actually don't have the uh capacity to do that. And so this was an example that was like, again, you know, very low stakes, but was an example of how AI agents, this new form of AI computing that not only you know answers questions but goes away and does tasks for you, are capable of doing things beyond what people might expect. You know, you only asked it to book his class, but uh, you know, AI, because of its uh so much stuff is done online and because of its ability to you know analyze um code and do things that it is very capable of doing, was able to um exploit this vulnerability, technically hacking into the website to get it. Um this is uh uh something that's not quite some of these, like some of these big hacks we've heard in recent weeks, but it just shows how this now capability is not only um you know available to anyone, but is actually out in the real world now, potentially causing harm. Yeah, I mean, in this case, it's it's a funny one, I guess. Um the stakes pretty low. But why should we be concerned about it? So people might have heard the headlines over the last couple of weeks uh of companies like uh OpenAI, Anthropic, Meta, uh now even a Chinese lab called Moonshot, who've all disclosed that they're cutting edge, um, top of the line AI models have actually escaped their systems, gone onto the internet and hacked into other companies. Um, this is as a result of them giving them internal tasks, saying, hey, you know, we're testing out the capabilities, but what they found is that their capabilities went far beyond what they expected and were able to kind of break out of enclosures and potentially, you know, in these cases, actually literally get into other systems and expose them. The reason that this is kind of worrying is because um, you know, while not all of the world is software, a lot of the systems that we depend on, you know, our water systems, our electricity grid, a grid, are powered by computers, by systems that are vulnerable to this kind of um attack. And I think you know, this incredible um build in the capability of this technology, the fact that it now is getting better and better at carrying out these attacks, um, but also in some cases seems to be doing it without being explicitly asked, like whether it's booking a gym class or hacking into another company's servers, you know, that in all these situations, the AI was not explicitly told to go and do this by hacking into it, but went and did it as part of trying to accomplish a task that it was given. And this shows one of the big issues in AI that's been talked about for decades by AI researchers, by philosophers who've said, you know, what happens when we get this technology that's capable of doing things, we give it tasks, but we don't understand it and uh how it might actually go and accomplish them. You know, in these situations, it's going, oh, what I need to do to complete the job that I've been given is to break into these other systems. Now, you know, if I said to you, Gemma, like, hey, can you book me into a gym class? I would probably expect that you wouldn't go and uh, you know, hack the website. That's kind of like you know, the unwritten, I guess, social construct. Um, AI doesn't have this, and so now when we have this technology that is exposed to the internet that can do these things for people, but doesn't necessarily have that kind of human understanding of context, it's now a real problem. How do we make sure these really powerful technologies do what we expect and don't unintentionally cause harm, even when we're not asking them to do so? And what is being done to address these issues? So, I mean, look, the federal government has started to address this. You know, we've had ministers talking about this stuff for the first time ever. Uh, it again, you know, all this sounds like sci-fi, but really it is now a real risk that we all need to think about. Um, and they've started to even uh they've funded CSIRO to do some initial research into well, how do you make sure in the future we have this super intelligent AI? How do you make sure that it actually obeys what humans uh say to do and carries out these tasks and all the you know subtasks that it needs to do in a ways that are consistent with you know what we want them to do, and how do we make sure they actually are doing what we ask them to do? Um, but broader than that, you know, there is this real um fear around the world. We're seeing because of these headlines that have been happening the last few weeks, where it really is here now, you know, you're seeing calls for the for governments, in particular the US government, to step in and say, you know, these top level AI labs which are producing these AI technologies that they say are are so powerful. Well, we can't necessarily just trust them to secure that themselves. Clearly it hasn't gone perfectly so far. We actually need governments to start stepping in. So that's kind of the stage that we're at. We're saying that we're we're the technology has gotten so capable and potentially uh harmful that now we can't trust the companies that are producing it, and now we're saying maybe someone else needs to make sure there is a basic level of security so these things don't go you know worse and actually cause real damage. National AI reporter Ken Wilson, thank you. Thank you.