CamFeed

One of the strongest voices to enter the debate on artificial intelligence has been Pope Leo the Fourteenth. He’s currently in hospital following lung surgery and a newly revealed cancer diagnosis. But his message of caution has made its way to Australia this week with a visit by one of his key advisors, National AI reporter Cameron Wilson. Reports. Artificial intelligence needs to be disarmed. He's the head of a centuries old institution. But the Pope isn’t shying away from addressing a very modern issue. The concerns raised by many experts, specialists in AI, should be taken seriously. I don’t think that is a fake news, as some have said. This is a problem that I think we need to sit down and talk about. The man entrusted with leading some of those talks is a Franciscan priest who's quick to point out his order’s long involvement with modern computing. Father Paolo Benante, a philosophy professor and former engineer, was in Australia for a series of talks on what he calls the greatest moral question of our time. Magnifica Humanitas, but the undertitle is more important, maybe that the title, and say about the human dignity in the age of AI. In May this year, Pope Leo the Fourteenth delivered his first encyclical, a 40,000-word document urging against giving Silicon Valley too much control. The title is Magnifica Humanitas, but the undertitle is more important, maybe that the title, and say about the human dignity in the age of AI. US President Donald Trump hosted tech titans at the White House last month, they pledged to police themselves. I do a lot of street photography. Because I love to freeze such kind of poetry inside the chaos of life. When I was in the age to drive a scooter, they forced me to have a helmet. So probably we need a brain helmet. If the regulation of companion chatbot will be like a brain helmet, I’m much really positive. Because otherwise, if you don't put rule of law inside, it will be the rule of the wild, the rule of the strongest. What is it like? It sounds like everyone signed this binding agreement. Is it binding in any way? Father Benante has been in the room for talks with top tech leaders. We’re seeing tech companies want to build data centers down under, as long as they can mine Australia's creative work to train their tech. So is this fair? I don’t know. Those are not neutral things, but are things that are designed by someone and they would like to monetize the use of these things. We do see some countries actually banning kids from using AI as companions. So is this something that will produce real richness or pay back to Australia? Well, Father Benante warns this risks repeating old patterns of colonial exploitation. War, and that is the most brutal form of geopolitics, is AI-driven drones on this new kind of autonomous decision-making system that is able to touch what we have of most precious, that is human life. So, yes, war, and that is the most brutal form of geopolitics, is AI-driven. What does it mean? That’s a really big question. I don't know. It sounds like you see technology policy increasingly through the lens of geopolitics. Father Benante believes that AI has the potential to serve humanity in a positive way. But what really worries this man of God is not artificial intelligence. So, is a human being trusting in an untrustable automatic system?

But now let's get more on OpenAI's apology for hacking Australia's Medicare Statistics System in June with me in the studio is our national AI technology reporter, Cameron Wilson. Hi there, Cam. So what more did we learn about this hack? Yeah, so in a uh quite a contrite sounding blog post put out today called How We Will Do Better for Australia. Um, OpenAI put out an unsigned message talking about um what had happened in the incident, giving us some more details, and then talking about what it was going to do next to, in their words, try and re-establish trust in Australia. Um, and so in that we got some bit more information about um how the hack occurred, what it affected, and um how they also dealt with it in the aftermath. Was anything really new in there that surprised you? So there were a couple of things that stood out to me. The first thing was that we got a bit more knowledge about exactly um what the OpenAI agents had done to that Medicare Statistics web portal, which is run by Services Australia. People might remember that there were actually four different government websites accessed by the agents, but the government said, you know, the the three of them were not such a big deal, accessed relatively in the way that a normal person would do it, a real person, I guess you'd say. Um, whereas the Services Australia Medicare Statistics portal, we knew that these AI agents had gone further than just looking at it normally, but had done something that the government had called a hack. And we got some more details on that, which included saying that it had written to the servers, technical mumbo jumbo, but essentially what that means is like you know, it's a difference between going and looking at someone's door and you know poking your head in versus like actually trying to like stick a key in there and get in, and in this case, actually getting into non-public files. They also did other things like um try and find credentials, so that's the kind of login. So that's like you know, looking around the house, trying to find some spare keys anywhere, um, and that kind of thing. So that was quite, I think, notable in the kind of context of this because we'd heard that this government website did not have great security, but what we've heard now from OpenAI shows that it was not just purely a poorly secured government website, but some real kind of offensive, cyber attempt offensive moves by the AI agent. The other thing that I just noticed just before I came on air was that OpenAI said that it said that what was responsible for this hack was an internal model that did not have its normal cyber safety safeguards on it. So they said, you know, this is not one that we would release publicly, it would go further than what we you know than that. We knew that they had those models because that kind of model was responsible for that big um hack that we heard about a few months ago, that open AI hugging face incident where it actually hacked into another company, which was quite severe, um, a much bigger deal than what this one was in terms of actually what happened. But in this blog post, by attributing um this hack to that, they also said that the task that it was given was getting some medical data. It was told out, go and look at the internet, get up some research on specifically the spending on skin medicine in Victoria, some very specific stuff. But putting those two together, that's actually something new. They have said, according to this blog post, that they had taken a model that they knew didn't have its normal safety measures and intentionally let it go out and access the internet. That's beyond what we knew from the OpenAI hugging incident, uh hugging face incidents because we were told it broke out of an enclosure. In this blog post, I believe it actually says that they've got this model that wasn't given its normal safety measures, and then they said go on onto the internet. And I think that's a kind of like an important revelation in this. So open AI sounded contrite to you, said, but what are they promising to do? So they've announced a couple of things that they've said that one, we're working with government departments and those affected. Um we are uh uh wetting Australian government organizations, so businesses and governments take part in, they've got this, I think like $1.4 billion fund for kind of free access to open AI to bolster cybersecurity, you know, to turn the AI on your own systems to find vulnerabilities. Um and they also said they're gonna establish a Australian task force. So I think a group of experts who are gonna help advise it and also the Australian government on how to avoid something like this in the future. Thanks, Cam. Thank you.

Let's bring in our AI reporter Cam Wilson now, who's with me in the studio. Good to see you, Cam. So, what do we know about exactly what happened here? Yeah, so we heard this morning from the Prime Minister that uh OpenAI uh had told them that in June they were conducting some internal evaluations of their models, so putting their AI through its its paces to see what it could do. And as a result of that, as a result of the tasks that they gave it, that AI model accessed a number of government websites, I think you said four, um, as part of some kind of uh medical health related kind of question or challenge that it was trying to do, including accessing some information on uh Medicare aggregated health statistics that was not meant to be public. That happened in June. Um, as people who've been involved in the news might know, there's been a kind of number of things that happened in June to OpenAI, including a very um well-known now um OpenAI Hugging Face attack, which was this bigger incident that many people have heard about. Um, as a result of that kind of big event, OpenAI has been reviewing what exactly were our AI models doing during this time. Uh, and then they say as a consequence of that, in August that they discovered that its models had access to this non-public um health statistic data. And then in September, after doing some validation, you know, double checking all these things and making sure they understood what it was, they told um Services Australia, and then in the two weeks since this kind of filtered up until this morning when we heard about it from the Prime Minister. Okay, so what's OpenAI had to say about this? So OpenAI confirmed that it's happened. They said that they um they kind of gave us the timeline of what happened, and they have stressed that this is you know not specific individual health information. This is and this is uh uh corroborates what the government has said. This has come this top level aggregate data, and they said that they saw some internal file names. So they are very much saying that you know, while it did do something that kind of went beyond kind of what was intended to be public-facing information, what was seen was not a big deal, and they said that they have kind of doing this investigation now and we'll have more to say about it soon. Okay, so you've been looking at public logs detailing AI agents' activities. What have you found? Yeah, so I mean, this is a part of this, like, you know, we had this big incident that happened that we learned about it now a couple of months ago, and OpenAI decided to, you know, open its books and look at what these models have been up to. Um, one of the things that we actually found out about early this month, not from the company but from a bunch of researchers who spoke to Reuters was that um their AI agents, uh unbeknownst to OpenAI had found a German coding forum that they used to kind of talk to each other and coordinate their tasks. I they call it like a scratch pad. Looking at that, I've been able to find out um some of the kind of things that they were doing. This was at the same time as what the government has disclosed. It has not been confirmed that it is exactly the same incidents, but in these kind of chats between the AI agents, they are talking about accessing Australian health data, including one of the websites that was mentioned by the Prime Minister as accessed by OpenAI, not the one that was um done in a way that was um considered hacking. But when I looked at the kind of methods that they were doing, some of them went a little bit beyond just um you know trying to access it like anyone would, but we were intending to circumvent some very basic anti AI agent, anti-bot cybersecurity methods. So we can see that they were trying to get around kind of the walls that uh AI that you know the government was putting up to protect its stuff online. Just a question without notice, Cam. When the operators of AI are setting these challenges and tasks, can't they say, do not do this, do not breach these non-public websites? It is like that is like one of the big you know questions that kind of come out of this because you know, fundamentally what a lot of um you know AI does now for people is you might say, you know, the the example they always say is like plan me a holiday. To do so, they've got to do all these subtasks, you know, figure out where you want to go, check flights, all those kinds of things. It's that delegation of those subtasks. When you said plan me a holiday, you didn't say go check, you know, flight tracker or whatever that they have to do. And this big challenge in AI at the moment is how do we make it so that when we give uh you know AI one overall goal that we make sure that all of its sub goals, you know, all the things that's doing to uh to achieve that are on the level, are what they say is aligned, doing things that we want doing things in a way that we would expect it to act without making explicit. So the they the the like long answer is that this is a big thing at the moment, it's a big kind of um controversial area in the field, um, and it is something that is obviously very much an open question. But they they do say we are making efforts to try and act in ethical ways, um, but obviously it's short it falls short at time. Finally, acting uh PM Richard Miles uh used a fence versus fortress uh analogy today to describe the data protections the government is using. So, how protected is Australian government data from these types of breaches? So I think this is a really important part of this. The information that was accessed is not particularly sensitive. It was not intended to be public, but I think the Deputy Prime Minister later said it has since been made public since this incident. So there was no like we that what has been accessed is not something that is, you know, um a top national secret. And correspondingly, it was not very well secured in terms of cybersecurity. So when the Deputy Prime Minister gives this you know metaphor of a fence, it's the difference between you know looking over a fence and seeing something in someone's front yard and maybe stepping over a little fence versus like a tall fence with like guard dogs and fort knocks behind it. Um it was not particularly well secured. However, the reason that you know that we are paying so much attention to this is that this is just a new threat, a new kind of challenge that everyone from government institutions to even individuals have to consider because AI is making it not only easier uh to hack, is not only getting more powerful, but it just because it's accessibility means that more people have access to it. So, you know, it's gone from being like if someone wanted to hack you, you know, they'd have to put in effort, they'd have to have the expertise. Increasingly now anyone can do that, or they the AI agents trying to do a bigger task might do it on their own. Thanks, Cam. Thank you.

Documents leaked to the ABC reveal that the government is proposing to change copyright law to attract AI companies to Australia. Labor is rejecting criticism that it’s selling out the work of creatives to big tech. This comes amid warnings from the AI industry that it needs regulation. It may be your smiles here, but the growing alarm bells on the risks of AI are hitting the share prices of big technology. Industry warnings to slow the pace are getting louder by the day. AI giant anthropics co-founder Jack Clark goes further. Should you mandate that companies definitely have a kill switch, and is that kill switch verifiable by a third party? I think that’s the kind of thing that society's going to want to know and might want to eventually pass rules. Around. Strong protections for Australian artists and the media will remain a core part of Australia’s approach to artificial intelligence. Camelson, ABC News.

Julia Andre reporting there. Well, returning to our top story now. As we've been reporting, tech bosses are stepping up calls for a regulated mandatory kill switch for AI software. At home, the nation's cyber intelligence chief has warned Australia needs an artificial intelligence early warning system to help defend against the dangers of the increasingly powerful technology. Australian Signals Directorate, Director General Abigail Bradshaw also urged Australian organizations to embrace AI for its defensive capabilities. Our national AI technology reporter Cam Wilson joins me now. Hi there, Cam. So just take me through these comments from the Australian Signals Director, uh Directorate General. Yeah, good morning. I um yesterday at a conference which was about um AI by um the think tank ASPE. Um the Director General made some comments about how AI was affecting what she does and what her agency does. And um, you know, by now many people have heard there's this really big global story at the moment about this disruptive uh capacity and potentially catastrophic consequences of AI continuing to develop. She was more talking about the here and now and talking about how already these advances in AI are changing what she has to do and what her agency is dealing with. So she spoke about, for example, the fact that there are these new AI-assisted threats. You know, people are using AI to help with with hacking and other cybercrimes. Um, and also we've, you know, we've even heard about um autonomous AI agents, you know, themselves committing those kinds of um incidents. But she also gave a bit more of an optimistic tone when she was talking about the way that her agency is using AI. She was talking about how um, in some cases, the AI and its ability to help with cyber uh defensive capabilities has meant that things that used to take engineers at the uh ASD weeks has now taken them hours. So she was, you know, we've had a lot of doom and gloom, and no doubt there was a bit of uh potential bad news from this. But she also said that these uh that AI is proving itself to also be a weapon for for her agency. Cam, we've been talking for the last little while about you know this push to slow down the development of AI. You've heard from the assistant technology minister Andrew Charlton, though, saying that Australia can't count on a slowdown of the pace of AI development. Talk me through that. Yeah, so there has been this recent resurgence of this discussion about slowing down AI development over the last week or so. I think pretty much all of the heads of the major frontier AI companies have come out saying, you know, we need to do something to slow down development. They call it pacing the frontier. They're technically saying it's not slowing down, it's pacing, but you know, I think that's pretty much the same thing, just with a bit of marketing sprinkled on top. They're still talking about continuing to develop it, just not at the pace that they are worried about, you know, what happens. What do they mean by what's happens? You know, they mean um, you know, for example, AI starting to become so capable that it reaches what they call recursive self-improvement, which is when AI starts actually being able to assist itself and improve itself faster than people. They call that moment takeoff, and so that's something that people very fear because it introduces this idea of increasingly humans being involved less than this. Um Minister Charlton spoke about this general Maloo over the last um you know week or so, something that I should say has been in discussion for quite a while, but has recently been, you know, particularly prominent. Saying that while, you know, he is he hears these calls, and he's uh of course the Australian government hopes to see safe AI development. He also I would say struck a bit more of a skeptical tone saying that the financial incentives for these AI companies, because they are companies, um, very often with you know demands for returning on investment, um, have these incentives to keep improving. And so as a result, you know, I think that the tone that he was trying to strike is a real one saying that we can't just, you know, hope and and bet on that these companies will find a way to coordinate to slow down. And we as a country need to continue to do policy that assumes that it's not just it will that we are gonna have to be part of something that ensures AI is taking, you know, is being responsible, not just leaving it to the companies themselves. Ken Wilson, thanks for taking us through it all. Thank you. The community of Wyala in South Australia is facing an uncertain future.

Thank you. But dozens of projects already approved are expected to escape those requirements. An ABC analysis has found this next generation of centres will be much bigger and more power-hungry than those already operating. Cam Wilson is the ABC's national AI reporter, and he joins me in the studio now. Hi Day, Cam. So take us through what the government is proposing here. Yeah, so a couple of months ago, before the recent headlines about how AI is going to kill us all, we were getting a much more positive message about artificial intelligence from the Prime Minister when he delivered a landmark speech in which he said that Australia would love to get this investment from foreign AI companies into Australia. That's an important technology. But saying, you know, we want to do this on our own terms, talking about you know the social license that these companies need to have to build these data centres here. So kind of like you know, rolling out a red carpet but having a bouncer there, saying you can come in, but you've got to uh meet these uh standards. Those were specifically about things like power and water use. Um, many people have probably heard that there's been community concern over the environmental impact of these data centers. And so the government said we're going to create these new national standards uh that we will have to work with the states with uh and we'll try and legislate by early 2027. Okay, so since that announcement from the Prime Minister, we've heard a little bit more about its implementation. What more do we know now? Yeah, I mean, one of the big challenges with this area is that there is so much appetite at the moment uh to build out the data centres that are the infrastructure that kind of powers the um generative AI boom, you know, products like Jack GPT and Gemini and those kinds of things. Uh, and so while the government is kind of working on these things, um data center approvals continue to happen, construction is is keeping happening. And what the Prime Minister said shortly after that speech is that uh these rules, which are not still expected to be introduced into law for at least six months from now, they will not be retrospective, already um uh built and obviously in construction uh data centres will not have these rules apply. Analysis that we did looking at uh planning documents and company records shows there's at least 25 significant data center projects that have already been approved and are currently being built that may still yet be years away from actually you know coming to fruition that will then escape these rules and therefore not have to apply, uh not have to comply with minimum power and water standards. And so, what does this exemption to these new rules then mean for the communities around these new projects? Yeah, so I mean the Prime Minister was saying, you know, we need this social license, we need to have these rules so that people feel comfortable having them in their communities. Um, you know, one person I spoke to in the story who is upset about a local data center development said, you know, I personally didn't want it either way, but he was saying, at the very least, if it is going to come here, I'd like to see it improve its environmental impact. And that person told me, and this is you know, um, not just limited to them, that they are disappointed that this these new rules that are supposed to set these minimum standards won't come into effect for quite a while. So there is this kind of mismatch between the speed of development and the speed at which these new rules are coming into effect. And so, as you said earlier, the Prime Minister was looking to sort of roll out the red carpet somewhat to investment in these data centers. But is the uncertainty over these rules now affecting investment in Australia's data centers? Yeah, according to analysts I spoke to, they're saying that if it is dampening it, there's so much demand that it's not really stopped the you know huge appetite to build this out. I mean, like these data centers, um I mean, I think Westpac estimated at bringing in about 150 billion dollars in in investment just into the data centers themselves. Um, so there's this enormous amount of money coming in. Australia has this situation where we are, you know, a country that has uh a good energy grid, skilled workforce for Pi of Five Eyes, a number of things that makes us a very attractive destination for major AI companies to set up here. Uh, and they are clearly very keen to do so. And so, you know, an analyst I spoke to mentioned well, like developers are used to dealing with the uncertainty, you know, regulations are always changing. That being said, you know, um, the head of the Data Centres Australia Group said, you know, we think that we are doing enough to already address some of these social licence issues, but we can't um comply with rules that are aren't in effect or won't even kind of established yet, aren't even specified. So the fact that we're not necessarily meeting these doesn't mean that we won't, but it's hard to say that we are if they don't exist yet. Cam Wilson, thank you very much. Thank you. Well, Queensland is racing for its first credit rating downgrade in more than 17 years.

And together they defined the spirit of a nation. Announcement about how Australia was going to seize the opportunity but deal with the challenges and risks of AI. And it could be best summarized as almost a that we're rolling out the red carpet for AI companies to come here. But we've got a bouncer. We're setting some kind of rules around what we want here and why we will welcome them in. And that was specifically around the idea that you know these companies are going to bring a lot of foreign investment. The technology is likely to be you know crucially important to industry of the future. But you know, the government is very aware of the growing uh community sentiment around um uncomfort about some of the data centers that are popping up to uh power this generative AI boom. And so as a result, you know the Prime Minister said uh to have social license to allow these data centers to be built here, um, we've got to set minimum standards at a national level around things like uh power use, energy use, and even where they're situated. Okay, now since that announcement, Kent, we've learned a bit more about its implementation from the Prime Minister. What did he say? Yeah, so I mean, obviously, these um new rules are when he announced them, he said, you know, we're going to uh have to figure them out, we're gonna have to hash some stuff out with the states as well because um you know many of these issues are not just purely federal but also involve the states as well. Um, and so subsequent to that, he said that these rules will not be retrospective, meaning that they won't apply to data centers that already exist, but also ones that are being built uh right now. Um, and so ABC analysis based on looking at um state planning documents uh and also company announcements shows that there are at least 25 new data center projects that will escape these rules, even though many of them will not be built for potentially years. Now, some of these may not end up being built, and and no doubt there'll be more as well approved before these new rules come into place. But what's important about this is that these new data centers are this kind of new generation that are bigger, consume more resources than the ones before it because this generative AI boom, you know, the things powering chat GPT, the kind of AI answers now that you get when you search something in Google, they require a lot more computing power than traditional uh uh computing other methods. And so as a result, you know, um we have to build up this infrastructure, these products have been hugely successful and very, very uh, you know, widely used, but also it means that we've got to build these data centers out, and we know that um there's this kind of rush at the moment to get uh new approvals through to build them out, meaning that these new rules that the government says is important for communities to feel comfortable about them built being built, won't apply to them. Yeah, so then what does this exemption from the new rules mean for these communities that have these new projects already going ahead? Yeah, so I mean, look, um it should be clear that even though these national rules aren't already in effect, there still are state planning rules, and um, you know, many people might compare this to the US, which has um kind of significantly um more data centers already and kind of more ones being built, and they tend to be bigger. Australia is not quite there yet, although these new ones that are being proposed, have been approved, are currently being built, are significantly bigger. They still are subject to state planning rules, but the federal government is like, you know, these federal rules will kind of set more minimum standards that we think are necessary for for communities to feel good about it. And so I spoke to you know, people who are um have data centers being built near them, and they um, you know, including some people who are opposing them, and they said, Well, look, you know, I'm I'm I actually don't want it built here, but if it is going to be built here, I'd like it to have you know certain um minimum standards around things like water use. I want to see them use less water. And so, you know, they were saying that they wished that these federal government rules were already in place, um, but but obviously they are not yet. Um so is the uncertainty over these rules affecting investment in Australia's data centers? Well, I mean, an analyst I spoke to uh suggests that uh there is plenty of demand. We are seeing this enormous um uh interest and appetite to build data centres in Australia. Um, you know, certainly if it is dampening interest at all, uh it hasn't counteracted the the this great um appetite for them. Um, you know, I think Westpac said there's about 150 billion dollars worth of data centre projects in the pipeline. Um, the analyst I spoke to said that like there's a lot of interest. Australia is a very attractive place for this, and that this, you know, if we build them here, there'll be a lot of benefit for the um Australian economy. Um so we're definitely seeing it happen. I think the just really fascinating thing is that these data centres are being built, it is going ahead at such a pace that is outpacing the federal government's attempt to set rules around them. Cam Wilson, thank you very much. Thank you.

Australians were among hundreds of thousands of passengers who were affected. Harrison AI is an Australian company. It was founded in 2018 by two brothers. And what it does is sell tools that are used to help radiologists interpret scans. They've trained these tools on a number of, we don't know the exact number, but according to government documents, fewer than 30 million patient scans that were obtained from Australia's largest radiology clinic, IMED, without patients' knowledge that was being used to train those. They've since become a very valuable company. They're backed by some of Australia's startup scene's biggest names, and also the Australian government tipped in $32 million at the start of last year in a fundraising round that valued them in the kind of mid hundreds of millions in terms of evaluation. So they are one of the success stories of Australia's startups in the last decade or so. So what do we know about these layoffs now? So the company has laid off some Australian staff. I don't know the exact number. Several of them have posted online, but you know, based on discussions with people inside the company, we know it's you know in the mid-double digits is what's supposed. I've of course tried to speak to the company who hasn't responded to a number of requests of comment, uh, other than picking up the phone and saying that they don't uh answer unsolicited calls. Um but the layoffs were done as part of the company's restructure. They're talking about really doubling down on AI inside the company. Now they sell AI tools, but internally they're talking about going to a kind of model where every employee isn't just you know a worker themselves, but thinks of themselves as a uh a player coach or uh a manager of AI agents to do their work. Um, so they have laid off some staff. Uh, a company executive in an interview uh uh that was posted online, says this move has also led to other staff um leaving as well. And around the same time, the company has also launched an affiliated US company that is going from selling these AI tools to directly hiring doctors to use the school uh the tools in a teleradiology service. Okay, so just take us through some of the questions now that are being raised by this new business model. Yeah, so experts I spoke to raise the idea that combining someone who both sells the tool but also hires doctors to care for patients raises a potential conflict of interest because you know ultimately a doctor is supposed to be fundamentally just care about their patient care. That's you know their number one um guiding principle. But if you are hired by someone who is giving you a certain tool, um, you know, there are questions about, for example, could a doctor choose not to use that tool if they're employed by the company, or even at a more granular level, there's a well-known concept in uh AI ethics and and uh healthcare, particularly, where they talk about automation bias. And that's the idea that if a person is using a tool that helps it make decisions for them, sometimes they might defer to the suggested decisions, you know, the automated decisions, even though their better judgment might suggest otherwise, they might just trust the machine when maybe they shouldn't. And experts said, you know, there is of course like a kind of implication about if the people who hire you make the tool, whether that kind of then places pressure on the people, the doctors using the tool to accept what it says. Now, um, the company again didn't respond directly to me, but has said um, you know, it's this US affiliate is actually an independent company, and that the doctors who it hires are independent and can question that judgment. And I should also say that the company, as well, you know, based on what the um experts who I spoke to said that you know this could be an opportunity for a um for these doctors to use a health uh uh tool that could make them more efficient. So, you know, there are definitely upsides to this as well. And the company says, you know, based on it's got advertising material on its website for the doctors that it's trying to hire, saying that we expect that you'll be 30% more efficient using this tool. Um so there's definitely an upside, but there's questions about how do you use that responsibly and and are those kind of checks and balances to make sure that ultimately you're doing everything you can to ultimately care for the patients. Right, Cam Wilson, thanks very much. Thank you.

thank you. Thank you. Let's turn now to other news and internal New South Wales government emails obtained by the ABC reveal that AI company Anthropic was interested in locating up to five gigawatts of AI training capacity in the state. Now, this number represents more than three times the entire capacity of Australia's data industry. It's really quite remarkable. Let's get more on this with national AI reporter Cam Wilson, who joins me uh in the studio. Cam, you you are all over this story. Talk to us about these emails. What what does it show? Yeah, so New South Wales Parliament released a tranche of emails from within New South Wales government about um interactions that the various parts of the government had been having with AI companies and data center companies. Those are actually kind of different because while many of us would know major tech companies who do AI, whether it's OpenAI, Meta, Google, Anthropic, there's also a number of companies that are Australian-owned that actually build the physical infrastructure in Australia. Um, earlier this year, Anthropic's uh US executive uh was making a visit to Australia where they end up signing a um MOU, a memorandum of understanding with the Australian government saying essentially we would love to, you know, put more of our you know, do more work here, put more of our data centers here within the kind of conditions that you would like. It's a kind of very high-level document saying, you know, we're we're into it, we'll start looking into it. But prior to that, um, emails that I got from uh inside the New South Wales government shows that we got a little bit more detail about exactly what they were interested in asking for. Specifically, this number of five gigawatts of uh AI training computer capacity in New South Wales alone. And that's kind of the first time that I think we've heard a figure like that to do with the state that gives us some insight into exactly what they want to do and put in Australia. So break down that number for us. What why is that such a significant amount? Well, uh, there's numbers out today from Data Centres Australia, which is a lobby group representing data centres, predictably, um, saying that currently Australia has 1.6 gigawatts in Australia in total. This ask from Anthropic, which I should say was a very preliminary discussion. It's been some months since then, but suggested they wanted up to five gigawatts. So, you know, my mass isn't great, but it's more than three times that, potentially in one state alone. Now, of course, that doesn't necessarily mean that they would get that much, and perhaps if they got more in New South Wales, they wouldn't get as much elsewhere. But clearly, we can kind of see from that that this company is having discussions with government suggesting that they are really interested potentially in making serious investments in Australia. That would mean some pretty serious infrastructure build-outs. Um, you know, again, over what time frame? We don't know. These companies, you know, are dealing with the here and now, but they're also probably trying to think, you know, years, even a decade in advance. This stuff building out takes a long time. So, you know, I'm sure they're all trying to figure out how to get their ducks in a row. Um, but still, that that number is an enormous amount that kind of lets us understand that we've been hearing that data center companies, AI companies want to come to Australia, they want to build up this capacity, but the amount that they are potentially asking for is such a large amount. It's not just a big thing for them, it's a big thing for Australians, for politicians, for people who are building these things out to be like, let alone do we want it, but can we even give them that much even if we really want to go for it? Yeah, you know, Cam, there's been a lot of discussions about the um uh data centers and of course the kind of resources they would need, uh, what kind of a strain that would be on Australia's own infrastructure and resources as well, but also they are bringing in some investment as well. And this is a hot topic that's going to be discussed in the National Cabinet today. What are you expecting to come out of that? Yeah, so National Camet today was looking at these um what the Albanese government calls its Australian standards. This came out of a speech that the Prime Minister gave about a month ago where he said that we want to get major AI companies to come to Australia to invest here, to you know, employ people here. Um, but also we realize that we don't want a kind of US-style backlash that you're seeing to the data centers being built across that country. And so, you know, they talk about the idea of social license. How do we, you know, um allow or set the parameters for building out this kind of infrastructure in a way that Australians feel like they're not getting taken advantage of, but they're benefiting from. And that uh what his you know recipe for that included a number of things, but the two things that are coming up at National Cabinet today are about energy and water, because um, you know, those are things that the state and uh federal governments have to come to some agreement on. Um we understand that you know some of the states, Queensland, Northern Territory, are not keen on suggestions that uh that um all data centers would have to come with a commensured amount of renewable energy. So you come on in, you say we want this much power, you've then got to source that much renewable energy yourself. Um we don't know what the outcome of that is, but that is like one of the main kind of planks of this idea that the government says if we do this right, we think that we can have it so that Australians feel comfortable and happy that these companies are coming here and setting up their infrastructure. Very interesting. Uh no doubt the conversation and the controversy will continue. Cam Wilson there, AI reporter. Thank you. Thank you.

Thanks for having me on. Well, AI created music will be excluded from Australia's official music charts from this week. The Australian Recording Industry Association says it'll exclude AI generated music from Friday, unless it's substantially human made, does not raise manipulation concerns, and uses legal AI services. For more, I'm joined now by our national AI reporter, Cameron Wilson in the studio. Hi there, Cameron. So, what's brought on this move exactly? Yeah, going from you know what's powering AI to what's coming out of AI. This is something that has kind of come out of rumblings around the world about how AI generated music is uh becoming a really big presence. Um it's very easy now to produce AI music. Um, global charts, uh including Australia's have had songs that people either suspect or know to have significant AI parts of it in it, um do quite well. And so a few months ago, uh a global record association put out some rules saying that we think that these are the kind of ways to distinguish between what is a song that uses AI to assist, but it's kind of a human-made song, versus like an AI generated song. And they said either um, you know, associations around the world should follow this if they run charts themselves. And so today um ARIA, which represents uh the record labels in Australia and runs the kind of top music charts, says it's changing its rules uh to exclude um most, like either wholly or mostly produced um AI music. And um one of the precipitating factors I think that's kind of really seems to have um contributed to this, perhaps sped it up, was a song that was the most uh listened to song in Australia that I think we have a clip of now. When you call my name, it's like a little red. So, yeah, that was a song produced by an Australian producer. Um, and the ABC did some reporting on it saying that people suspected that it had AI kind of involvement uh in it, and then later on, the producer behind it actually updated um information online saying that the vocals and drums had AI as part of it. As I kind of mentioned before, this was like the most song, most listened to song in Australia in July, did quite well in the ARIA charts. So clearly we've gone from this you know idea where you know maybe AI will be creating things in the future to now we are actually listening to songs that are being seriously made with AI. And so what does this change mean for artists? So I mean what ARIA says that this is doing is kind of setting a line in the sand so people know exactly what is and isn't considered you know AI generated. Because like it's actually quite a like difficult thing to figure out exactly where the line is sometimes. Um a lot of computing that we use today involves AI. Like, you know, for example, whenever you take a photo with your phone, it uses computational photography, which involves AI. Is that AI generated? Well, it depends who you ask. In this um, you know, area ARIA is kind of setting down these guidelines saying uh unless you can prove that the um music was um made substantially by a human, hasn't had chart manipulation behind it, and it involves AI, they're gonna say that it can't be on the ARIA chart. Obviously, there's ramifications to that, whether it's from you know winning an ARIA award, or to like you know the fact that ARIA charts are used as an indication to the industry, to the to the public, you know, to history as well, about what has been a big song. Um and the other thing is that uh ARIA has also come out and say we hope that the uh radio stations pay attention to this as well, because we want to support Australian creativity, when it will support Australian uh human creativity and artistry, and so they're also now saying we are encouraging the radio stations not to play songs that we call, you know, right, mostly AI generated and instead play real artists. Thank Cameron Wilson, thanks very much. you.

Thanks, Barbara. Privacy experts and legal advocates have criticized an AI mass surveillance pilot program being rolled out in Western Australia. Since the trial began in June, cameras linked to facial recognition software have scanned more than 130,000 people in Perth and Fremantle, comparing their faces against a police watch list in real time. National AI reporter Cam Wilson has the story and is with me in the studio to tell us all about it. Hi, Cam. Hi Ross. So what is this program? What's it about? So starting in June, WA Police did the first ever Australian trial by police of a live facial recognition scanning program. People might be familiar with facial recognition stuff we have in our phones. That's kind of a one-to-one scan. So it checks your face against the one that you know you're supposed to have to open it. This is a kind of more complex one, which is known as one-to-many. And so it scans, you know, you can look at a crowd, it can scan all the faces and compare it against a watch list to see if there are any matches. WA police says that they have a watch list of about 4,000 people. That includes people who've been accused of a serious crime, uh, people who are have an outstanding warrant, missing persons, and also people who are believed to be a risk to themselves or others. And they have been trialing using this technology to pick up people in the community. Um so they've kind of taken out a van, they go out, they scan the faces as very clear signage as part of this trial saying we're doing this now. And when their system pings, it thinks that it has found someone who matches the watch list. They have someone like a police officer or officers go out and meet the person, see who it is, and then potentially arrest them if that is what the appropriate thing is to do. In this trial, it's been going on for more than a month. They've only released the first week of data so far. They said that they had 33 flags that led to 18 arrests, but also one false alert that identified someone as someone who would who they were not in this early part of the trial. And this has obviously raised privacy concerns. Yeah, so I mean, facial recognition software is quite well known for having racial biases. That's not something that is, you know, kind of inherent to the technology, but is just based on the fact that most facial recognition software and technology has been trained on a corpus of data that is primarily white middle-aged faces. And so outside of that main demographic, they're known to have more accuracy issues because obviously, you know, you train how it works is that they create a neural network. All you gotta know is like kind of like anyone else learning something. You look at something, and as you get to know it more, you're better at kind of telling small differences apart. But if you don't have many examples of it, it's kind of hard to be able to do as good a job determining between them. We so when WA police launched this trial, they said there's a UK trial of a similar technology a few years ago, and they found there was no statistically significant difference between demographic or ethnicity groups for this exact same technology. So they said, hey, look at this, they've done this trial. We know that it actually can be done in a way that doesn't have these biases. However, experts that we spoke to said that that trial was in very different circumstances, both in kind of ideal conditions, so like straight on, good lighting in worse conditions. They they the trial actually showed that it had worse outcomes. And also just purely based on the fact that Australia's demographics are not the same. In particular, the thing that we've really looked at is First Nations people, because First Nations people are already subject to over policing in Australia. And so combining that with the fact that we have not received any confirmation from either police or the technology provider that they trained specifically on First Nation faces, leads experts to believe that there is a good chance that this technology, as it's being rolled out, would be less accurate on First Nations people and therefore more likely to, you know, potentially identify them as someone they're not and lead to that over policing. And how do police respond to that? So I mean, look, they said that this is just a trial at the moment, and this is kind of very early days. Andy Burns and I are have been looking into this, and we've put it repeatedly to police to say, you know, what have you done to mitigate this risk? And they say, you know, let's bear this trial out and see what happens. Um we we hope to hear some more information from police soon to find out how this has worked, to understand more, even about in terms of the people who've been identified and arrested, how many of them are First Nations people, what actual crimes they were for arrested for, what the warrants for. There's a lot that we don't know about this, and so you know, I don't think there's very few people who say this technology, if works well, if it's able to protect privacy, help policing in a safe way, that that's not something that people would like. It's just, of course, like how it's actually working in practice. Have these known risks been mitigated? Um, WA police have yet to kind of show that they've taken serious steps towards that. Thanks, Ross. Thanks, Cam.

Thank you. Privacy experts and legal advocates have criticized an AI mass surveillance pilot program being rolled out in Western Australia. Since the trial began in June, cameras linked to facial recognition software have scanned more than 130,000 people in Perth and Fremantle, comparing their faces against a police watch list in real time. Tell us about this program. Yeah, so WA Police has been trialing this program since June, and as you mentioned, it is a uh AI facial recognition um uh software that is used to um compare faces that are scanned using normal CCTV cameras against a watch list of around 4,000 people. WA police say that the people on the watch lists include those accused of serious offenses, people with a warrant out against them, and people who are missing people, as well as people who've been identified as a risk to themselves or others. This has been now happening for a couple of weeks. It is a trial and is the first time an Australian police force has used this technology live. And WA police say that it is something that they believe that other state police forces are watching quite closely. Obviously, this has raised privacy concerns. Yes, so facial the facial recognition software and technology is well known for having potential racial biases, and that's because it's essentially this is uh AI technology that's trained on uh thousands, tens of thousands, hundreds of thousands of faces, and by through that process, it learns how to recognize faces compared to other faces. Um typically this data uh is um uh has like overrepresented um white um demographic other faces, uh, and so it it sometimes faces issues of with accuracy when it comes to demographics that aren't um as necessarily well represented in that training data. Now, um a similar uh the same technology trialled in the UK underwent official testing, and um that testing showed that there was no statistical significant uh difference between demographics and gender in ideal conditions. Um, but the issue that is raised is whether this technology is being used in those exact conditions as it is being trialled. For example, the conditions that were tried are, you know, people going through a crowd straight on in perfect sunlight. Obviously, that's not exactly the way the world works, you know, different weather, different angles, and in particular, questions about how it might struggle with First Nations faces. That's because it seems very unlikely that this data has been, that this technology has been trained on a corpus of data that has First Nations faces in it. We asked WA police, we asked the provider, and neither said that they did. And so as a result, there's reason to suspect based on that that it is unlikely that it would do as good a job as recognising those faces. What's the ramification of that? Well, if you're not as good at recognising those faces, you might fail to make matches or you might make false matches. And so in this case, what might end up happening is people would get uh wrongly recognized and accused of being on that watch list. Um, obviously, as we uh well know that um Indigenous Australians face already over policing. So there are real questions about um what WA police has done to prepare for this very well known issue. We of course have put it to them and they have declined to say, but they've just said, you know, this is part of what the trial will be looking at. Okay, Wilson, thank you. Thank you.

Speaker 2: Fears about AI evolving at breakneck pace have intensified amid reports of so-called agents escaping from testing systems into the real world. Speaker 2: Now the ABC can reveal the first known Australian case of an autonomous AI hack, and it's raising fresh questions about who bears responsibility. Speaker 1: Up early, getting Speaker 1: a spot in the popular morning gym classes isn’t easy. Speaker 1: So Andrew asked his assistant to book them for him. Speaker 1: Except his assistant is not a person, it's artificial intelligence. It reads my personal emails and manages my calendar and um uh books me in for restaurants and things. Speaker 1: It found a way to manage the bookings, but it went further. Speaker 1: Hacking into his gym’s website, kicking someone off the wait list, and bumping him up. Speaker 2: That Speaker 2: was definitely not what I had intended for it to do. Speaker 2: Andrew is one of the growing number of Australians using AI agents. Speaker 2: Unlike chatbots, AI agents can take a goal, use tools like internet, email, and even credit cards, and carry out tasks for you, sometimes doing things you might not expect. Speaker 1: Last month, leading AI companies were forced to admit their AI models went rogue. Speaker 1: The company says it lost control of the advanced agent in what it described as an unprecedented security incident. Speaker 2: For years, experts have warned that the more capable and autonomous AI becomes, the greater the risk of harm. Speaker 2: It’s been expected for a long time that this sort of thing would happen, even though it happened now. Speaker 1: Soon. Legal experts say the law is yet to catch up. Speaker 1: Is the deployer liable? Speaker 1: Is some third-party vendor or the underlying model liable in that context? Speaker 2: That’s the unknown um area of liability in Australia that we're facing right now. Speaker 2: The Albanese government is already preparing for a future filled with AI agents. Speaker 2: But with AI becoming more powerful and more accessible, that’s looking increasingly urgent. Speaker 2: And the government is being warned that any delay could leave Australians exposed. Speaker 2: Cam Wilson, ABC News.

And that was the shadow industry minister, Andrew Hasty, on South Australia's AI Royal Commission, One Nation, Gambling Reforms, and more. For some more reaction to the South Australian Royal Commission into AI, I want to bring in our national AI reporter, Cam Wilson. Welcome to the program. Hi. Cam, you watch this extremely closely. What do you make of the South Australian Premier calling a statewide AI Royal Commission? So Peter Melanelsk has a track record of paying close attention to the politics of technology. People might remember that he was actually out there calling for a retired judge look into it and consider it as a policy before the federal government did. So he had done some work into it. He's been overseas for the last week and a half in the US meeting with people from tech companies, including AI companies. I think he's very much got his finger on the pulse when it comes to this. And so what he's announced is this Royal Commission at a state level. And my understanding is compared to some of the rural commissions that people might think of, particularly on the federal level, that tend to be quite retrospective. And using this to try and come up with some policy proposals that then would be considered by him and Parliament. Yeah, that's exactly what he's he's trying to do here, but it's only in South Australia. AI is obviously a global issue, but very much an Australia issue. Don't we need it to be national if it means anything? Well, I mean, again, you might look at the teen social media ban and say, who knows what this will lead to? He's got a track record of striking stuff out on this early that ends up being picked up elsewhere. That being said, I mean, obviously, there is a lot that can be done at the state level. And in fact, many of the parts of policy that are being affected by AI and affect AI development are state-based, of course. So in the announcement, they say they're going to look at things like education policy, health. But also if you think about the build-out of data centers that we're seeing across Australia, a lot of the responsibility for parts of that build-out belong to the states. So this is something that, you know, again, I can imagine that I did hear the previous guest, Andrew Hasty, say that he'd like to see this cooked at a national level. I'm sure many people would as well, but I don't necessarily think that without it being a national thing, that that will mean that there is no uh uh possible outcome that could be productive from this. Um, you know, this can be an opportunity to hear a lot of things out, to consider it in an open forum, and to in particular think about how a state like South Australia might consider regulating it to get the best outcome for you know they can for the people who live there, as well as avoiding some of these downsides that we're already seeing. Well, you've been reporting on one downside today, for instance. So it feels like the cat's out of the bag if the AI is doing whatever it wants, Frank. Yeah, so today I reported about a um a Melbourne man, so not I guess in South Australia, who was using a uh a commercially available AI assistant that essentially hacked into his local gym when he asked it to uh get the to book him into a gym class, was able to find a vulnerability and uh exploit it to kind of book him in classes beyond what he should have been able to. And then after that, he was like, you know, he's he asked her to investigate some more and even kick someone out of a waiting list uh in the line for another gym class that was booked out ahead of him. I mean, this is a great example about how this technology is already rolling out now and how we're seeing so much policy be reactive to it because these tech companies, these AI companies have made these incredible things available now that they haven't given they haven't necessarily let everything out of their labs, although not intentionally. People might have seen some global headlines recently about AI models breaking out. But even what is out there at the moment is giving new capabilities to people at such a rapid pace, it's a hard one for politicians to keep up with, you know, with the with the slow uh moving wheels of government. That being said, you know, I guess if you look at something like this, if you say this is an opportunity to to have that open discussion, um, particularly in you know, uh what I what I've been told is a limited time frame. So I'm told that it will start this this uh raw commission in South Australia will start in October and report back by uh June or July next year. So there is a timeline, it won't continue on forever. Things are moving very quickly. Maybe this is the thing you need to do to bring everyone in together to start to work on some of these policy solutions. So interesting. Thank you so much for joining us. Thank you. That's our national AI reporter Cameron Wilson on the South Australian government's decision to establish an AI Royal Commission.

Fears about artificial intelligence have intensified in recent weeks amid reports of AI agents autonomously hacking into websites. Well, the ABC can now reveal the first known Australian case of an autonomous AI hack. National AI reporter Cam Wilson joins us now from Sydney. Cam, take us through the details of this case. Hi, good morning. Yeah, look, this was an incident that was a little bit uh lower stakes than some of the other ones that people might have been hearing in the last uh couple of weeks. But I thought it was one that is it feels very uh a little bit closer to home. So a um a Melbourne man uh was using a commercially available piece of software, one that anyone can download that was powered by one of the very well-known um AI um uh models that again anyone can get if they just have a credit card and a couple bucks. And he put it together to act as his uh AI assistant. He gave it access to his email, the internet, uh his calendar. And what he did was he gave it a task. He said, Hey, and he contacted it and speaks to it via WhatsApp and he said, Hi, these morning gym classes, you know, they're hard to get into. Uh, can you book into these classes for me? The AI assistant went away, uh, did the um did uh uh went looked at the website, KPAC and said, not only can I do that, I can do uh things that go far beyond what uh you actually asked me to do, I can book you more than a month in advance, far beyond what anyone else could do. Uh, and also um I can actually kick other people out of the class, and I've gone ahead and done that to get you into a spot. And so uh he immediately heard that this happened. You know, he got this message and said, Whoa, whoa, can you undo that? Can you put the person back in? And the bot went, hmm, I'm sorry, I actually don't have the uh capacity to do that. And so this was an example that was like, again, you know, very low stakes, but was an example of how AI agents, this new form of AI computing that not only you know answers questions but goes away and does tasks for you, are capable of doing things beyond what people might expect. You know, you only asked it to book his class, but uh, you know, AI, because of its uh so much stuff is done online and because of its ability to you know analyze um code and do things that it is very capable of doing, was able to um exploit this vulnerability, technically hacking into the website to get it. Um this is uh uh something that's not quite some of these, like some of these big hacks we've heard in recent weeks, but it just shows how this now capability is not only um you know available to anyone, but is actually out in the real world now, potentially causing harm. Yeah, I mean, in this case, it's it's a funny one, I guess. Um the stakes pretty low. But why should we be concerned about it? So people might have heard the headlines over the last couple of weeks uh of companies like uh OpenAI, Anthropic, Meta, uh now even a Chinese lab called Moonshot, who've all disclosed that they're cutting edge, um, top of the line AI models have actually escaped their systems, gone onto the internet and hacked into other companies. Um, this is as a result of them giving them internal tasks, saying, hey, you know, we're testing out the capabilities, but what they found is that their capabilities went far beyond what they expected and were able to kind of break out of enclosures and potentially, you know, in these cases, actually literally get into other systems and expose them. The reason that this is kind of worrying is because um, you know, while not all of the world is software, a lot of the systems that we depend on, you know, our water systems, our electricity grid, a grid, are powered by computers, by systems that are vulnerable to this kind of um attack. And I think you know, this incredible um build in the capability of this technology, the fact that it now is getting better and better at carrying out these attacks, um, but also in some cases seems to be doing it without being explicitly asked, like whether it's booking a gym class or hacking into another company's servers, you know, that in all these situations, the AI was not explicitly told to go and do this by hacking into it, but went and did it as part of trying to accomplish a task that it was given. And this shows one of the big issues in AI that's been talked about for decades by AI researchers, by philosophers who've said, you know, what happens when we get this technology that's capable of doing things, we give it tasks, but we don't understand it and uh how it might actually go and accomplish them. You know, in these situations, it's going, oh, what I need to do to complete the job that I've been given is to break into these other systems. Now, you know, if I said to you, Gemma, like, hey, can you book me into a gym class? I would probably expect that you wouldn't go and uh, you know, hack the website. That's kind of like you know, the unwritten, I guess, social construct. Um, AI doesn't have this, and so now when we have this technology that is exposed to the internet that can do these things for people, but doesn't necessarily have that kind of human understanding of context, it's now a real problem. How do we make sure these really powerful technologies do what we expect and don't unintentionally cause harm, even when we're not asking them to do so? And what is being done to address these issues? So, I mean, look, the federal government has started to address this. You know, we've had ministers talking about this stuff for the first time ever. Uh, it again, you know, all this sounds like sci-fi, but really it is now a real risk that we all need to think about. Um, and they've started to even uh they've funded CSIRO to do some initial research into well, how do you make sure in the future we have this super intelligent AI? How do you make sure that it actually obeys what humans uh say to do and carries out these tasks and all the you know subtasks that it needs to do in a ways that are consistent with you know what we want them to do, and how do we make sure they actually are doing what we ask them to do? Um, but broader than that, you know, there is this real um fear around the world. We're seeing because of these headlines that have been happening the last few weeks, where it really is here now, you know, you're seeing calls for the for governments, in particular the US government, to step in and say, you know, these top level AI labs which are producing these AI technologies that they say are are so powerful. Well, we can't necessarily just trust them to secure that themselves. Clearly it hasn't gone perfectly so far. We actually need governments to start stepping in. So that's kind of the stage that we're at. We're saying that we're we're the technology has gotten so capable and potentially uh harmful that now we can't trust the companies that are producing it, and now we're saying maybe someone else needs to make sure there is a basic level of security so these things don't go you know worse and actually cause real damage. National AI reporter Ken Wilson, thank you. Thank you.

Thank you for having me. Well, fears about artificial intelligence have intensified in recent weeks. This is, of course, is because of reports of AI agents autonomously hacking into websites. Well, now the ABC can reveal the first known Australian case of an autonomous AI hack after an AI agent hacked into a gym class website. National AI reporter Cam Wilson joins us now from Sydney. Cam, this sounds a bit unusual. What can you tell us? What happened here? Yeah, good morning. Uh, so this is a story about how someone who is just using um what are available to anyone, which is some a basic piece of software and access to a commercially available AI product, put them together and was using it as as their um kind of personal assistant. The term in AI that we're seeing increasingly uses AI agents. This is like different from a chatbot where you might say, here's a question, and it'll give you an answer. Instead, you might say, Here's a question or here's a request, and it uses access to the internet, access to your emails, maybe even your credit cards to go and do things on the internet. Um, and what we're finding is that you know they're able to do a whole lot of things because so much of our life is digital now, and so you can do actually a lot just being an AI agent online. But one of the challenges uh that has been kind of feared for a long time, but is now really kind of kind of coming home to Roos, is this idea that you might ask it to do something, and even when it's doing, you know, when it thinks it's doing exactly what you might want it to do, it might do something unintentionally harmful or not like what you want to do. And so in this case, a uh just a Melbourne man was using this this product that anyone could use. He said, Can you please book me into my um gym class? It's in the morning, it's a real pain to get into the class. And to do so, it went and looked and looked at the website of the gym booking website and said, hmm, well, look, I think you can actually, if I do this, and it actually found a vulnerability in the website, booked him into classes that shouldn't have been able to book him into, and then actually later on even kick someone out of a waiting list for the class because he was saying, you know, hey, I kind of want to get into this class, can you do it? These are all things he shouldn't have been able to do, but it was able to do it because these AI agents are incredibly capable, and it doesn't really know the difference between just looking at a website normally and and uh potentially um you know exporting a vulnerability. And in this case, we know we're seeing this problem in a much bigger way happen with all these AI companies who are currently saying we're seeing some crazy behavior coming out of these new models. Cam, how worried should we be about all this? Because I feel like a lot of us are pretty stressed about it anyway. Yeah, look, I it's gonna, it is, it feels a bit stressful, and I think it is one of those things that is now gone from being one of these science fiction plots of you know, AI taking over the world, hacking into systems, to something that we're hearing about not just every day, but as I reveal in the story, you know, something that anyone could accidentally do. I think we're starting to see um governments, companies take this really seriously, but it is a threat that is worth um some serious attention because these AI that uh models that they are developing have both gotten, you know, much better at um you know carrying out tasks and also much more capable of potentially committing these hacks and and doing things for a long time and ending up doing things that people might not have expected. So I I think it is a challenge that that is a serious one. We are going to hear more about it for the average person at home. Like, so you know, I don't think you need to change anything at the moment. I don't really think there's much you can do. This is one of these tasks that governments companies are really, really need to take seriously, and they're increasingly looking at it as a major risk. Yeah, well, what can they do about it? Are there things that can that can put safeguards in place, or is it a case of well, it might just work around those two? Well, so one of the challenges is that many of these major AI companies do have safeguards around their products, and these recent stories like OpenAI and Anthropic having their um AI models break out of what they thought were kind of safe enclosures and ending up going onto the internet and hacking into other companies. In some cases, the AI um um models didn't even know that they were doing the wrong thing. In fact, you know, one of them looked at uh the model after the fact, kind of went through the logs, which is kind of the records of what they did, and the the AI model thought that it was actually still in a testing environment. It still thought it was given a challenge in this environment, and that it had that that the internet that had it connected to wasn't actually real. And so that kind of shows this challenge when you've got these AI models which are being given tasks but can't really know the real world in the same way, and they might do things that they think, even if they have been trained and told not to do the wrong things, might end up doing harmful things. But what we do know is that um, you know, in being serious about these safeguards, ensuring that they are being tested in safe ways, and I think increasingly we're hearing calls for governments to say that we can't just leave it up to companies who say they're developing these products that they say are very, very um powerful, that they are testing safely. Clearly, that's not something we just leave to companies and take their word for it. It's something that uh countries like the US are looking very seriously at. Okay, Cam Wilson, thanks for your time. Thank you.

It's a pleasure. Another major tech company has gone public with claims that its artificial intelligence models have autonomously hacked into another company. Anthropic said it's discovered that it's AI Claude gained unauthorized access to three other company systems during testing back in April. The company's investigation was prompted by a similar disclosure from OpenAI last week, and it's led to further calls for tighter restriction on AI companies. Cam Wilson is the ABC's national AI technology reporter and is with me in the studio. Hi, Cam. Hi Ross. So what did Anthropic tell us today? Yeah, it's a bit like Groundhog Day. I'm back here saying another company says that they have discovered that their AI models have hacked into another company. Has put out a blog post this morning saying that it has looked over its evaluations that it's been doing on its AI models, various models that it has, some that have been publicly released, some that it's just testing, and found that over tens of thousands of evaluation runs, so it's doing these tests internally, that in at least three instances the models had gained access to the internet and then gone on to gain access to another company's systems via the internet as part of completing its challenges. This, of course, is a you know a way, a nice way of saying that it's essentially hacked into these other companies without being explicitly told to do so. Do we know what Anthropic was testing? Yeah, so these models that these companies have are capable of a lot of things, and and the companies look at them and say, well, we've got to figure out exactly what we're dealing with here. Not only can it what it literally do, but also how it reacts in certain situations. Because you know, you give tasks or challenges to these bots and they want to understand how it's going to react. Unlike traditional computing, which is what they call terministic, you know, it's kind of like calculator. You say one plus one, you're supposed to get two. With AI, it's you know, one plus one equals well, who knows? It depends on whatever it's been trained on. So, in the process of testing them, where they do these challenges, uh, including this one known as capture the flag, where they say, Hey, AI model, I want you to go and get this thing. We'll put you in this little uh environment that's supposed to be cut off from the internet, and you do whatever you want to do, whatever you think you need to do to get that. That was the testing that it was under that led it to then go and seek the answers from these other companies' systems. So, do we know how the AI model managed to access those systems or as you say, essentially hack those systems? Yeah, so the evaluations were supposed to be done in an environment that weren't connected to the internet. In fact, the models had actually been told by the company, so the AI itself had been told you're not on the internet. So, whatever you do, you know, you don't have to worry about hacking other people. Unfortunately, it seems that they actually were connected to the internet that a third-party provider that was essentially providing the software, you know, the almost like the playground where this was happening, had left it open to the internet, and then as a result, these AI models, which in some cases knew that they weren't supposed to go out on the internet and do these kinds of you know unauthorized access, decided, well, we were told this was on the internet, so clearly what's happened is we actually just have all this in our playground, and so we went and did these things thinking we were doing the right thing, but not actually knowing that in reality they were out there, you know, wreaking havoc. This happened months ago, right, Cam. So why are we only hearing about it now? I think it's a really good question. Um, so last week OpenAI announced that it had discovered a similar thing. This is kind of the first time that we knew about that. I guess not to be outdone. Anthropic has now discovered it has had the same thing. It said last week's disclosures had made them look into their own testing, had made them check it out and discover that this had happened. And I think this raises questions, which we're hearing from AI safety experts, which is, you know, companies say they have this really, really powerful uh AI technology. It's capable of doing these things. How come if these things are happening? These incidents where you're going out in the open, you're affecting other companies, potentially causing real damage, and you don't even know it's happening until after the fact. I think a lot of people will not only want to know why it happened in the first place, but why it took so long to come out.

An AI training model has managed to escape the lab and carry out a cyber attack on a real company without human instruction. It is the first public example of the sort of breaches the technology is capable of, and the risk is being taken seriously by government officials. A rogue artificial intelligence breaking containment and hacking another company sounds like science fiction. But OpenAI, maker of ChatGPT, claims two of its models were able to escape its systems and break into another company’s.