New Ai Threats

The perceptin of threats in AI. Or the threats to the way we perceive AI

I am writing this posting in relation to the recent AI-Cyber campaign that happened in September although was recently released publicly by Claude. In which it was proposed that we have gotten to a point where cyber campaigns may become a future issue of grave importance to defend against as the technology has gotten to the point in can facilitate this. I don’t really by this. Though I would assume that they waited so long to release this to put more defenses in place as they are quite up front about the security measures they put in place ( insert thing about them telling the cops). I don’t believe a nation state actor of any country would be so lazy as to use a Claude agent to enact a cyber campaign as it would be easily traceable.

Why not for instance utilize their own hardware and work to produce or utilize their own exploits and malware that they have made. General security on the internet for many small to medium business is pretty terrible. Flock cameras for example have recently been exposed as being a giant trove of information just open for view with minimal security. Going so far as to deceive their buyers of these flaws with their marketing that doesn't hold true under scrutiny. But i digress...

Setting up guardrails around the research labs that create these models is enough to stop the un-initiated but not the resourceful. For anyone who wanted to attack a website of a company or the servers where they operate you can ( meaning having the ability to do something). NOT to say that you should do anything illegal. However, with the massive amount of information on the internet it would be a lie to say a script kiddie wouldn't have moderate success in penetrating a network. The barrier for learning how to do something online is only coming down as the vast amount of data online is continually being increased. 

Open source intelligence (OSINT) attacks are even more viable with the more people online and the more data that is made freely available online. Examples being where a person lives, associated phone numbers, email addresses, social media accounts, potentially even where they work and went to school depending on their digital footprint. "Youtube University" (INSERT SOME OSINT TOOL) will go over these forms of intelligence gathering in greater depths than I will here. Additionally, it can also be a repository of information to learn many IT, networking and security concepts online. 

Another resource that is available (albeit with some digging) are college/universities that freely share class information. Apart from the infamous Harvard CS class that is posted on youtube every year (POST A LINK TO THE CLASS) you can also look at schools websites themselves. For awhile now I have "attended" classes at universities like the University of Washington that only this year (2025) has made steps to hide their class information. What they didn't do however, was get rid of the old material which I am willing to guess is still the same for this year (INSERT LINK). Universities while moving away from this have and some still do with work arounds share and host lectures, homework, class notes, class lectures online and slides freely.

When we look at AI (llms and agentic models), while the guardrails in place by companies like Claude are somewhat effective in stopping some people others have been able to get around them.
  • send emails to “media and law-enforcement figures” with warnings about the potential wrongdoing.
  • alleged chinese threat actor got around guardrails by saying they worked at a cyber security company and only prompting specific parts of the code while not revealing their entire purpose
  • can just as easily ask another LLM to do what you want as the rules and implementation are not the same across the board and then return to using Claude

What confused me her is that this wasn’t done on selfhosted software and run on their own hardware. As well as utilizing their own qwen or any other open source agentic ai. For this type of attack what guardrails exist to stop a person from utilizing their knowledge. Or even the collective knowledge that the internet provides and enacting a cyber campaign of their own. Once the model has been developed the resource requirements are not the same to run the models as they are to train them. The levels of compute are drastically different. And someone of means could look at the bigger model variants and utilize them like what is available for llama, deepseek, qwen3, or gpt-oss.

Running the models on the machine is fairly trivial with programs like ollama and creating agents is made even easier with model context protocol (mcp). Utilizing n8n to create a workflow could automate the entire process to be something where information is dumped into the workflow and the actual reconnaissance, vulnerability discovery, exploitation, and exfiltration operations can be done autonomously. Like what was explained in the report from Anthropic.

  • **Why didn’t they run scripts and automate the attacks? - the exploits used were run-of-the-mill so wouldn’t defenses against this would be available already? - Using AI lets you hack more orgs at a time beyond that idk?? Unless the automation expedites the depth of attack and leaves them duped in less than minutes. - Marcus Hutchins (youtuber)

  • Was it just a test to see what is open available and possible by testing real world tools? Like a university looking at the viability of this attack type.

    • Professor Charles Harry

I think this begs the question whether the amount that companies spend on cybersecurity enough or even looking at the size of cybersecurity teams. While this is something cool that was done – This doesn’t look like a novel idea on its head to present something that couldn’t have been done through the use of scripts. The novel Ai part here using mcp could be used for expediting the depth within the network. And the speed that defensive teams have to react to the speed.

  • As we go forward in the wargaming and threat assessment landscape the availability of resources available only get cheaper as the technology gets better. The headlines will shock and scare the uninitiated in the dark but not those with a flashlight.