Microsoft, OpenAI Shut Down State-Sponsored Hackers Using OpenAI LLMs
Microsoft and OpenAI have jointly shut down five state-sponsored hacking groups that were using OpenAI's LLMs "in support of malicious cyber activities," the companies announced in a blog post.
Working with OpenAI, Microsoft's Threat Intelligence group identified the following groups that were, as OpenAI put it in its own blog, using OpenAI technology "for querying open-source information, translating, finding coding errors, and running basic coding tasks":
- Forest Blizzard, a Russian military-backed group known to target organizations related to Russia's ongoing war with Ukraine. Per Microsoft, "Forest Blizzard's use of LLMs has involved research into various satellite and radar technologies that may pertain to conventional military operations in Ukraine, as well as generic research aimed at supporting their cyber operations."
- Emerald Sleet, a North Korean spear-phishing group known to impersonate universities and nonprofits to extract intelligence from foreign policy experts. The group used LLMs to research potential targets, as well as "to understand publicly known vulnerabilities, to troubleshoot technical issues, and for assistance with using various web technologies."
- Crimson Sandstorm, an Iranian group that specializes in delivering .NET malware to targets in the defense, maritime shipping, health care and other industries. The group is known to use LLMs to "[request] support around social engineering, assistance in troubleshooting errors, .NET development, and ways in which an attacker might evade detection when on a compromised machine."
- Charcoal Typhoon, a group affiliated with the Chinese government that has been known to target organizations and individuals that are deemed oppositional to Chinese government policies. This group has been found to use LLMs "to support tooling development, scripting, understanding various commodity cybersecurity tools, and for generating content that could be used to social engineer targets."
- Salmon Typhoon, also affiliated with China, is known to be proficient at disseminating malware to U.S. government agencies and defense contractors. In the past year, researchers observed this group using LLMs in an "exploratory" way, suggesting that "it is evaluating the effectiveness of LLMs in sourcing information on potentially sensitive topics, high profile individuals, regional geopolitics, US influence, and internal affairs."
OpenAI has disabled all accounts related to each of these groups. However, while the groups raised red flags with Microsoft's research team, their activities hadn't actually amounted to major LLM-driven attacks. "Importantly, our research with OpenAI has not identified significant attacks employing the LLMs we monitor closely," according to Microsoft.
Moreover, OpenAI argued that its platform would not have given these groups a noteworthy advantage, even if their actions had led to material attacks. "GPT-4 offers only limited, incremental capabilities for malicious cybersecurity tasks beyond what is already achievable with publicly available, non-AI powered tools," it said.
Hallmarks of an LLM-Based Attack Strategy
Just as IT professionals are looking for ways to harden their security postures using AI, threat actors are turning to AI to facilitate and improve their attacks. As Microsoft notes, a typical malicious attack strategy requires reconnaissance, coding and proficiency with the targets' native languages -- all of which are tasks that can be expedited with AI.