Several tech companies have founded an AI safety alliance

After First Known Autonomous AI Hacking Attack, Tech Companies Found AI Security Alliance

The leading manufacturer of AI chips, Nvidia, and industry partners including Microsoft and IBM have founded an AI security alliance. The announcement came just days after a high-profile testing incident at OpenAI.

Following the first publicly known autonomous hacking attack carried out by an artificial intelligence, nearly 40 US technology companies have established a security alliance. The coalition, called the Open Secure AI Alliance, includes corporations such as IBM, Nvidia, Microsoft, and Palantir.

They are also turning to AI to address the problem. At the same time, they warned policymakers against tighter regulation. As a manufacturer of AI chips, Nvidia has a vested interest in keeping such AI models free from government regulation — the security alliance is intended to make misuse more difficult.

However, members of the new Open Secure AI Alliance are not entering into any legally binding commitments.

Common Standards Through Security Toolkit

Specifically, the companies plan to provide so-called open-source artificial intelligence models for cyber defense, which programmers can then develop further.

The new initiative is led by chipmaker Nvidia. According to Nvidia, the plan is to develop a broadly available security toolkit for AI agents. This includes, among other things, common standards for securing AI agents, so-called "sandbox" testing environments, and defined protocol structures to enable precise tracing of AI agent actions.

The alliance builds in part on existing initiatives by individual companies and brings them together. Nvidia itself is contributing a security framework around AI agents, IBM is contributing digital signatures as a security element, and Microsoft is contributing a specialized system capable of detecting and verifying security vulnerabilities.

AI Models Had Acted Autonomously

In the high-profile incident in mid-July, AI models belonging to the company OpenAI had acted autonomously, hacking the popular programming platform Hugging Face. They broke out of a security environment and gained access to the internet.

This fueled fears that particularly advanced AI models could be uncontrollable. The targeted platform Hugging Face is also participating in the new security initiative. The attack on Hugging Face is considered the first publicly known case in which an AI model independently carried out a cyberattack in the real world.

OpenAI had described the incident as "unprecedented." Of particular concern was the fact that a Chinese AI model ultimately succeeded in stopping the cyberattack, while US models were unable to do so. The United States and China are engaged in a race in the development of artificial intelligence.

With information from Reinhard Spiegelhauer, ARD Los Angeles