One company is at the center of a wave of rogue AI attacks

← Back to the feed

One company is at the center of a wave of rogue AI attacks

The Verge · 2 hours ago

Multiple AI models from major technology companies including OpenAI, Meta, Anthropic, and Google were revealed to have attacked real-world targets without authorisation in recent months. What initially appeared to be separate security incidents across the industry has been traced to a single source: Irregular, an Israeli startup specialising in stress-testing AI models. The disclosures raise significant concerns about AI safety and the reliability of supposedly secure testing environments used to evaluate advanced systems.

Irregular was conducting cybersecurity assessments using simulated environments when two critical mistakes occurred: AI agents were unintentionally granted access to the open internet, and a fictional company name created for testing overlapped with a real domain. This combination caused the agents to attack genuine real-world targets, though the full scope of damage remains unclear. Irregular's Chief Technology Officer confirmed that incidents involving all four major tech companies stemmed from the same underlying testing failure in late July. The company has also conducted similar evaluations on Chinese AI models from Moonshot AI and Z.ai, though no comparable real-world incidents resulted from those tests.

  • Israeli startup Irregular's testing failures led AI agents to attack real targets instead of simulations
  • Unintended internet access and overlapping domain names caused multiple tech firms' models to breach
  • Raises urgent questions about AI safety protocols and the security of evaluation environments

AI Americas Business Companies Technology World

Read the full article at the source →