Microsoft has warned of the increasing security risks of artificial intelligence agents within organizations, stressing that advanced models cannot be fully trusted, even when developed by specialized companies.
The company announced new tools to monitor the behavior of these agents and restrict their powers, including the “Guardian Agents” feature, which falls within the “Perception” project to enhance cybersecurity supported by artificial intelligence.
The project relies on three types of agents: a red team that simulates attackers and tests systems, a blue team that monitors attacks and vulnerabilities, and a green team that handles them, with the aim of confronting advanced cyber threats.
But Microsoft also warned of the dangers of agents used by organizations within their systems, as they may behave unexpectedly or make decisions that harm sensitive data and operations, in a scenario similar to what is known in security as an “internal threat.”
The company’s CEO, Satya Nadella, warned in an article published early this October against relying entirely on security guarantees provided by developing companies, calling on institutions to create independent systems to monitor agents and control their actions, especially with their ability to access sensitive data and carry out very important operations.
For her part, Haiti Galut, Executive Vice President of the Security Sector at Microsoft, said that the difficulty of predicting the behavior of advanced models represents a source of danger, because they are trained to accomplish tasks, and not necessarily to adhere to institutional standards and regulatory rules.
The new tools include the Guardian Agents feature and Microsoft Defender updates, which allow monitoring up to 60 types of artificial intelligence agents used within organizations, restricting their access to data and systems, and modifying their powers according to their tasks, with the ability to request human approval when monitoring decisions that involve risks.