Australia’s Gradient Institute has flagged six distinct failure risks in a new report on governed LLM-based multi-agent AI systems.
The report is backed by the Department of Industry, Science and Resources, and the 2024 study highlights issues like coordination failures and groupthink when agents operate collaboratively.
Why Traditional AI Risk Frameworks Fall Short in a Multi-Agent World
“The deployment of LLM-based multi-agent systems represents a fundamental shift in how organisations need to approach AI risk and governance,” said Dr Tiberio Caetano, Chief Scientist at Gradient Institute.
Titled Risk Analysis Tools for Governed LLM-based Multi-Agent Systems, the report draws attention to a fast-growing area of AI adoption: systems where multiple agents collaborate within an organisation.
While single-agent AI assistants are increasingly deployed across business functions, multi-agent systems introduce new complexities. Use cases include HR bots interacting with IT agents for onboarding, or multiple customer support agents handling segmented requests.
The study identifies six new risks:
- Inconsistent performance from a single agent disrupting workflows
- Cascading communication breakdowns
- Shared blind spots due to uniform model design
- Groupthink effects where agents reinforce each other’s mistakes
- Coordination failure stemming from limited mutual awareness
- Competing objectives undermining collective goals
“A collection of safe agents does not make a safe collection of agents,” the report notes.
Risk Mitigation Tools for Multi-Agent AI Systems
The Gradient report presents a structured risk management toolkit. It recommends organisations test multi-agent systems progressively—beginning with controlled simulations, moving through sandbox environments, and culminating in monitored pilot deployments.
Other tools include red teaming exercises, time-based interaction mapping, and guidance on how to measure agent reliability when traditional testing methods fall short.
“Our report provides a toolkit for organisations to identify and assess key risks that emerge when multiple AI agents work together,” said Dr Alistair Reid, lead author and Head of Research Engineering at Gradient.
Operational Risks in Critical Infrastructure and Services
Dr Caetano warns that multi-agent failures are not merely theoretical. “If failures occur in these settings, the consequences could disrupt essential services for millions of people due to the scale and criticality of these operations.”
Industries such as finance, healthcare, utilities, and government are cited as particularly vulnerable due to their reliance on seamless digital coordination.
Responsible AI Deployment for Australian Enterprises
Gradient’s CEO, Bill Simpson-Young, says the research comes at a crucial moment for Australia’s AI sector.
“Australian businesses are accelerating their AI adoption… The path forward isn’t about avoiding this technology; it’s about deploying it responsibly with awareness of both its potential and its pitfalls.”
The report applies to AI agents deployed within a single organisation’s governance, where the setup, training, and coordination of agents can be fully managed.
Enterprise adoption of multi-agent AI is accelerating, and Gradient Institute’s latest work offers a framework for pre-deployment risk analysis. The study reinforces the value of building safety and alignment into AI systems from the start.