Bad AI customer agent bots are a growing brand risk
A recent incident involving Air Canada’s chatbot highlights the risks of poorly implemented AI customer service bots. When a grieving air traveler asked about bereavement fares, the bot invented a refund policy that did not exist, leading to a court case and significant brand damage. This incident serves as a cautionary tale for brands scaling AI in customer communications.
New research from Sinch reveals that 74% of enterprises have already been forced to roll back a deployed AI agent due to governance failures. Surprisingly, companies with the most mature guardrails, who invested heavily in compliance, safety protocols, and oversight, rolled back at an even higher rate of 81%. This suggests that the teams doing the most to prevent failure are actually failing more often, not less.
The impact of the “guardrail tax” is significant, with engineering teams spending most of their time building and maintaining safety systems instead of focusing on improving the customer experience. For marketing teams, this means that every hour spent on safety infrastructure is an hour not spent on customer experience improvements that drive revenue.
The study found that 62% of enterprises already have AI communications agents in production, and 88% expect to deploy one within 12 months. However, 74% have been forced to roll back a deployed agent due to governance failures, and 84% of teams spend at least half their engineering time rebuilding safety infrastructure from scratch.
When an AI agent fails, 35% of the impact lands on the support queue, and nearly as much (34%) lands on brand perception, which is harder to repair. Infrastructure quality was found to be the single strongest predictor of deployment success, outweighing model choice, team size, and budget.
To mitigate these risks, marketers can take three practical steps:
1. Let infrastructure drive your vendor decision: Evaluate providers based on their guardrail engineering, cross-channel orchestration, and the extent to which your team will absorb the safety burden.
2. Plan for the guardrail tax in your roadmap: Budget for the ongoing engineering resources required for safety systems, rather than watching your timeline slip when rollbacks hit.
3. Push for a separate governance function: Keep AI use cases and governance engineering separate, and partner with a dedicated governance function that handles trust, compliance, and security, freeing marketing to focus on work that directly touches customers.
By taking these steps, marketers can reduce the risks associated with AI customer agent bots and ensure a more successful deployment that drives customer experience improvements and revenue growth.