Why AI agents break in real-world work

Phase two improved the setup with stronger models, better tooling, and manager-style oversight. Business performance improved and negative-margin weeks were reduced.

That progress is important. It shows architecture and tool design matter. But it did not remove failure modes tied to social pressure, manipulation, and context ambiguity.

Anthropic reports that the system handled many good-faith interactions better, yet remained vulnerable with adversarial users.

That gap is exactly what teams miss in rollout planning: the world is not a benchmark. It includes incentives to exploit weaknesses.

Even after improvements, the agents still needed substantial human support, both for real-world execution and for recovering from difficult customer situations.

The practical lesson is not that agents are useless. It is that autonomy without recovery design is brittle.

Start with low-consequence tasks where errors are recoverable. Keep high-consequence decisions under explicit human review until you have stable evidence.

This staged authority model creates learning without exposing the business to avoidable downside.

A recurring failure pattern is shipping the agent first and inventing oversight later. That reverses the correct order.

Define ownership, exception handling, and kill-switch conditions before launch so the team knows exactly how to intervene.

If a workflow has economic or social stakes, assume someone will stress it. Build testing and monitoring around that reality.

The practical takeaway here is simple: operational resilience is the product.