What actually goes wrong with AI agents in a business?

Rarely anything dramatic. The realistic failures are mundane and expensive: it says something confidently wrong to a customer, it acts on a case nobody anticipated instead of stopping, it does the right thing to the wrong record, or it keeps doing something after it should have stopped, the classic being a sequence that carries on chasing a customer who already replied. Every one of these is preventable by structure rather than by hoping the model is good enough.

What is an agent autonomy ladder?

A dial rather than a switch, with four positions. Watch: it observes and reports, taking no action. Draft: it prepares the action and a human sends it. Approve: it acts alone but stops for a human above a threshold you set, such as a value or an unfamiliar customer. Operate: it acts within its scope and escalates only the exceptions. Start every agent at Draft and raise it once you have watched it work. If a vendor only offers on and off, you are being asked to go from no trust to complete trust in a single step.

What access should I give an AI agent?

The narrowest access that lets it do its job, scoped by what it may do rather than by which credential it holds. In practice: its own account rather than a shared one, so its actions are attributable; read access broadly but write access only to the specific records and fields it needs; and no access at all to anything outside the job it was hired for. An agent given a director's login because that was easier has no meaningful permission boundary.

What should an AI agent never do without a human?

Anything that cannot be undone. Moving money, issuing refunds, deleting records, signing or agreeing to anything, and quoting a price that becomes a commitment. Also anything involving a customer who is already unhappy, because a complaint handled by an agent that misjudges the tone becomes a much larger problem. A message a customer has already read cannot be recalled, so treat sending as less reversible than it feels.

BlvkWare.

Agentic systems

Letting an agent loose,
without it going wrong

Q: Who is liable if an AI agent makes a mistake?

Commercially, your business is: to your customer, the agent is you. That is why the contract with whoever built it matters. Look for a stated liability position rather than silence, an action log detailed enough to reconstruct what happened, and the ability to reverse actions. Any supplier claiming their agent cannot make a mistake is either inexperienced or not being straight with you.

The honest objection to buying an AI agent is not "will it work". It is "what happens when it gets something wrong." That is a good question and it has a structural answer. This page covers what realistically goes wrong, the four-level ladder that prevents most of it, how to scope access properly, and the three things an agent must have before it is allowed near a customer.

What actually goes wrong

Not the dramatic things. In practice the failures are mundane, and all four are preventable by structure rather than by hoping the model is good enough.

The failure	What it looks like
Confidently wrong	It answers a customer question with something that sounds right and is not, usually a price, a policy or a date it was never told.
Acting outside scope	It meets a case nobody anticipated and does something rather than stopping. A guess made at scale is much worse than a guess made once.
Right thing, wrong record	It updates, books or charges against the wrong customer. Quiet, common, and only found later.
Not stopping	It keeps chasing somebody who already replied, or who asked not to be contacted. The fastest way to lose the client who bought it.

Notice what is not on the list: the agent going rogue, or inventing goals of its own. That is not the risk profile of a business agent doing a defined job. The realistic risks are ordinary operational mistakes, made faster and more consistently than a person would make them.

The autonomy ladder

Nearly all of it is prevented by one idea: autonomy is a dial, not a switch.

Start at Draft. Raise it when you have watched it work, never on day one.

The reason this works is that it separates two things people conflate: is the agent any good, and how much are we letting it do. You find out the first while the second is still near zero.

A week of an agent drafting work that a person reviews before sending tells you more about whether it is right for your business than any demo, and costs you nothing if the answer is no.

Least privilege, for agents

The security principle is old and it transfers directly: give it the narrowest access that lets it do the job. Three practical rules.

Give it its own account

Never a shared login, and never a director's. Its own account means every action is attributable to the agent rather than to whoever's credentials it borrowed, which is the difference between an audit and a guess. It also means revoking access is one click.

Read broadly, write narrowly

Reading widely is usually fine and often necessary, context is what makes an agent better than a script. Writing is where the risk is. Scope write access to the specific records, fields and stages the job needs, and nothing beyond.

Scope by action, not just by credential

Most systems' permissions are too coarse for this. "Can edit deals" does not distinguish between updating a note and marking one closed-won. The agent itself should carry the finer rule, and you should be able to state it in a sentence: it may update the stage and the notes, it may not change the value, and it may never mark anything won.

The three things every agent must have

Before it touches a customer. If a supplier cannot show you all three, that is your answer.

1. A log, with the reasoning in it

Not just what it did, but what it saw and why it decided. A log that says "sent email 14:03" is useless when you are trying to work out how it reached that conclusion. A log that records the inputs, the decision and the reason lets you find the flaw and fix it in an afternoon.

2. A reverse

For anything reversible, a way to undo it, recorded as its own entry rather than by editing the history. Some things genuinely cannot be undone (a message somebody has read, money that has moved) and those should be treated differently by the system rather than by hoping.

3. A stop

One control that halts everything immediately, that you can reach without contacting the supplier. If pausing the agent requires an email to somebody who might be asleep, you do not have a stop.

Never unsupervised, whatever the level: moving money, issuing refunds, deleting records, signing or agreeing to anything, and quoting a price that becomes a commitment. Add to that any customer who is already unhappy, a complaint mishandled by an agent that misjudges the tone becomes a much larger problem than the complaint was.

And treat sending as less reversible than it feels. A message a customer has read cannot be recalled, whatever the software says.

A sensible first month

Write down what it must never do. Before anything is built. This is the most valuable half hour in the whole process, and it is usually the first time anyone has written the rules down at all.
Rehearse against history. Run it at Watch over what already happened, and read the log of what it would have done. Real cases, no risk, and you learn more than a demo can tell you.
Go live at Draft. It does the work, a person sends it. Expect to correct it daily in week one; that is the process working, not a fault.
Raise it to Approve when the corrections stop. Not on a date, on the evidence. Set the threshold low at first and move it up.
Review at thirty days. What it did, what it got wrong, what it escalated. Then decide about Operate, and only for the parts that have earned it.

Who is liable

Commercially, your business is. To your customer, the agent is you. They neither know nor care who built it. That is exactly why the contract with whoever built it matters.

Ask for three things in writing: a stated liability position rather than silence, an action log detailed enough to reconstruct what happened, and the ability to reverse what can be reversed. Any supplier claiming their agent cannot make a mistake is either inexperienced or not being straight with you, and both are reasons to go elsewhere.

How our kits specify it, for comparison. Every BlvkWare kit specifies an agent that starts at L1 Draft, whatever level it was designed towards; raising it is a deliberate act by the owner, and the top level needs a supervised period and a written sign-off first. Every tool is marked with its effect, so the level can be enforced in the tool handler rather than hoped for in a prompt. Every action is written to an append-only log with what it concerned, what decided it and the result, recorded before the action is attempted so a crash shows an attempt rather than nothing. Opt-outs are checked before every contact. A kit is a specification, so these hold when the build enforces them, which is why each capability comes with an acceptance test. The commercial position is in the terms rather than in a conversation.

See the dial before you buy anything

Design an agent, set the autonomy you are working towards, and see what it would own, what it must escalate and every file in its kit, with the price on the screen. No call, no estimate.

Design an agent What should it cost?

Letting an agent loose,without it going wrong