The question to ask an AI agent vendor — what happens before it sends

In short: an AI agent that can email your customers is a system with write access to your reputation. The safety of one is not decided by which model it uses, but by four boring mechanics: whether outbound messages stop for approval, whether spending is capped per run, what happens when the account runs dry, and whether a stalled decision blocks the next run. Ask a vendor those four questions. A demo answers none of them.

The demo is not the product

Agent demos are all built the same way. A goal is typed, a plan appears, tools are called, something useful happens. It is genuinely impressive and it tells you almost nothing, because the run you are watching is the run that worked.

The runs that decide whether you keep the agent are the other ones. The agent that misreads a refund policy and offers one it should not. The agent that loops on a failing API call at four in the morning. The agent that decides the right move is to email two thousand people.

None of those are model failures exactly — a better model has fewer of them, never none. They are questions about what the surrounding system does when the model is wrong, and that is a design decision the vendor made long before you saw the demo.

Four mechanics worth more than the model

Does anything stop before it sends? The meaningful line is not "does it have guardrails" — everyone says yes — but whether an outbound message is *held*, as a state, before it reaches a person. In ProxyAI, a run that wants to send a message, an email or a call stops and is marked HELD. You get an email, the run appears in a Needs You queue, and the step timeline shows each proposed write: the recipient, the subject, the start of the body. Approve sends everything in the run; reject discards it.

That is a deliberately blunt instrument. Approval is all or nothing — you cannot approve three messages of five. Fine-grained approval sounds better and is worse: it turns a decision you make in one glance into a checklist you skim, and skimming is how the bad one gets through.

Is spending capped per run, not per month? A monthly budget is a bill you find out about later. A per-run cap stops the individual run that has gone wrong while it is going wrong. ProxyAI's default is $2.00 per run, and a run that reaches it stops.

What happens when the balance is empty? This is the question nobody asks and everybody finds out. The honest answer should be a refusal, not a partial action. Compiles are refused, runs fail with no_credits, and a customer who invokes a task from chat is told it is unavailable — rather than receiving half of something.

Does a pending decision block the next run? An agent that keeps starting scheduled runs while three approvals sit unread is manufacturing a queue you will never clear. Held runs wait seven days and then expire, and the agent does not start new scheduled runs while a decision is pending. That is less convenient and it is the correct trade.

The exception that makes it usable

A system where everything stops for approval is one nobody switches on. The exception has to be principled rather than a convenience toggle, and the principle is destination.

Two destinations never hold: a report to your own inbox, and a spreadsheet you picked yourself. Neither reaches a customer, so neither can embarrass you. That single carve-out is what lets a reporting agent run unattended from the first day while a customer-facing one stays supervised — without a setting that quietly turns supervision off everywhere.

What "human in the loop" should mean

The phrase has been worn smooth. Used carelessly it means a person exists somewhere near the system. Used precisely it means there is a state the work sits in, visible, with the action not yet taken, and a person's decision is what moves it.

The test is simple: ask where you would look, right now, to see everything an agent has proposed and not yet done. If the answer is a log you would have to read after the fact, the loop is not there — you have an audit trail, which is a different and much later thing.

Start with the boring agent

The strongest argument for agents is not the most autonomous one. It is a daily summary of what happened in your store, landing in your inbox, costing a fraction of a cent, touching no customer. It is genuinely useful, and it is how you learn what the thing gets right before you let it speak to anyone.

Then move one step out: an agent that drafts replies and holds them. You read the drafts for a week. If they are good you approve them quickly; if they are not you have lost nothing.

The vendors most worth trusting are the ones whose documentation makes that progression easy to find — including the part where it says what the agent cannot do. A capability list with no limits section is a sales page.

FAQ

What does "human in the loop" mean for an AI agent?

Used precisely, it means the agent's proposed action sits in a visible state — not yet taken — and a person's decision is what moves it forward. Used loosely it means only that a person exists somewhere near the system. The test is whether you can see, right now, everything the agent has proposed and not yet done. If the only record is a log you read afterwards, that is an audit trail, not a loop.

Can an AI agent send emails to customers without approval?

That depends entirely on the product, which is why it is worth asking directly. In ProxyAI, a run that wants to message, email or call a customer stops and is marked HELD until you approve it. The exceptions are destinations that cannot embarrass you — a report to your own inbox, and a spreadsheet you chose yourself — so reporting agents run unattended while customer-facing ones stay supervised.

What stops an AI agent spending too much on one task?

A per-run ceiling, as distinct from a monthly budget. A monthly cap is a bill you discover later; a per-run cap stops the individual run while it is going wrong. ProxyAI's default is $2.00 per run, and a run that reaches it stops rather than continuing.

What happens if an agent is waiting for approval and never gets it?

Held runs expire after seven days, and the agent does not begin new scheduled runs while a decision is pending. That is deliberately less convenient than letting it carry on: an agent that keeps queueing work while approvals go unread builds a backlog nobody ever clears.

Where should I start with AI agents?

With an agent that touches no customer — a daily summary of what happened in your store, sent to your own inbox. It is genuinely useful, it costs a fraction of a cent, and it is how you learn what the agent gets right before you let it speak to anyone.

*Agent Tasks is ProxyAI's agent add-on. The Agent Tasks guide covers approvals, run costs, and the limits, and the rates page has the model prices behind them.*