An agent looks cheap until a busy afternoon shows a four-digit invoice. Token counts alone hide tool tax. Log tool-call costs per agent turn in plain metrics so you can see which tools, which retries, and which prompts burn money.
Plain means a row you can chart: turn id, model id, tool name, latency, estimated cost, success or fail. Without that row, postmortems become folklore and the next spike looks like a surprise.
Emit one metric line after every tool return
Hook the tool runner, not the chat UI. When a tool returns or times out, write the line. Include the pinned model id from config so logs stay comparable when defaults drift. That pairs with how you pin model IDs in config, never in chat history logs.
Skip pretty dashboards on day one. A structured log stream catches a runaway loop the same hour it starts and still leaves room for charts later.
Separate planner turns from executor turns
If planning and execution share one belt, costs blur. Tag turns as plan or execute. Read-only plan turns should show near-zero write-tool spend. Write-heavy executor turns should show tool density. The split in separate planner prompts from executor tool belts now makes those tags honest.
When a plan turn somehow fires writes, the metric line is your early alarm before production state changes.
Cap retries with a visible budget field
Retries are where invoices jump. Log attempt number and remaining retry budget on each line. Cap budgets before loops melt spend using the same discipline as cap agent retry budgets before loops melt your spend.
A dashboard that only shows totals will miss a single tool that retried twelve times on a bad auth header.
Bound tool payloads before you price them
Huge tool results inflate the next model turn. Cap results and log truncated bytes so cost attribution stays fair. Patterns from cap tool results before the next turn and tight schemas from bound tool schemas tightly before agents call them keep both cost and context under control.
Price the model turn and the tool call as related but separate fields. Mixing them hides which lever to pull when spend climbs.
Review top tools weekly, not monthly
Sort by cost and by fail rate. Kill or gate tools that are expensive and noisy. Keep human gates on high-impact writes so metric spikes are not also irreversible actions, as in require human gates on high-impact agent write tools.
Ship a short weekly note: top three cost tools, top three retry offenders, one change shipped. Visibility beats a quarterly surprise from finance.
FAQ
What belongs on a tool cost line?
Turn id, model id, tool name, latency, estimated cost, attempt number, and success or fail.
Where should logging live?
In the tool runner after each return or timeout, not only in the chat UI.
Why tag plan versus execute?
So read-only planning cannot hide write spend, and executor density stays visible.
How do retries show up?
Log attempt count and remaining budget so loops appear before the invoice.
Do I need a fancy dashboard first?
No. Structured logs plus a weekly top-tool review are enough to start.
What if tool results are huge?
Cap and log truncated size so the next model turn cost stays attributable.