A tool call limit is the cap on how many functions an in-app model may invoke for one user action. The limit covers count, which tools, and what arguments are allowed. Without it, a chat box can loop, spend the day’s budget, or call a write endpoint the screen never offered.
The model proposes a call. Your app decides whether to run it. That split is the whole control. If the model can reach your HTTP client directly, you do not have a limit. You have a hope.
What to bound
Bound every tool the product exposes, including read tools. A search that returns another tenant’s rows is a data leak even when nothing was written.
- Maximum calls per user turn. Two or three is plenty for a support reply. One is enough for a lookup.
- An allowlist of tool names for that screen. A billing page does not need the delete account tool.
- Argument checks: the account id must match the session, amounts must sit inside a range, and dates must parse.
- A time and token budget for the whole turn, so a retry loop cannot hide inside one user click.
- Writes require a confirm step the model cannot skip by phrasing the request differently.
Count attempts, not only successes. A tool that fails and retries ten times still costs money. The cap should trip on the attempt, then stop the turn with a plain error the user can retry on purpose.
Design choices that matter
Execute tools on your server, not in the client prompt. The model returns a name and arguments. Your code validates, runs, and feeds back a short result. Send back the fields the screen needs, not the raw upstream payload.
Return a typed error when a call is refused, and stop the loop. Do not let the model try another way against a denied write. A second phrasing of the same refund is still the same refund. Put an idempotency key on write tools so a retry cannot create a second charge.
Log actor, tool name, argument hash, decision, and cost. Keep raw customer text out of that log unless you already have a retention rule for it.
Common mistakes
- Giving the model every internal API because the demo needed them once.
- Counting only successful calls, so failures and retries blow the budget.
- Trusting the model to pass the current user’s id instead of reading it from the session.
- Letting a tool result include secrets that go back into the next prompt.
- No timeout, so a hung downstream call holds the worker until the process is killed.
Safe rollout
Ship tools one at a time. Start with reads. Add a write only after the confirm step and the argument checks have tests. Use two users in those tests: the owner, and a stranger whose id must be rejected.
Alert on turns that hit the call cap. That spike is either a loop or a user who needs a different screen, not a reason to raise the cap in a hurry. If a tool leaves the product, remove it from the allowlist in the same change.
What to tell partners
If you expose an assistant to partners, publish the tool list and the cap per turn. Say that unknown tools are ignored, and that writes stay pending until a user confirms. Partners should build to that contract, not to a sample transcript.
Tool call limits keep an AI feature inside the product you actually shipped. Name the tools, count the calls, and refuse the rest in your code.
When you buy or sell an app that lets a model take actions, ask for the tool allowlist and the cap from the last release. A chatbot with unbounded writes is a support queue, not an asset. A written limit should travel with the handover notes.