Agent configs often pin “latest” just as early apps pinned floating dependencies. A silent model bump can change tool behavior, eval scores, and cost. Pinning model versions in agent config, like dependency locks, makes upgrades deliberate. You can still move forward without shipping mystery diffs.
Treat Model IDs Like Locked Packages
Record exact model names and versions in the same files you review for code. If prompts live in the repo, model pins belong beside them. Diff prompt packs in PR review the same way; see diff prompt changes in PR review like code diffs.
Floating aliases hide provider default changes. Locks make the change visible in git history.
Separate Planner and Executor Pins
Planner prompts and executor tool belts often need different models. Pin each role independently so a cheap executor bump does not rewrite planning style. That separation is covered in separate planner prompts from executor tool belts now.
Ship Eval Fixtures With Every Pin Change
A model bump without fixtures is a deployment based on hope. Attach a small eval pack that fails loudly on regressions. Ship fixtures beside skill packs so reviewers see expected behavior; see ship eval fixtures beside every new agent skill pack.
Include at least one overnight failure fixture from production-shaped runs when you raise pins.
Upgrade Through a PR Checklist
Require: old pin, new pin, cost delta estimate, eval summary, and rollback pin. Reject PRs that only change “latest” to another alias. Human gates still matter for high-impact write tools after a bump; see require human gates on high-impact agent write tools.
Watch Latency and Cost After Merge
Pins change spend even when quality looks fine. Track tool-call costs per turn for a few days after each bump. Plain metrics catch surprises early; see log tool-call costs per agent turn in plain metrics.
Keep a Rollback Pin in Config Comments
Keep the previous good pin in a comment or companion field. Rollback should be a one-line revert, not an archaeology session during an outage.
Add a CI check that fails when agent config references aliases such as latest or newest. Broken builds teach faster than style guides.
Document Pin Owners Per Service
Every pin needs a named owner in the config map. When evals fail after a bump, you want a person, not a channel guess. Owners also decide when a pin is allowed to float during a controlled canary.
Publish a one-page pin policy: how long a canary runs, what metrics block promotion, and who can force a rollback without waiting for a full meeting.
Run the practice once this week, write what broke, and adjust one variable only on the next pass.
FAQ
What if the vendor only offers aliases?
Pin the alias and record the resolved version from API metadata on each deploy.
Do local agents need the same locks?
Yes. Local drift becomes production drift when someone copies a profile.
How often should we bump?
On a schedule or when evals prove a win, not because a blog post announced a model.
Can different services use different pins?
Yes. Document which service owns which pin in the config map.
Should prompts and models change in the same PR?
Prefer separate PRs so you can attribute regressions.
What belongs in the lock file?
Model id, optional revision, provider, and the date you verified evals.