Secret rotation is the planned replacement of a credential before it is stolen, copied into a log, or left in a repo that no longer needs it. Waiting for an incident means you rotate under a clock, with half the clients still holding the old value.
A rotation window is the overlap during which both the old secret and the new one are accepted. That overlap is the product. Without it, every consumer breaks at the same minute. With a sloppy window, the old secret never dies and the exercise was theater.
What rotation protects
API keys, signing keys, database passwords, and third-party tokens all age. They land in CI variables, laptop shells, crash dumps, and vendor dashboards. Rotation shrinks the time a leaked copy stays useful.
Protect these classes first:
- Keys that can move money, send mail, or read customer data.
- Signing keys used to mint session tokens or verify inbound events.
- CI tokens that can publish packages or open production shells.
- Shared break-glass credentials that more than two people can see.
Design choices that matter
Publish the window the same way you publish an API change. Operators and partners need dates, not a surprise 401.
- Issue the new secret before you revoke the old one. Dual-accept for a fixed period, often 24 to 72 hours for machine clients, shorter for human passwords.
- Stamp each secret with an ID. Logs and tokens should name which key signed or authenticated, so you can see stragglers before cutoff.
- Store secrets in a manager the app reads at startup and on a refresh signal. A redeploy should not be the only way to pick up a new value.
- Separate read of the secret from use of the secret. A worker that cached the old password in memory will miss the window unless you reload.
- Cut the old secret automatically at the published time. A manual we’ll-revoke-later step gets skipped.
Prefer two active key versions over a flag that disables verification. For signing keys, accept signatures from version N and N-1, and sign new tokens with N only.
Common mistakes
Rotation fails in boring ways.
- Rotating in the vendor console and forgetting the app config, so production starts failing health checks.
- Putting the new secret in chat or a ticket just for the deploy.
- One shared key for every environment, so a staging leak forces a production cut with no warning.
- No owner for the cutoff. The window opens and never closes.
- SDKs that pin the key at process start and never reload, so long-lived workers keep using the retired value after you think the cut is done.
- Rotating the signing key and the API key in the same minute. Split the changes so you can tell which one broke traffic.
Safe rollout
Write the runbook before the first scheduled rotation. Name who creates the new secret, who updates each consumer, who watches error rates, and who revokes.
Do a dry run against staging with a synthetic client that still sends the old value until cutoff, then a second client on the new value. Confirm both succeed during the window and only the new value succeeds after it.
On production, announce the window in the partner changelog. Watch auth failure rate, not just CPU. Keep the previous secret revocable in one action if you see unexpected use from a host you do not recognize. That is different from leaving it valid until you are sure.
What to tell operators
Document the header or field that carries the key ID, the date format you use for cutoff, and how a client should reload config without a full outage. Ask partners to support two active keys. A client that can hold only one value will skip the window and page you at cutoff.
Schedule rotation for credentials that touch customer data. A window you can explain in a sentence is a control. A secret that never changes is a standing exception.