Skip to content

06. Versioning prompts and models

Rollback works because there is a previous version to flip to. Versioning is the discipline that makes prompts and models first-class versioned artefacts — like code, but with their own lifecycle. The discipline parallels module 13 (prompt lifecycle) and chapters of 02_ai_infrastructure/01 (model gateway); this chapter is the release-management view.

A platform engineer at a Chennai SaaS company audits the AI artefacts after a year of operation. Prompts are in a registry; each has a version; each version has a changelog. Models are pinned per alias; the routing policy is versioned; migrations follow the canary discipline. When something needs rollback, the previous version is identifiable, available, and reachable. Before the discipline: prompts were embedded in code; model versions were buried in deploys; rollback required source archaeology. The before/after is the value of versioning.

This chapter is the versioning discipline as it relates to release management.

What the versioning makes possible

Three release-management capabilities:

Identifiable previous version. "What did the prompt look like before?" is answerable in seconds, not archaeology.

Available previous version. The prior version is in the registry, retained for the rollback window, deployable on demand.

Fast pointer flip. Rollback is a route weight or alias mapping change — minutes, not deploys.

Without versioning, none of the three are reliable; release management collapses to deploy management.

The prompt registry as the substrate

Per chapter 05 of 14_legacy_ai_modernization and module 13. The registry holds:

  • Each prompt as an artefact (YAML/JSON file or service entry)
  • Each version with name, version, owner, changelog, eval reference
  • Historical versions retained for the rollback window
  • A "current production" pointer per prompt

The release management interacts with the registry:

  • Pre-release: the new version exists in the registry; the eval is run against it.
  • Canary: the routing reads from the registry; the new version is at a percentage; the old at the rest.
  • Rollback: the new version's percentage drops to 0; the old continues.
  • Post-promotion: the new version is at 100%; the old remains in the registry for some weeks.

The model routing policy as the substrate

Per 02_ai_infrastructure/01 chapter 03 and chapter 09. The routing policy holds:

  • Alias-to-version mappings with weights for canary
  • Pinned concrete versions (never "latest")
  • Historical mappings (via the policy's version control)

Release management interacts with the routing policy:

  • Pre-release: the new pinned version is added as a candidate with weight 0; the eval is run.
  • Canary: the new version's weight is shifted up by canary step.
  • Rollback: the new version's weight drops to 0; the previous remains.
  • Promotion: the new version is at 100%; the previous remains as a fallback at weight 0.

Retention of previous versions

The previous version must be available for rollback. Retention:

  • Active rollback window. The previous version is retained for at least the rollback window (typically 30-90 days after promotion).
  • Historical retention. Older versions are retained per audit or reproducibility needs (often years).
  • Retired versions. When a version has not been used or rolled back in the active window, it can be archived.

For models specifically, the provider may retire the version (chapter 09 of 02_ai_infrastructure/01); the platform's rollback target may disappear. The discipline: track provider deprecation; rollback target is always a still-supported model.

What the versioning does not solve

  • Eval-set staleness. The version is preserved; the eval against which it was promoted may have aged. Re-running the previous version against current eval is the verification.
  • External state changes. A prompt that worked previously may not work the same now if upstream data has changed.
  • Cross-version interactions. A prompt change that interacted with a contemporaneous model change; rolling back one without the other may produce unexpected behaviour.

The discipline is to retain the version and recognise these limits.

What the version captures

Beyond the prompt text or model identifier:

  • The parameters. Temperature, max_output_tokens, stop sequences. These are part of the artefact.
  • The eval reference. Which version of the eval set the prompt was tested against.
  • The metadata. Owner, created_at, promoted_at, changelog.
  • The dependencies. Which model alias the prompt is paired with; which tool versions it expects.

A complete version is the artefact plus its operational context.

Common mistakes

Prompts in code. Versioning is via git only; rollback is via deploy; speed is bounded by deploy time.

Hardcoded model versions. No alias indirection; rollback is per-call-site code change.

Previous version retired too soon. Rollback target is gone; rollback becomes "ship a new version that matches the old."

Versioning the artefact but not the parameters. Rollback restores the text but the parameters changed; behaviour is different.

No retention discipline. Versions accumulate or are removed unpredictably; rollback is unreliable.

Interview Q&A

Q1. Why is versioning prompts in a registry better than versioning them in git? Git versions are file-history; rollback is a deploy. The registry decouples the version from the deploy: a rollback is a routing flip, not a redeploy. The registry supports per-version metadata (eval reference, owner, parameters) that git alone does not. The registry supports per-route assignment (different prompts per feature) without code change. Git is the substrate for many registries; the registry is the discipline. Wrong-answer notes: "git is enough" misses the deploy-time-to-rollback gap.

Q2. The previous version of a model has been retired by the provider. The team needs to rollback. What do you do? Roll forward instead. The previous version is unavailable; rolling back is impossible. Roll forward with the next-best model (often the next-newest version) as the canary's replacement; the canary may shift to a third candidate while investigating. The discipline is to track provider deprecation and ensure the rollback target is always a still-supported model — verify this as part of the release planning. Wrong-answer notes: "redeploy the old model" is impossible if the provider has retired it.

Q3. The team is rolling back a prompt change from three weeks ago. What is in the registry? The current version (the one being rolled back from). The previous version (the rollback target). Each has its own metadata: prompt text, parameters, eval reference, owner, changelog. The routing policy currently points to the current version (at 100%); the rollback shifts the weight to the previous version. The previous version is still in the registry, retained for the rollback window. The version's eval reference may be different from current; re-baselining if eval has evolved. Wrong-answer notes: "the previous version is in git" misses the route flip vs deploy difference.

Q4. Walk through what gets captured per version. The artefact (prompt text or model identifier). The parameters (temperature, max_output_tokens, etc.). The eval reference (which version of the eval set this was tested against). The metadata (owner, created_at, promoted_at, changelog). The dependencies (model alias, tool versions). The version is the complete operational context, not just the text. Without parameters and dependencies, the version is incomplete; behaviour cannot be reliably reproduced. Wrong-answer notes: "just the text" produces incomplete versions that don't reproduce behaviour.

What to do differently after reading this

  • Versioned prompts in a registry; versioned models in routing policy.
  • Each version captures the full operational context.
  • Retain previous versions for the rollback window; archive afterwards.
  • Track provider deprecation; the rollback target must be available.
  • Rollback is a route or alias flip, not a deploy.

Bridge. Versioning is the artefact substrate. Communication is the human discipline that wraps each release. The next chapter is the communication that keeps stakeholders informed before, during, and after. → 07-release-communication.md