13. Honest admission¶

Twelve chapters of discipline. None of them solve the problem entirely. This chapter lists the limits a thoughtful lead is transparent about — with their team, their stakeholders, and themselves — when telling the story of inheriting an AI system and making it operable.

The discipline this module taught is real. The audit, the eval backstop, the structural moves, the strangler pattern, the stakeholder cadence — together they turn a frozen inherited system into a modernising one within ninety days. They do not turn it into a new system, and they do not eliminate the problems the original team left behind. The honest admissions below are the gaps where modernisation is bounded and where a lead should expect to make judgment calls rather than apply rules.

1 — Behaviour the inherited system relied on may never be fully recoverable¶

The previous team made decisions, often undocumented, that shaped the system's behaviour. Some of those decisions were good; some were mistakes; some were workarounds for now-irrelevant constraints. Inherited code carries all three indistinguishably. The audit and the eval recover the behaviour that exists; they do not always recover the intent behind it. Some behaviours will turn out to have been workarounds whose removal is safe; others will turn out to have been load-bearing for reasons nobody remembers until something downstream breaks.

The mitigation is conservatism — preserve behaviour until you understand why it exists — but conservatism is sometimes wrong. A lead navigates this by treating every "is this behaviour intentional?" question as a real investigation, not a quick decision.

2 — The eval covers what you know about¶

The eval backstop catches regressions on the failure modes you identified and the happy path you sampled. It does not catch regressions on cases you did not include — long-tail failures that occur rarely in production, edge cases the team has never noticed, behaviours that emerged after a previous model version that the current version handled differently.

The eval grows over time as new cases are discovered, but at any moment, its coverage is partial. A lead who treats the eval as exhaustive will be surprised by the case it did not cover. The discipline is to keep growing the eval and to maintain humility about what it cannot yet catch.

3 — Strangler pattern depends on extractable boundaries¶

The strangler migration works when the system has, or can be refactored to have, clean boundaries. Some inherited systems are entangled in ways that resist boundary extraction — everything calls everything, side effects are scattered, state is global. The first move in those cases is introducing a boundary, which is its own multi-week effort before strangler proceeds. The "strangler is incremental" framing assumes you have something to incrementally strangle.

For genuinely entangled systems, modernisation is slower and may include a bigger structural refactor early, before strangler is usable. Be honest with stakeholders about the timeline.

4 — Some inherited systems should be retired, not modernised¶

The chapter-1 framework had four states (frozen, observable, eval-backed, modular). It implicitly assumed every system can move along that progression. Some systems should not — they were misconceived at the design level, their behaviour is unsalvageable, or the cost of modernisation exceeds the cost of building a replacement.

A lead's job, six to twelve weeks in, is to honestly assess whether the modernisation path is the right path. If the system's fundamental shape is wrong (a single agent doing four jobs poorly, an architecture that no eval can rescue), the modernisation is throwing good engineering after bad. The replacement conversation is hard but sometimes correct.

5 — Stakeholder management has limits¶

Chapter 10 taught the discipline. The discipline does not always win. Some PMs will not absorb "two months of structural work"; some executives will demand commitments the technical reality cannot defend; some customers will leave during modernisation. The lead can communicate well and still face stakeholder departures.

The honest stance: stakeholder management improves outcomes, but it does not guarantee them. If stakeholders override the engineering judgement and force features onto an unstable system, the lead's job is to surface the risk clearly and execute the override carefully. Sometimes the override is the right call; sometimes it is not; the lead does not always get to make the call.

6 — Modernising under deprecation pressure compresses everything¶

A model retiring in 45 days forces the migration's pace. The audit shrinks. The eval shrinks. The canary rolls faster than is comfortable. The structural work that should have happened first runs in parallel with the migration. The deprecation pressure is the most common reason a modernisation goes wrong in the first ninety days.

The mitigation is anticipation — track deprecation calendars in advance — but anticipation does not retroactively help an already-tight deadline. A lead in this situation has to make compromises and document them; the system after such a migration usually carries more risk than one migrated at sustainable pace.

7 — Data debt is the slowest to address¶

Chapter 9 covered the data side. Of all the modernisation tracks, this one is the longest. Cross-team ownership, source freshness SLAs, retrieval modernisation, training-data provenance — each is multi-month work involving teams you do not own. A lead who promises "data modernisation in 90 days" is overcommitting.

The honest framing: stabilise the data side enough to operate (audit context, capture sources, establish freshness signals) in the first 90 days; the deeper modernisation extends across quarters. Communicate this timeline early.

8 — Some inherited systems have been "modernised" before, badly¶

You inherit a system that the previous team also tried to modernise. There are half-built abstractions, abandoned eval directories, a partially-implemented prompt registry that was never connected, a strangler boundary that never finished. The state is worse than a system that was never touched, because the half-finished work is harder to remove than the original mess.

The mitigation is to evaluate the half-built work on its merits — is the abstraction useful even if incomplete? — and either complete it or remove it. The lead's job is to choose deliberately rather than carry the half-built artefacts forward as more dead weight.

9 — The team you inherit may not be the team you need¶

The engineers on the team when you arrive may have different skills than the modernisation requires. Some have been operating the legacy with deep knowledge; some are eager to ship features and resent the structural work; some are leaving and not invested. The modernisation depends on the team; the team's composition affects what is possible.

A lead's job includes the team change as much as the technical change — hiring, role realignment, sometimes departures. This is its own discipline (module 20_engineering_leadership_judgment covers it more deeply); the modernisation surfaces the need.

10 — The discipline is young¶

Compared to legacy-code modernisation (Feathers' book is from 2004; the patterns are older), legacy-AI modernisation is two to three years old as a discipline. The patterns in this module are real and worked, but the field has not converged. New tools — better eval frameworks, gateway products, prompt-registry SaaS — appear quarterly. Some of what this module recommends today will look quaint in two years.

The honest stance: hold the discipline confidently — the underlying reasoning is sound — and with humility — the specific shapes will evolve. A lead who modernises a system in 2026 will be modernising different systems with different tools in 2028, but the principles (audit, eval, observability, strangler, calendar discipline) will hold.

What this module does not teach¶

Listed once:

The technical details of evals (04_ai_product_evals covers golden sets, judges, calibration)
The internals of agent architecture (module 01)
Gateway operations (02_ai_infrastructure/01)
Incident response (module 05)
The team-leadership dimensions of being a new lead (module 20_engineering_leadership_judgment)
Specific provider migrations (vendor docs)

This module assumes those neighbours exist or are being learned in parallel. Without them, the modernisation has the shape but not the substance.

How to use this module after reading it¶

A realistic learning path:

If you are about to inherit a system, read chapters 01-03 in advance. The day-one audit and the eval backstop are urgent enough that you should be ready to start them in week one.
If you are modernising a system now, walk the chapter-12 checklist; identify reds and yellows; prioritise the reds.
If you are a lead who has been operating an inherited system for a while without doing the modernisation, the chapter-04 sequencing applies — start with one bleeding fix and the eval, and build from there.
Re-read chapter 13 every quarter. The limits surface again as you do more modernisations.

Closing¶

Inheriting an AI system is the most common applied-lead scenario in modern engineering and the one most under-served by training material. The greenfield modules build new things; this module turns the dial back to what you actually face on day one of a new role.

The discipline is real. The 90-day path from frozen to sustainable is achievable for moderate systems. The work is not glamorous; the result is a system you can operate and evolve.

That is what production-grade legacy modernisation looks like for AI systems.

Bridge. This module ends. The next module, 15_reasoning_routing_verification, is about a related but distinct discipline — picking the right reasoning effort and verification pattern per call. After modernising an inherited system, the questions of how to use models efficiently and how to verify their outputs become the new frontier. → ../15_reasoning_routing_verification/00-eli5.md