08. Honest Admission: What Security Still Cannot Promise¶

⏱️ Estimated time: 28 min | Level: advanced

ELI5 callback: In the apartment building, the front door checks identity, the elevator key limits movement, the wall keeps tenants apart, the audit records rule-following, and the safe protects valuables.

1) Start with humility, not slogans¶

Security engineering is about reducing risk, not manufacturing certainty.

See. If anyone says perfectly secure, start worrying immediately.

So what to do? Speak in threats, controls, residual risk, and response speed.

Good teams admit what they know, what they assume, and what they cannot yet defend well.

Now watch. Honest admission is not weakness. It is how serious systems stay adaptable.

The front door reminds you that identity checks can still be phished or bypassed.
The elevator key reminds you that policy bugs can still grant too much.
The wall reminds you that shared systems can still leak through overlooked paths.
The audit reminds you that evidence helps response, not invincibility.
The safe reminds you that encryption protects secrets, but not every runtime exposure.
Threat models are living tools, not one-time documents.
Residual risk should be named clearly to leaders and customers when needed.
Designing for recovery is as important as designing for prevention.
Uncertainty increases when dependencies, models, and vendors multiply.

2) Supply chain attacks changed the game¶

Modern systems depend on packages, base images, CI platforms, SaaS vendors, and cloud controls you did not write.

That dependency chain creates leverage for attackers because one upstream compromise can travel far downstream.

Simple, no? You can secure your code and still inherit someone else's bad day.

Software bills of materials help visibility, but visibility is not the same as safety.

Provenance, signing, and trusted build pipelines matter because trust now travels through delivery systems too.

Pin dependencies where practical and review update channels intentionally.
Treat CI credentials and artifact registries as highly privileged targets.
Use signed artifacts and verify provenance where the ecosystem supports it well.
Reduce build-time internet reach when reproducibility and trust matter.
Inventory vendors because unknown dependencies cannot be defended intelligently.

┌────────┐ package ┌────────┐ image ┌──────────┐ │ Source │───────────▶│ Build │──────────▶│ Runtime │ └────────┘ └────────┘ └──────────┘ │ │ │ └──── provenance, signing, attestations ────┘

3) AI-specific threats are still moving targets¶

AI systems introduce prompt injection, model extraction, training data leakage, and unsafe tool use.

See. Traditional web security still applies, but the interface and failure modes are weirder now.

A language model may follow malicious instructions hidden inside user content or retrieved documents.

Evaluation is difficult because safe behaviour is probabilistic, context-heavy, and sometimes easy to bypass creatively.

Now watch. Guardrails help, but complete guarantees remain elusive.

Separate model-facing content from trusted system instructions as much as architecture allows.
Constrain tool use with allow-lists, output checks, and human approval for high-risk actions.
Log prompts and tool calls carefully while respecting privacy constraints.
Assume indirect prompt injection can arrive through files, web pages, and connectors.
Treat model output as untrusted until downstream validators approve it.
Continuously red-team new model versions because behaviour shifts after updates.
AI security is partly application security and partly behavioural safety engineering.
That mixed nature means controls are less settled than classic login or TLS patterns.
Teams should say that openly.

4) Zero-days and attacker economics keep changing¶

Some vulnerabilities remain unknown to defenders until they are discovered or exploited. That is the zero-day problem.

You cannot patch what nobody knows exists. You can reduce blast radius and improve detection speed.

See. Attackers also choose targets economically. Cheap attacks that scale will keep appearing.

That means defenders must think beyond rare elegant exploits and address common boring weaknesses too.

So what to do? Design layers, observability, segmentation, and fast rollback around uncertainty.

Minimize privilege so one unknown bug has less reach.
Invest in incident response paths before the incident arrives.
Segment networks, tenants, and data domains so compromise does not spread freely.
Patch fast, but also know which systems you can isolate while patching.
Monitor unusual behaviour because signature-based thinking will always lag some attacks.
Practice restore and rollback so response is not theoretical.

5) The honest operating model¶

The mature stance is simple: controls reduce risk, evidence improves trust, and humility keeps learning alive.

Now watch. Teams that admit uncertainty usually improve faster than teams defending slogans.

Security roadmaps should include unknowns, experiments, and explicit assumptions.

Customers do not need panic. They need clarity about what is protected well and what is still evolving.

Simple, no? Honest communication is part of system design too.

Keep threat models current when architecture, vendors, or AI capabilities change.
Prioritize fixes by impact and exploitability, not only by vulnerability score.
Use layered controls so one surprise does not become total failure.
Teach teams to separate measurable controls from comforting language.
Review supplier and model risks as part of platform planning, not after procurement.
Make postmortems educational and blameless so learning compounds.

Humility is operational strength when systems grow complex.

Unknown dependencies create unknown blast radius.

Security maturity includes saying no to false confidence.

The honest answer in interviews is sometimes I would validate that assumption.

Residual risk should be written, not implied.

Fast rollback is a security control when the unknown appears.

AI features add new attack surfaces and new ambiguity.

Supply chains deserve the same threat attention as APIs.

Uncertainty should create discipline, not paralysis.

Perfect security is marketing language, not engineering language.

Where this lives in the wild¶

Platform reviews discussing dependency trust, CI integrity, and vendor exposure.
AI product teams evaluating prompt injection and tool misuse risk.
Executive security updates that need clear residual risk language.
Incident response planning for unknown vulnerabilities and fast rollback.
Architecture reviews that must balance speed, cost, and evolving threat models.

Pause and recall¶

Why is security better framed as risk reduction than certainty?
How do supply chain attacks bypass otherwise good application security?
Why are AI-specific threats harder to fully guarantee against today?
What design habits help when zero-days are unavoidable?

Interview Q&A¶

Q: Why is perfect security an unhealthy goal statement? A: Because real systems have unknowns, dependencies, and evolving threats, so the honest goal is reducing risk and improving response. Common wrong answer to avoid: "With enough effort, any serious platform can become perfectly secure."

Q: What makes supply chain attacks dangerous? A: They exploit trusted upstream components or build paths, letting one compromise spread across many downstream systems. Common wrong answer to avoid: "If our repository is private, supply chain risk is low."

Q: Why are AI threats still unsettled? A: Model behaviour is probabilistic, prompt channels are flexible, and tool integrations create new ways for malicious input to steer behaviour. Common wrong answer to avoid: "AI security is just old OWASP rules with new branding."

Q: How should teams handle unknown future vulnerabilities? A: Use layered controls, least privilege, segmentation, monitoring, and rehearsed incident response so unknown flaws have smaller blast radius. Common wrong answer to avoid: "You cannot do anything meaningful until the vulnerability is disclosed."

Apply now (5 min)¶

Write down one major security assumption in your current system, like trusted CI or trusted model output.

Then ask what fails if that assumption becomes false tomorrow.

List one preventive control, one detection control, and one recovery control for that case.

If recovery is blank, that is your next engineering task.

Bridge. Systems secured. Now let's build the data factory that feeds them. → ../11_data_platform_pipeline_design/00-eli5.md