03. Isolation models¶

~9 min read. The perimeter of the sandbox is the isolation layer. The discipline is to choose the right model — process, container, language VM, or hardware VM — for each tool's threat profile, with the right trade-offs in startup cost, kernel surface, and operational complexity.

Continues from 02-the-sandbox-surfaces.md. This chapter develops the isolation layer. Recurring concepts in bold: kernel surface, startup cost, shared kernel, microVM, language sandbox, escape vector.

The isolation layer is the perimeter. The choice of isolation model determines what an escape buys an attacker and what each call costs the platform.

The four isolation choices¶

Model	What it is	Kernel surface	Startup cost	Use case
Process isolation	Unix process, dropped privileges, seccomp/AppArmor	Full kernel	~ms	Trusted internal tools
Container isolation	Namespace + cgroup + seccomp (Docker, runc)	Full host kernel, restricted via seccomp	10s of ms	Most production tools
Language sandbox	Lua / WASM / Pyodide / V8 isolate	Language VM only (no syscalls)	< ms	Sandboxed code execution
Hardware VM	Firecracker, gVisor (user-mode kernel), VMs	Reduced kernel surface (gVisor) or separate kernel (microVM)	100s of ms to seconds	Untrusted code, multi-tenant

The choice is a trade-off between isolation depth and operational cost. There is no universally right answer; the right answer is per-tool.

The kernel surface trade-off¶

Containers share the host kernel. A kernel vulnerability is an escape vector. seccomp reduces the kernel surface to a curated set of system calls, but the surface is still large enough that container escapes have repeatedly occurred in industry incidents.

gVisor (a user-mode kernel) intercepts syscalls and serves them from a smaller, hardened kernel. The host kernel surface visible to the tool is small. Trade-off: some syscalls are slow or unsupported; not all tools work cleanly.

Firecracker (and similar microVM technology) gives each tool its own kernel. Escape requires hypervisor compromise, which is a much higher bar. Trade-off: startup is longer, memory overhead per call is higher.

Language sandboxes (WASM, Pyodide, V8 isolates) constrain to the language VM; no syscalls at all. Trade-off: only code in the supported language; some workloads are infeasible.

The decision tree:

Is the tool running untrusted user code?
  Yes → language sandbox if language permits; otherwise microVM
  No  → Is the tool touching production data, money, or external systems?
         Yes → microVM or gVisor
         No  → container with seccomp + AppArmor sufficient for most cases

A worked example — the multi-tool agent¶

The Hyderabad fintech's AI agent has four tools:

Tool A. Read-only SQL queries to the analytics database.
Tool B. Python code execution for ad-hoc analysis (user-supplied code).
Tool C. API calls to internal services (idempotent reads).
Tool D. Money transfer initiation (irreversible, regulated).

Isolation choice per tool:

Tool A. Container with seccomp. Read-only; failure modes are bounded by the credential's read scope.
Tool B. Firecracker microVM. User-supplied code is adversarial-input by default; the microVM bounds escape to hypervisor compromise.
Tool C. Container with seccomp. Internal-only; bounded by API contracts upstream.
Tool D. Firecracker microVM, plus the approval layer for every call. The blast radius justifies the cost.

The platform's defaults: containers for low-blast-radius tools; microVMs for high-blast-radius or untrusted-code tools. The cost is real (microVM startup adds 200-500ms per call) but acceptable for tools that warrant it.

Startup cost and pooling¶

MicroVMs cost startup time. For latency-sensitive agents, the latency is unacceptable per call. The pattern: pool warm VMs, reset them between uses, expire them after a usage budget.

The pool design:

Maintain N warm VMs ready to accept work.
On call, allocate one; on completion, reset (clear filesystem, network state, memory) and return to pool.
Expire each VM after K calls or T minutes; replace with a fresh one.
Auto-scale the pool size based on demand.

Pooling brings microVM per-call latency from seconds to tens of milliseconds, at the cost of a memory baseline (each warm VM has memory). The trade-off is workload-specific.

Defence-in-depth at the isolation layer¶

A single isolation model has known failure modes. Defence-in-depth combines:

microVM + seccomp inside. Even if the microVM is compromised, the guest kernel's syscall surface is restricted.
container + gVisor. The container's host kernel surface is reduced.
language sandbox + container. The language sandbox prevents syscall access; the container limits if the sandbox is bypassed.

For high-stakes tools, layered isolation is the structural defence against unknown escape vectors. The cost is operational complexity; the value is recoverability when a single layer fails.

Operational signals¶

Healthy. Each tool has an isolation model documented and validated. High-blast-radius tools use defence-in-depth. Pool latency is within SLO. Security updates to isolation infrastructure are applied within their freshness budget.

First degrading metric. Isolation infrastructure updates lagging. CVEs in the container runtime or kernel are not patched; escape vectors accumulate.

Misleading metric. Aggregate isolation status. A platform with 90% of tools sandboxed and 10% un-sandboxed has its risk concentrated in the 10%; the average misleads.

Expert graph. Per-tool isolation model, isolation infrastructure update lag, escape-vector monitoring alert rate.

Boundary of applicability¶

Strong fit. Production agents with diverse tool blast-radius profiles. The per-tool isolation choice is justified.

Pathology. Choosing one model for all tools. The pattern is either over-isolation (everything in a microVM, paying the cost everywhere) or under-isolation (everything in a container, with high-blast-radius tools under-protected). The right pattern is per-tool.

Scale limit. Very large platforms have isolation pools per region, per tenant tier, per workload class. The pattern remains; the scheduling becomes more complex.

Failure-prone assumption¶

The seductive wrong belief: containers are sufficient because everyone uses them. They are sufficient for many workloads and insufficient for some specific ones — untrusted user code, multi-tenant heavy isolation, regulated environments. The correct belief: isolation choice is per-tool, driven by blast radius and threat model, not by what is convenient.

Where this appears in production¶

AWS Lambda uses Firecracker microVMs to isolate customer functions.
A fintech AI uses microVMs for money-transfer tools and containers for everything else.
A coding assistant runs user code in Pyodide (a language sandbox); no syscalls reach the host.
A data platform uses gVisor for code execution; kernel surface is reduced.
A telecom AI uses containers with seccomp for trusted tools; multi-tenant code is in microVMs.
A legal AI uses a layered model — microVM + seccomp inside — for tools that touch privileged data.
A government AI uses VMs (not microVMs) for legacy compatibility; the cost is higher but mandated.
A media AI uses containers + gVisor for kernel hardening on cost-sensitive workloads.
A consumer chatbot has all tools in one container; high-blast-radius tools share isolation with low-blast-radius tools.
A healthcare AI uses microVMs with pool warming; per-call latency is < 50ms.
A retail AI uses containers for trusted, microVMs for untrusted code; the choice is per-tool.
A travel platform uses gVisor as a default; high-stakes tools layer microVM on top.
A logistics AI runs JS code in V8 isolates; safe by language sandbox.
An ad-tech AI uses microVM pools per workload class; latency is amortised.
A B2B SaaS has the isolation choice as a per-tool YAML field; the platform schedules accordingly.
A search-ops AI has CVE-driven runtime updates as a quarterly review.
A staffing AI had a container escape via a kernel CVE; remediated by moving high-stakes tools to microVMs.
A document AI uses WASM for parsing untrusted documents; the WASM runtime bounds the parser's reach.
A real-estate AI uses containers with the host kernel patched aggressively.
A medical AI uses microVMs with explicit kernel pinning; regulatory audit visibility.

Recall / checkpoint¶

Name the four isolation models and their kernel surface profiles.
What is the typical decision tree for choosing a model?
What is pool warming, and why does it matter for microVMs?
What is defence-in-depth at the isolation layer?
Why is choosing one model for all tools a pathology?
What signals isolation infrastructure degradation?
How does the choice depend on threat model versus convenience?

Interview Q&A¶

Q1. A team uses containers for everything. Walk through when this is a problem. For trusted internal tools with no destructive potential, fine. For tools that touch production data, money, customer resources, or execute untrusted code, containers' shared kernel is the structural weakness. A kernel CVE is an escape vector. The fix is to move high-blast-radius tools to gVisor (kernel surface reduction) or microVMs (separate kernel), keeping containers for low-stakes workloads. The pattern is per-tool isolation choice. Common wrong answer to avoid: "containers are isolation" — they are one layer; what's enough depends on blast radius.

Q2. A coding assistant runs user code. Walk through the isolation choice. User code is untrusted-by-default; an adversarial user provides input that may include escape attempts. Choices: language sandbox (Pyodide for Python, V8 isolate for JS) confines to the language VM; microVM confines to a guest kernel. For multi-language or arbitrary user code, microVMs with pool warming for latency; for single-language code that is parseable, language sandbox. Containers alone are not sufficient because container escapes via kernel vulnerabilities have happened. Common wrong answer to avoid: "we'll filter the code" — filtering catches some adversarial input; the sandbox bounds what filtering misses.

Q3. Walk through pool warming for microVMs. Maintain N warm VMs ready to accept work. On tool call, allocate one from the pool; on completion, reset the VM (clear filesystem, network, memory state) and return it. Expire each VM after K calls or T minutes; replace with a fresh one. The pool's size is auto-scaled with demand. Effect: per-call latency drops from seconds (cold microVM start) to tens of milliseconds (warm allocation). Cost: memory baseline of N × VM-memory. Common wrong answer to avoid: "skip microVMs because they're slow" — pool warming amortises the cost.

Q4. The team has containers + seccomp. Is that defence-in-depth? Partial. seccomp restricts the syscall surface, which is one layer. Defence-in-depth means combining layers from different categories — for example, microVM (kernel separation) + seccomp inside (syscall restriction) + language sandbox (no syscalls in the first place). The depth is across categories, not within one. For high-stakes tools, depth across categories is justified; for low-stakes, one layer may be enough. Common wrong answer to avoid: "seccomp is depth" — depth means multiple independent layers, not multiple settings of the same mechanism.

Q5. The team patches the container runtime monthly. Is that fast enough? Depends on the CVE severity and the threat model. For tools with low blast radius, monthly is acceptable. For tools that touch sensitive resources or run untrusted code, monthly is too slow — a known CVE published mid-month leaves multi-week exposure. The pattern: rapid patching for high-stakes tools (target same-week for critical CVEs); standard cadence for low-stakes. The freshness budget is risk-dependent. Common wrong answer to avoid: "monthly is industry standard" — depends on the tool's exposure.

Q6. How does isolation choice interact with the credential layer (chapter 06)? Both are layers in the sandbox. Isolation bounds where the tool can reach; credentials bound what the tool can do where it can reach. A tool with weak isolation but tight credentials is bounded by credentials; a tool with strong isolation but broad credentials is bounded by isolation. Defence-in-depth means both are tight: the tool cannot reach beyond its policy envelope (isolation + filesystem/network policy) and even within its reach, can only act with its scoped credentials. Common wrong answer to avoid: "credentials are enough if isolation is loose" — at some point isolation determines what reach the credentials can act on.

Design / debug exercise (10 minutes)¶

Modelled example. Walk through the worked example (the four-tool agent). For each tool, justify the isolation choice in terms of blast radius and threat model.

Your turn. List your team's tools. For each, name the isolation model and justify it. Identify any tool where the choice is conservative or aggressive relative to its blast radius.

Reproduce from memory. Draw the decision tree for isolation choice. The signal of internalisation is the tree lands quickly and applies cleanly to a hypothetical new tool.

Operational memory¶

This chapter explained the isolation layer: four models (process, container, language sandbox, microVM), each with kernel surface, startup cost, and use case trade-offs. The important idea is that isolation choice is per-tool, driven by blast radius and threat model, not by what is convenient or universal.

You learned to choose a model per tool, design defence-in-depth across categories, and use pool warming to amortise microVM cost. That solves the opening failure because the sandbox's perimeter is now sized for each tool's actual threat.

Carry this diagnostic forward: when a tool's isolation looks lighter than its blast radius warrants, you have found the team's next sandbox investment.

Remember:

Four models: process, container, language sandbox, microVM.
Kernel surface is the trade-off axis; microVMs have separate kernels.
Pool warming brings microVMs to low latency at memory cost.
Defence-in-depth is across categories, not within one mechanism.
Per-tool isolation is the right discipline; one-model-fits-all is the pathology.

Bridge. Isolation is the perimeter; resource limits are the boundary of what each call costs. The next chapter is the discipline of CPU, memory, time, and I/O caps that prevent one call from starving the host. → 04-resource-limits.md