Your AI Agent Is Chaos. Tame It.

This is Part 3 of Agentic AI for Offensive Security, the foundations track of this blog. It runs alongside The Agentic Red Team, the hands-on build series where these concepts turn into running code. Part 1 named the three animals. Part 2 explained why the dangerous one is dangerous. This one is about caging it.

Here is the problem every engineer hits the first time they ship an agent. You run it twice on the same input and you get two different answers. A support bot refunds one customer and argues with the next over identical tickets. A coding agent fixes the bug on Tuesday and invents a new one from the same prompt on Wednesday. The thing will not hold still, and you cannot ship something you cannot predict.

People keep asking me how I make a nondeterministic model behave deterministically. The honest answer is that I don't, because I can't, and neither can anyone selling you a tool that claims to. The model is a demon. You don't argue a demon into predictability. You build a room it can't leave.

That reframe is the whole article. You do not make the model deterministic. You make the system around it deterministic, and you shrink the nondeterministic surface down to only the parts that genuinely need judgment. Then you put hard bounds on those parts. Everything else is plumbing, and plumbing is something engineers have known how to make reliable for fifty years.

Why can't you just make the model deterministic?

Because nondeterminism is the feature you're paying for.

The same property that lets a research agent find a source nobody pointed it at, or lets an offensive agent find an admin panel nobody told it about, the one from Part 1, is the property that makes its output impossible to predict line by line. Set the temperature to zero and you reduce the variance, you do not remove it. Floating point math, batching on the inference side, tiny changes in the prompt, all of it leaks variance back in. Two runs of the "same" agent on the "same" task will diverge. Plan for it.

So the goal was never determinism in the model. The goal is a system whose behavior you can reason about even though one component inside it gambles every time it speaks. That is a normal engineering problem. We wrap unreliable things in reliable structure constantly. A network is unreliable; TCP is not. A disk fails; RAID does not. The model is your unreliable component. Your job is the TCP around it.

The determinism spectrum

Not every step an agent takes needs judgment. Most don't. The mistake I see in every junior agent build is treating the whole pipeline as one big creative act, when really it's a spectrum, and most of it should be nailed to the floor.

The mechanical steps sit at the near-deterministic end. Look up a record in a database. Pull a customer's order history. Resolve a domain and pull its DNS records. Parse a response into fields. There is exactly one correct way to do each of these, and a language model is the worst possible tool for them. You write these as plain code and call them like functions. The model never gets a vote on whether the lookup ran correctly.

Judgment sits at the irreducibly creative end. "Given what I just saw, what's the right next move?" For a support agent that's deciding whether this really is a refund case. For an offensive agent that's "what's worth attacking here, and how?" That is the demon's actual job. You cannot script it, because if you could script it you wouldn't need the agent. This is the narrow slice where nondeterminism earns its keep.

The irreversible actions sit at the human-gated end. This is the step with blast radius. Issuing the refund. Deleting the records. Sending a payload that could take down a service. Anything that touches a real system in a way you can't take back. The model can propose it. The model does not get to pull the trigger alone.

The skill is sorting every action your agent can take onto that line, then matching the control to the position. Deterministic steps get code. Creative steps get a bounded model call. High blast radius steps get a human. Most teams put the model in charge of all three and then act surprised.

How do you bound the creative part?

You constrain its inputs and you validate its outputs. The model gets to be creative inside a box, and you check everything that comes out of the box before anything downstream trusts it.

The first lever is structured output. Do not let the model return loose prose that you then pick apart with string matching and hope. Make it answer in a fixed shape you defined up front, a small set of named fields with strict types, and reject anything that does not fit that shape exactly. A free-text answer is a parsing nightmare, and in a security context it is an attack surface. A validated object is a contract.

In practice the shape is tight. You decide in advance which actions are even allowed and you give the model a closed list to pick from, so it cannot ask for an operation you never built a handler for. You cap the length of any free-text field, so it cannot smuggle a wall of text into a slot you expected to be one line. You force it to attach a confidence number you can threshold on. And you make it declare, up front, whether the action it is proposing needs a human to sign off. The creativity lives in the values it chooses. The structure it has to answer in does not move. Swap the field names and the same idea fits a support bot or a data pipeline; only the stakes of getting it wrong change.

What do you do when validation fails?

You hand the model its own mistake and ask again, and you cap how many times you will ask. This is the second lever, and it's where a lot of agent builds quietly fall apart.

When the model returns something off-contract, you don't crash and you don't paper over it. You feed it back its own broken answer and the exact reason it failed, and you ask it to try again. Bounded. Three attempts, then you stop and escalate to a person, because a model that can't produce a valid answer after three tries is a model that's confused, and a confused agent still taking actions is exactly the failure mode you're trying to prevent. No infinite loop. No silent garbage passed downstream. A loud, honest stop.

That retry loop is deterministic structure wrapped around a nondeterministic call. The model is free to be wrong. The system is not free to act on wrong output, and it is not free to spin forever trying.

What's a control gate, and why name it?

A control gate is the single chokepoint every action passes through before it's allowed to execute. It is the one place where you enforce scope, blast radius, and human approval. Name it, make it one function, and route everything through it. A named gate is one you can audit, log, and reason about. An unnamed check copied across forty functions is one you'll eventually forget to call, and the one you forget is the one that fires at the wrong target.

Two things the gate enforces above all. First, an allowlist: the agent may only touch the things you explicitly authorized, and everything else is denied by default. For a support agent that means it can issue a refund on an order tied to this customer and nothing else. For an offensive agent it means it may only touch targets named in the engagement, because an autonomous agent that can be talked into scanning an out-of-scope host through one poisoned response, the prompt injection risk from Part 2, is a liability with your name on the contract. Deny by default. Allow on purpose.

Second, human approval on high blast radius. The model already declared, in its structured answer, whether this action needs sign-off. The gate reads that flag. If it's set, the agent stops and waits for a person. The gate also holds a confidence floor, so a low-confidence guess never gets to act on its own. Inside that gate nothing is creative. Membership in the allowlist is a lookup. The confidence floor is a number comparison. The approval check is a yes or no. All deterministic, all auditable, the same on every single run.

Where does this connect to the actual build?

This is the conceptual companion to the control-plane build over in The Agentic Red Team. That post wires these exact ideas into a running agent in Claude Code, with the real code, and walks it through jailbreaking Gandalf level by level. Read this one to understand why the control plane is shaped the way it is. Read that one to watch it work against a target that fights back.

The pattern there is the pattern here, and it is the same pattern whether the agent answers tickets or runs an assessment. Mechanical steps are code. The model gets one narrow, shape-bounded job: propose the next move. Every move it proposes passes through one named gate that checks scope, checks confidence, and routes anything with real blast radius to a human before it happens. The demon does the thinking. The room does the containing.

The thing to actually take away

Stop trying to tame the model. It is not tamable and it is not supposed to be, because the wildness is the value. Tame the system.

Sort every action onto the determinism spectrum. Code for the mechanical, a bounded and validated model call for the genuinely creative, a human for anything with blast radius. Force a structured answer and reject what's off-contract. Retry with the error fed back, capped, then stop loud. Funnel every action through one named gate that denies what you didn't authorize by default and holds the trigger on anything irreversible until a person says go.

Do that and the picture of failure flips. The unbounded version is an agent that refunds the wrong account a thousand times before anyone notices, or scans the wrong host at two in the morning with your name on the contract. The bounded version is an agent you can actually put to work. Same power. A room it can't leave. That is the whole game, and it's the difference between a tool you can ship and a science project you have to babysit.

Next in this track we shift from caging the agent to using it as a weapon: a working taxonomy of how you actually hack AI. Everything lands at krypteiasec.com first.