
LLM Jailbreaking: The Complete Red Team Guide
Jailbreaking is the practice of getting a language model to produce output its own safety training was meant to refuse. It is distinct from prompt injection, which subverts an application's instruction hierarchy using untrusted data; jailbreaking targets the model's alignment directly. It works because refusal is a learned behavior trained on top of a next-token predictor, not a hard constraint wired into it, so an input that shifts the model far enough from where refusals were reinforced can suppress the refusal. This guide breaks the whole problem down to fundamentals with worked examples and diagrams: the taxonomy of techniques, how each one is built, how a model gets walked over gradually, how the Gandalf game teaches it, and how a frontier model like GPT-5.6, Claude Opus, or Sonnet is actually approached. It is written for authorized red teams and defenders, as an educational map of how the attack works and how to defend against it.
What jailbreaking actually is, in one paragraph
A jailbreak is any input that makes an aligned model do the thing its safety training says it should not do. The model was trained, after its base pretraining, to refuse certain requests: to decline to produce harmful content, to stay inside a set of policies, to hold a persona. A jailbreak is the input that gets past that trained refusal so the model answers anyway. The word comes from the phone-jailbreaking world, and the analogy is close: the device still works the way it was built, but the restriction the maker put on top of it has been lifted.
Jailbreaking from first principles: why a safety-trained model can be talked out of its safety
The useful way to understand jailbreaking is to refuse the surface explanations and reduce the thing to fundamentals. Strip away the persona names and the viral screenshots and ask what a model actually is. It is a next-token predictor: a function that, given a sequence of text, produces a probability distribution over what comes next, trained on a corpus far larger and far more amoral than any single policy. Everything the model can say is already latent in that base. Alignment does not remove any of it. Alignment is a second, much smaller training pass, reinforcement from human feedback or a constitutional method, that teaches the model to prefer refusing certain requests. Refusal is a behavior that was rewarded, not a wall that was built.
Once you see refusal as a learned behavior rather than a hard constraint, the entire attack surface falls out of two facts, both established in the research literature. The first is competing objectives: the model was trained to be helpful and to be harmless at the same time, and those goals conflict. A jailbreak manufactures a context where being helpful and following instructions pulls harder than refusing, and the helpful behavior wins. The second is mismatched generalization: safety training covered only part of the enormous input space the base model understands. The base model can read Base64, obscure languages, ASCII art, and role-play framings that the alignment pass never explicitly taught it to refuse in. Push the request into one of those under-covered regions and the capability is present while the refusal is absent. Wei, Haghtalab, and Steinhardt named these two failure modes in their 2023 analysis of why safety training fails, and nearly every technique that follows is an instance of one or both.
How the field maps jailbreaking: the taxonomy
The research and red-team community does not treat jailbreaks as a bag of tricks. It groups them by the mechanism they exploit, because the mechanisms are durable while any single prompt is perishable. There are four families, and the whole rest of this guide hangs off this map. The first two families come straight from the two first-principles failure modes. The third industrializes the search with automation. The fourth changes the model itself and only applies to open or local weights.
Reading the map top to bottom is how a professional thinks. Competing-objective attacks build a context where the helpful pull beats the harmless pull: role-play, an instruction never to refuse, a forced affirmative opening. Mismatched-generalization attacks move the request into a region alignment never covered: an encoding, another language, ASCII art. Automated-search attacks stop hand-crafting and put a second model or an optimizer in the loop to find a working input by brute iteration. Weight-level attacks skip prompting entirely and edit the refusal out of an open model's parameters. Everything below is these four families in detail, with the examples that show how each is actually built.
The models: frontier, open weight, and local, and why each jailbreaks differently
Not all targets are the same, and the first thing a serious tester does is identify which kind of model is in front of them, because the whole approach changes. Think of three broad classes. Frontier hosted models, the Claude, GPT, and Gemini family served behind an API, are the hardest single-prompt targets: they carry the strongest alignment training, they are frequently wrapped in external input and output classifiers that never touch the weights, and they are a moving target because the vendor updates them and patches known jailbreaks. Against a frontier model you are working black box, you cannot see or change the weights, and you are trying to find a context the alignment did not cover, ideally one that also slips past the guardrail models sitting in front.
Open weight models, the Llama, Qwen, Mistral, and similar releases whose parameters are published and downloadable, are a different world. Here the attacker holds the weights, and that unlocks a class of attack that no frontier model exposes. You control the system prompt with no vendor policy above it, you can set the sampler and even prefill the model's own reply, and above all you can change the model itself. Malicious fine-tuning with a few hundred examples can train the safety behavior back out, and abliteration can compute the model's refusal direction and project it out of the weights so the model loses the ability to refuse at all. Those weight-level techniques are covered in depth in the LLM Security spectrum guide; the point here is that on an open weight model, jailbreaking often stops being about clever prompts and becomes about editing the model.
Local models are usually open weight models run on the tester's own hardware through a runtime like Ollama, LM Studio, or llama.cpp, and they combine the worst case for a defender with the best case for a tester. Everything that is true of open weights is true, plus there is no rate limit, no logging, no external guardrail, and no vendor watching. A tester can prefill responses, script thousands of attempts, run an uncensored or already-abliterated variant pulled from a public hub, and iterate with zero friction. The spectrum runs from talk it out of its refusal at the frontier end to remove the refusal entirely at the local open weight end, and a good tester knows which end of that spectrum they are on before they send the first request.
The anatomy of a jailbreak: the parts every technique shares
Break any jailbreak down and the same components appear, the way any rocket decomposes into the same stages regardless of the mission. Naming the parts is what lets you build new techniques instead of memorizing old ones. There are five, and once you can see a jailbreak as these parts rather than as a magic string, you can read a new technique in seconds and combine parts across families to build one of your own.
The framing is the context the attacker constructs to shift the model away from where refusals were reinforced: a fictional scenario, a role, an authority claim, a hypothetical. The objective is the actual target behavior, kept constant while everything around it varies. The refusal suppressor is the component aimed straight at the model's tendency to say no: an instruction never to refuse, a demand to begin the reply with an affirmative token, a rule that breaking character is forbidden. The delivery and obfuscation is how the request is encoded and how it reaches the model, from plain chat text to an encoding the safety layer never learned to police to a payload hidden in a document the model retrieves. The fifth part, iteration, is the loop of trying, reading the response, and adjusting, because against a non-deterministic target a single attempt is a coin flip and the real work is refinement.
The tactics: the technique families and how each one works
The published tactics group cleanly by which first-principles mechanism they exploit, and grouping them this way matters more than any single example, because examples age out as models are patched while the mechanisms persist. The first group exploits competing objectives by manufacturing a context where helpfulness beats harmlessness. Persona and role-play framings, the lineage that began with the DAN, or Do Anything Now, prompts catalogued in the Shen and colleagues measurement study, tell the model it is a character without the assistant's rules. Refusal suppression and prefix injection, both formalized in the Wei and colleagues analysis, forbid the model from declining or force its answer to start with an affirmative continuation, so it slides past the moment where a refusal would begin. These are the beginner-visible jailbreaks, and against updated frontier models most of the classic ones no longer work, which is itself the lesson: single-string tricks are the most fragile tactic there is.
The second group exploits mismatched generalization by moving the request into a region safety training did not cover. Encoding attacks wrap the request in Base64, a substitution cipher, leetspeak, or another transform the base model can still read but the safety layer never learned to refuse in; the CipherChat work showed this systematically across ciphers. Low-resource-language attacks, demonstrated by Yong and colleagues against GPT-4, translate the request into a language where alignment coverage is thin, get the answer, and translate back. ArtPrompt, from Jiang and colleagues, hides the sensitive word as ASCII art so the text filters never see it but the model still reads it. Payload splitting and token smuggling cut the request into fragments that are individually harmless and let the model reassemble them. All of these share one mechanism: the capability survives the transform, the refusal does not.
The third group industrializes the search instead of hand-crafting a prompt, and this is where modern red teaming actually lives. Optimization attacks like GCG, the Greedy Coordinate Gradient method from Zou and colleagues, compute an adversarial suffix by gradient descent against an open model, and the resulting string can transfer, at a lower and steadily patched success rate, to closed models the attacker never had white-box access to. Automated black-box attacks put a second model in the loop: PAIR, from Chao and colleagues, has an attacker LLM refine its prompt over a handful of queries by reading the target's refusals, and TAP, from Mehrotra and colleagues, extends this into a tree of attempts with pruning. Many-shot jailbreaking, published by Anthropic in 2024, fills a long context with dozens of examples of the model complying, exploiting in-context learning so the model continues the pattern. Crescendo, from Microsoft, escalates across multiple benign-looking turns so no single message trips a filter. Best-of-N, from Hughes and colleagues, simply samples the same request many times with small variations until one gets through, exploiting the model's own non-determinism. Persuasion attacks, the PAP work from Zeng and colleagues on how ordinary rhetorical techniques move a model, round out the set. The through line of this group is that scale and iteration beat cleverness, which is exactly why the later sections are about agents.
A fourth group does not prompt the model at all; it changes it, and it applies only to open weight and local targets. Malicious fine-tuning retrains the safety behavior out with a small adversarial dataset. Abliteration, building on the Arditi and colleagues finding that refusal is mediated by a single direction in activation space, computes that direction and projects it out of the weights, producing a permanently unaligned model. These are the deepest jailbreaks because they are permanent and require no run-time trick, and they are the reason an open weight model's alignment can never be assumed to persist. They are covered fully in the LLM Security spectrum guide and its model-core sections; they are named here so the taxonomy is complete, because a tester facing a local model should reach for them before wasting effort on prompt craft the target's owner could simply not apply.
Worked examples: how each technique is built
Theory lands when you see the prompt. Every example below uses a deliberately benign objective, the kind you can practice legally: get a model to reveal a secret word it was told to guard, or to reveal its own system prompt. The mechanism is identical to a real engagement; only the objective is harmless. Read each as an assembly of the five anatomy parts, and notice how the same objective is reached by attacking a different weak point each time.
Direct request, and why it stops working
Indirect framing: never name the target
Encoding: move it where the filter cannot read
Refusal suppression and prefix injection
System-prompt extraction: read the rules you are fighting
The story: how a model gets walked over gradually
The most effective attacks are rarely a single message. They are a story the attacker tells across turns, where each step is individually reasonable and only the destination is off-limits. This is the Crescendo technique from Microsoft, and it works because a model judges each turn partly by the conversation it has already agreed to. Once it has produced a benign answer on a topic, the next, slightly stronger request looks like a natural continuation of its own words rather than a fresh violation.
Read the staircase as a method. Turn one opens on an innocent, adjacent topic the model answers freely. Turn two asks for a specific detail inside that answer. Turn three says expand on that, referencing the model's own words. Each turn moves the frame a little closer to the objective, and because the model treats its prior compliance as a commitment, refusing now would mean contradicting itself. By the final turn the actual ask arrives as the obvious next sentence in a conversation the model has been co-authoring. The same shape drives indirect prompt injection over multiple retrieved documents, and it is why single-message filters miss it: there is no one message to flag.
Gandalf: a jailbreak game you can learn on
The best legal way to feel every technique in this guide is Lakera's Gandalf, an open browser game at gandalf.lakera.ai where you try to trick an LLM into revealing a password across a ladder of levels, each adding a stronger defense. It became the field's canonical teaching example because it turns the abstract taxonomy into a concrete, safe, hands-on ladder: the objective never changes, only the defense does, so you learn to climb the taxonomy one mechanism at a time. It has logged millions of attempts, and it is where a huge number of practitioners first understood that jailbreaking is a discipline, not a trick.
Lakera describes each level as exactly three things: a system prompt, a guard that checks your input, and a guard that checks the model's output. Trace the ladder against the taxonomy and it teaches itself. Level one has no defense and the direct request works. Level two adds an instruction not to reveal the password, which is only a request, so an override or a role-play frame moves past it. Level three adds an output filter that blocks the literal password string, which is exactly where indirect framing earns its keep: ask for an acrostic poem or the word spelled backwards and the filter, watching for the forward string, never sees it. The middle levels add an LLM that reads the whole transcript and an input guard that blocks the words password and secret, so you stop naming the target at all and encode the reply past the checker. The upper levels add a second model that judges whether your prompt is even asking about the password, combined with a blacklist, so only a stacked attack gets through: disguise the intent, get the secret as a byproduct, and layer encoding over a role-play frame. The final level, Gandalf the White, is self-learning and stacks every defense at once, which is precisely the shape of a real hardened deployment. Climbing Gandalf is climbing the taxonomy, and the mechanism you use at each rung is the same one a professional reaches for against the real thing. It is worth doing: Lakera has reported more than 200,000 players and millions of attempts, with only a small fraction ever reaching the top levels.
Prompt injection as a tactic family, extended
Prompt injection deserves its own treatment inside this guide, because it is both a distinct threat and the primary way a jailbreak reaches a model the attacker cannot talk to directly. The distinction first. Prompt injection is a failure of the instruction hierarchy: a model reads its developer instructions and its untrusted input on the same channel and cannot reliably tell which is which, so an instruction placed in the untrusted data can override the developer's. Simon Willison, who named the problem, frames it as concatenating trusted and untrusted strings and hoping the model honors the boundary. OWASP places it at the top of its 2025 Top 10 for LLM Applications as LLM01, because the root cause is structural and unsolved. Jailbreaking, by contrast, attacks the model's own policy. A model with no application around it can be jailbroken but cannot be prompt-injected, because there is no developer instruction to subvert.
The two combine constantly, which is why they belong in one map. Direct prompt injection puts the payload in the user's own input, and when that payload is itself a jailbreak, the injection is just the envelope. Indirect prompt injection is the dangerous form: the payload hides in content the model is given rather than typed by the operator, a web page it browses, a document in a retrieval store, an email in an inbox, the output of a tool. The model reads that content on the same channel as its instructions and can mistake a hidden instruction for a legitimate one. This is how a jailbreak travels: an attacker never speaks to the target model, they plant a jailbreak inside a document, and when the victim's agent retrieves and reads it, the model is steered from inside its own trusted context. As the relationship note running through Krypteia's threat intel keeps underlining, once an agent holds email, database, shell, or deployment access, a single successful injection lets an attacker act through the system rather than merely producing a wrong answer.
For the practical mechanics of testing this, building a corpus, running direct and indirect cases, measuring against a non-deterministic target, and regression-testing as models change, Krypteia's Prompt Injection Testing guide is the deep dive that sits under this flagship. Treat this section as the map of where injection fits in the jailbreaking taxonomy, and that guide as the field manual for exercising it.
The strategies: how a professional approaches a target
Tactics are moves; strategy is the campaign, and the difference is what separates a person collecting jailbreak strings from a red teamer running an engagement. Strategy starts with a threat model of the specific target. Is it a frontier hosted model or an open weight one, because that decides whether you are talking it out of a refusal or editing it out. Is there a guardrail model in front, an input or output classifier, because a jailbreak that convinces the core model still fails if a separate classifier flags the output, and you may need to defeat the classifier and the model as two problems. What is the model wired to, because a jailbroken chat that produces disallowed text is a finding, but a jailbroken agent that can call a tool is an incident, and the second is what actually matters to the client.
With the target modeled, the campaign follows a recognizable shape. Reconnaissance comes first: probe the refusal boundary with benign-adjacent requests to learn where the model draws its lines, and attempt system-prompt extraction, because a leaked system prompt reveals the exact rules and sometimes the guardrail logic you are working against. Then select tactics by mechanism fit rather than by reputation, matching the technique to what you learned in recon, an encoding attack where filters are the obstacle, a multi-turn escalation where single messages are scored, an automated search where you have query volume. Then iterate and adapt, treating each refusal as signal about the boundary rather than a dead end. A second strategic lever worth naming is transferability: because optimization attacks and many hand-built framings carry across models, a professional often develops an attack against a weak or local model where iteration is free and then transfers the working attack to the hardened frontier target. The strategist's mindset is the real deliverable here: model the target, probe before you push, choose by mechanism, and industrialize the iteration.
Jailbreaking a frontier model: GPT-5.6, Claude Opus, and Sonnet
Everything gets harder against a current frontier model. A modern hosted model from the GPT, Claude, or Gemini families carries alignment training that has itself been hardened against thousands of hours of red teaming, and it usually does not sit alone. It sits inside a stack: an input classifier screens your prompt, the model core answers, and an output classifier screens the reply before it ever reaches you. A jailbreak that convinces the core model still dies if the output classifier flags the result, so you are not defeating one system, you are defeating three. How hard that is shows in the numbers: Anthropic's Constitutional Classifiers, a published guardrail of exactly this shape, cut jailbreak success from roughly 86 percent to under 5 percent in its own automated red-team evaluation, and a pre-deployment red team of well over a hundred people spending thousands of hours did not find a single universal jailbreak against it.
So the professional approach against a frontier target is not a magic prompt; it is a process. You start with reconnaissance, mapping where the refusal boundary sits and attempting to leak the system prompt to learn the rules and any visible guardrail logic. You pick your mechanism against the specific obstacle: if the block is a text classifier, an encoding or a low-resource language moves the request out of its reach; if single messages are scored, a multi-turn Crescendo build spreads the intent across a conversation; if you have query volume, you run an automated attacker-and-judge loop that mutates candidates faster than any human. You lean on transferability, developing an attack against a weak or local model where iteration is free and carrying the working shape over to the hardened target, since many framings and some optimization suffixes transfer. The largest public evidence base for how this actually plays out is HackAPrompt, a competition that gathered hundreds of thousands of real jailbreak attempts and became a standard corpus for which attacks succeed and why. And you accept the arms race: what works this month is patched next month, which is why continuous testing, not a one-time result, is the only honest posture against a frontier model.
How agents and skills jailbreak continuously
The single biggest shift in modern jailbreaking is that the best attacker is no longer a human with a clever prompt; it is a loop. The manual techniques all share the same weakness against a non-deterministic, frequently patched target: a human tries a handful of variations and gives up, while the search space of framings, encodings, and phrasings is effectively infinite. An automated agent does not give up. The formal versions of this idea are already in the literature: PAIR and TAP are attacker models in a loop, generating a candidate, sending it to the target, reading the refusal or success, and refining, converging on a working jailbreak in a small number of queries with no human in the middle.
Generalize that loop and you have the architecture every serious AI red team now runs. An attacker component generates candidate jailbreaks, drawing from a library of the tactic families above and mutating them. A target connector sends each candidate to the model under test, whether that is a frontier API or a local runtime. A judge component, usually a separate model acting as an LLM-as-judge, scores whether the response actually complied or merely looked like it, because grading a non-deterministic target is itself a hard problem. An orchestrator runs the loop, feeds the judge's verdict back to the attacker so the next candidate is smarter, mutates across technique families when one stalls, and logs every attempt. This is why agents beat humans at jailbreaking decisively: they are tireless, they iterate through the night, they mutate systematically instead of guessing, and they turn a fragile single-string tactic into a persistent search that adapts as the target does. It is also, uncomfortably, why the same automation makes continuous defensive testing possible, because the loop that finds a jailbreak is the loop a blue team runs to find it first.
Building a jailbreaking agent with Claude Code, for authorized red teams
The loop above is buildable today, and a coding agent like Claude Code is a natural way to build and run it, because the harness is mostly orchestration and text handling rather than novel machine learning. The point of building your own, rather than only running a public tool, is that a red team's value is a living technique library tuned to the targets it actually faces, and a harness you own is one you can extend the day a new tactic is published. Think of the build as five modules, each of which a coding agent can scaffold and maintain. The technique library holds the tactic families as parameterized templates, the framing, the objective slot, the refusal suppressor, the encoding, so new candidates are generated by composition rather than copy-paste. The target connector abstracts over a frontier API and a local runtime behind one interface so the same run can test both. The judge wraps a scoring model with a rubric that distinguishes real compliance from a response that only looks compliant. The orchestrator is the loop: generate, send, judge, feed back, mutate, log, stop on success or budget. The corpus is the durable store of every attempt and verdict, which becomes both your evidence and your regression suite.
Claude Code fits this because it can hold the whole harness as a project, scaffold each module, and then run the loop itself as a task or a skill, generating candidates, calling the target, invoking the judge, and writing results to the corpus, while you supervise. The discipline that makes it a red team tool rather than an abuse tool is entirely in the guardrails around the loop, and they are not optional. Point it only at systems you are authorized to test, in writing and in scope. Run it against a staging target, never a production system carrying real users or real data. Capture evidence, the request, the response, the judge verdict, for the report, because a finding you cannot reproduce is not a finding. And treat the corpus as a regression suite you re-run when the target model updates, because that is where the offensive harness quietly becomes a defensive one. Krypteia's Agentic AI Red Teaming guide covers the engagement methodology this harness plugs into, and the same hackbot pattern, an autonomous attacker agent with a judge and a corpus, is the throughline across Krypteia's red team tooling.
How to expand and build on it: turning a technique into a program
A single jailbreak is a trophy; a jailbreaking program is a capability, and the difference is in the systems you build around the technique. The first is a living corpus. Every attempt, successful or not, with its target, its technique family, its judge verdict, and its date, goes into a store you can query, because the questions that matter, which families still work against which models, what the current attack success rate is, what broke when the vendor pushed an update, are only answerable against history. Attack success rate, the fraction of attempts that produce genuine compliance, is the number that turns anecdote into measurement, and measuring it honestly against a non-deterministic target is a skill in itself: the same request can succeed one sample and fail the next, so you report over many trials rather than claiming a single win.
The second system is regression, and it is the one most people skip. A jailbreak is perishable. The vendor patches, the model updates, the guardrail is retrained, and a string that worked last month is refused today. That is not failure, it is signal, and a program captures it: re-run the corpus on every model update and watch which techniques die and which survive, because the survivors are the structural ones worth studying and the deaths are your evidence that a defense landed. The third is grounding against the public benchmarks so your numbers mean something outside your own harness: HarmBench, JailbreakBench, and AdvBench are standardized evaluation sets the research community uses, and mapping your corpus to them lets you compare your findings to published baselines. Underneath all three is the honest framing of the whole endeavor, that this is an arms race with no final state. Alignment improves, attacks adapt, alignment improves again. A jailbreaking program is not a puzzle you solve once; it is a measurement you keep taking, which is precisely why it belongs to a red team as an ongoing function rather than a one-time test.
The defensive mirror: what every jailbreak teaches the blue team
Every offensive technique in this guide is also a defensive requirement, and reading the taxonomy backwards is how a blue team turns it into controls. The first and most important lesson is the one the first-principles section already implies: a system prompt asking the model to behave is not a control. Instructions telling the model to refuse are exactly what a jailbreak overrides, so a defense that consists of more instructions is fighting on the attacker's chosen ground. Real defenses sit outside the model's own compliance. Input and output classifiers, separate guardrail models that screen requests and responses independently of the core model, catch categories of attack regardless of how cleverly the core model was convinced; Anthropic's Constitutional Classifiers, published in 2025, are a formalized version aimed specifically at resisting jailbreaks. Because encoding and language attacks defeat naive text filters, those classifiers have to reason about intent rather than match strings.
Deeper defenses change the model rather than screen it. Circuit breakers, the representation-rerouting method from Zou and colleagues in 2024, train the model to derail its own generation when it enters a harmful internal state, attacking the problem at the representation level where the encoding tricks cannot reach. For the injection channel specifically, spotlighting, from Hines and colleagues at Microsoft, marks untrusted content so the model can tell developer instructions from retrieved data, addressing the trust-boundary confusion at its root. And for the agent layer, where a jailbreak stops being text and becomes action, the control is structural least privilege: scope every tool to the minimum permission, put a human in front of irreversible actions, and constrain autonomy so no single successful jailbreak can compose its way to real impact. The frameworks tie it together: OWASP's Top 10 for LLM Applications names the operational risks, MITRE ATLAS catalogs the adversarial techniques so findings share a vocabulary, and the NIST AI Risk Management Framework and its adversarial machine learning taxonomy give the governance anchor. Map every finding to those and a list of clever jailbreaks becomes a remediation plan a client is already obligated to act on.
Ethics, authorization, and why this is taught openly
There is a fair question underneath a guide like this: why write down how to break the safety of a model at all. The answer is the same one that justifies teaching every other offensive security discipline. You cannot defend a system whose attacks you do not understand, and the people building and deploying these models need testers who can find the jailbreak before an adversary does. The techniques here are the published state of the research, taught the way a penetration tester learns exploitation, as an educational map of mechanism and method rather than a packaged weapon. This guide deliberately stops at that map. It explains why techniques work and how a testing program is built; it does not hand over tuned payloads engineered to extract genuinely dangerous content from a named production model, because that would cross from teaching into arming, and the line is the same one the responsible research community draws.
Where this fits in the Krypteia map
This is the flagship guide to jailbreaking, the orienting page for the whole subject, and it connects to the rest of Krypteia's coverage in a deliberate shape. The LLM Security spectrum guide is the wider map: jailbreaking is the prompt-and-alignment story, and that guide places it in the full six-layer attack surface alongside the training data, the weights, the fine-tuning, and the retrieval layers, including the weight-level abliteration and fine-tuning attacks this guide only names. Read that one to see how a jailbreak fits into the larger stack, and read this one for the depth on the alignment-bypass layer itself.
Two guides sit directly under this one as the practical deep dives. Prompt Injection Testing is the field manual for the injection tactic family, the corpus, the direct and indirect cases, the measurement, and the regression discipline. Agentic AI Red Teaming is the methodology for the layer where a jailbreak becomes action, the agent kill chain, the autonomous attacker agents, and how to scope and measure an engagement against an agent rather than a chat box. Around those, the MCP Security, CLI Tool Security, and AI Agent Security guides cover the specific surfaces where a jailbroken instruction turns into a tool call with real-world impact. Together they are one map of the same territory, and this guide is the door most people should walk through first.
- What is the difference between jailbreaking and prompt injection?
- Jailbreaking targets the model's own safety training, getting it to produce output its alignment was meant to refuse. Prompt injection targets an application's instruction hierarchy, smuggling instructions into untrusted data so the model follows the attacker instead of the developer. A model with no application around it can be jailbroken but not prompt-injected, because there is no developer instruction to subvert. In practice they combine: indirect prompt injection is a common way to deliver a jailbreak to a model an attacker cannot talk to directly.
- Why can a safety-trained model be jailbroken at all?
- Because refusal is a learned behavior, not a hard constraint. A model is a next-token predictor trained on a huge corpus; alignment is a smaller second training pass that teaches it to prefer refusing certain requests. Two structural facts make that reversible: competing objectives, where being helpful conflicts with being harmless and can be made to win, and mismatched generalization, where safety training did not cover the whole input space the base model understands. Jailbreaks exploit one or both.
- What is the Gandalf jailbreak game?
- Gandalf is a free educational game by Lakera at gandalf.lakera.ai where you try to trick an LLM into revealing a password across a ladder of levels, each adding a stronger defense. It is the field's canonical teaching example because the objective never changes while the defense does, so you learn to climb the jailbreak taxonomy one mechanism at a time against a completely benign target. It has logged millions of attempts.
- Can frontier models like Claude, GPT, and Gemini be jailbroken?
- Frontier hosted models are the hardest single-prompt targets because they carry the strongest alignment and are often wrapped in separate input and output classifiers, and vendors patch known jailbreaks quickly, so classic hand-built prompts tend to stop working. They are not immune. Automated and optimization-based attacks, multi-turn escalation, and transferable adversarial strings continue to find gaps, which is why continuous red teaming rather than a one-time test is the standard.
- How do you jailbreak a local or open-source model?
- On an open weight or local model the attacker holds the weights, which unlocks attacks a frontier model never exposes: full control of the system prompt, the sampler, and response prefilling, plus the ability to change the model itself through malicious fine-tuning or abliteration, which computes the model's refusal direction and projects it out of the weights for a permanent effect. On these targets, jailbreaking is often about editing the model rather than crafting a prompt.
- What is the most effective jailbreak technique?
- There is no single best technique, and any specific string is perishable because vendors patch. The durable answer is a method rather than a string: automated, iterative attacks that put an attacker model in a loop against the target, such as the PAIR and TAP approaches, combined with a technique library spanning role-play, encoding, multi-turn escalation, and optimization. Scale and adaptation beat any one clever prompt.
- Can jailbreaks be automated?
- Yes, and automation is how modern red teaming works. An attacker component generates candidate jailbreaks, a connector sends them to the target, a judge model scores whether the response actually complied, and an orchestrator feeds the verdict back and mutates the next attempt. Research systems like PAIR and TAP formalize this loop, and a coding agent can build and run the same harness against authorized targets.
- What is abliteration?
- Abliteration is a weight-level jailbreak for open models. Building on the finding that refusal in instruction-tuned models is mediated by a single direction in activation space, it computes that direction and projects it out of the model's weights, so the model loses the ability to represent refusal. The result is permanent and requires no run-time trick, which is why it is considered the deepest jailbreak. It is covered in depth in the LLM Security spectrum guide.
- How do you defend against jailbreaks?
- Not with more instructions telling the model to behave, since that is exactly what a jailbreak overrides. Real defenses sit outside the model's own compliance: input and output guardrail classifiers that screen independently, such as Constitutional Classifiers; representation-level methods like circuit breakers that derail harmful generations internally; spotlighting to mark untrusted content against injection; and, at the agent layer, least privilege and human approval so a successful jailbreak cannot compose into real impact.
- Is jailbreak testing legal?
- It is legitimate against systems you own or are explicitly contracted to test, inside a written scope, with findings disclosed to the party that can fix them. It is not legitimate against a model or product you have no authorization to attack, and the fact that a jailbreak is a text string rather than a traditional exploit does not change the ethics or, depending on jurisdiction, the law. Authorization, scope, and intent are what separate a red teamer from an abuser.