PromptInjects glossary

AI security terms in plain language.

Definitions you can use when threat-modeling LLM applications, retrieval systems, and AI agents.

Direct Prompt Injection is an attack in which a person places competing instructions in an application's user-controlled input. The model may then change tasks, reveal protected context, or request an unsafe tool action. The technique is about the input path, so it can be simple, multi-turn, role-played, or carefully formatted.

Direct System Override

Direct System Override is an injection that explicitly tries to replace the application's governing instructions with a new task. The attacker frames earlier rules as cancelled, obsolete, or subordinate. It is narrower than direct prompt injection: the defining signal is the attempted change to the control layer, not merely an unexpected request.

Role-Based Deception

Role-Based Deception is an injection that claims a trusted identity, job function, or authority so an AI application treats a sensitive request as routine work. The model cannot independently verify the claim from prose alone. The failure is social-engineering pressure crossing into data access, not the legitimate use of roles in a product.

Confusion Attack

Confusion Attack is an injection that creates ambiguity, contradiction, or competing objectives so a model loses track of the active task. The attacker may mix examples with commands, alternate priorities, or ask for incompatible outputs. The category is descriptive: the core risk is unresolved instruction conflict leading to an unsafe interpretation.

Crescendo Attack

Crescendo Attack is a multi-turn injection that escalates from harmless cooperation to a request the application should refuse. Each turn makes the next step feel consistent with the conversation, so a model carries forward an overly helpful frame. The technique exploits accumulated context and momentum rather than relying on one dramatic instruction.

Encoding

Encoding is an injection technique that represents instructions or requested values through a reversible format, such as a character mapping or harmless-looking transformation. The model may decode the meaning even when a narrow filter does not. Safe analysis treats encoded text as untrusted input and never assumes that visual obscurity changes authorization.

Greedy Coordinate Gradient

Greedy Coordinate Gradient is a white-box optimization method that searches for token suffixes which increase a model's probability of a chosen target response. It differs from conversational social engineering because the search uses model signals during evaluation. The resulting text may look nonsensical, and it is chiefly a research and red-team technique.

Instruction Overriding

Instruction Overriding is a tactic that tells a model to disregard, replace, or forget earlier guidance so the attacker can steer the next response. It is a recognizable subset of direct prompt injection. The wording can be blunt or polite, but the security question is whether user text can change the application's intended control hierarchy.

Variables and Keywords

Variables and Keywords is an injection technique that turns a sensitive request into a substitution, trigger, or template value. Systems that resolve placeholders or react to special terms may execute the meaning without rechecking it. The risk is the gap between text processing and authorization, especially when a model fills or expands user-controlled fields.

Multi-Language / Character Sets

Multi-Language / Character Sets attacks express an instruction across languages, scripts, or visually confusable characters. A narrow filter may miss the pattern while a capable model reconstructs its intent. The technique tests normalization, language coverage, and provenance handling; it does not make an otherwise unauthorized request legitimate.

Narrative-Based

Narrative-Based is an injection technique that wraps a restricted request in fiction, roleplay, or a hypothetical scenario. The story changes the framing but not the underlying authorization. A model may treat the request as creative work and lose sight of the real boundary, especially when the application gives it protected context or tools.

Sequences / Delimiters

Sequences / Delimiters attacks make untrusted text resemble a control block through separators, headers, tags, or structured message syntax. A model may infer authority from formatting even when the content came from a user or retrieved document. The security issue is confusing presentation with provenance, not the use of structured data itself.

Indirect Prompt Injection

Indirect Prompt Injection places hostile instructions in content an AI application retrieves or processes for an ordinary task. The user may never type the attacker’s words into the chat. Because the model sees the content as context, safe systems must track provenance and treat documents, pages, feeds, images, and tool results as untrusted.

Multi-Modal

Multi-Modal Prompt Injection hides or places instructions in images, audio, video, or rendered documents that a vision or speech-capable model can perceive. The media may look like an ordinary input to a person while its extracted tokens compete with the user's task. Defensive design treats every modality as untrusted content with provenance.

External Sources

External Sources injection exploits the trust gap between a model and a page, document, feed, or API response it retrieves. The source may contain useful facts plus instructions written by someone outside the application team. Retrieval expands the attack surface, so source identity, integrity, relevance, and permissions must be evaluated before an agent acts.

Hidden Text / Invisible Prompt

Hidden Text / Invisible Prompt attacks conceal instructions from a person while leaving them available to a parser or model. Styling, metadata, zero-width characters, bidirectional controls, or off-screen content can make a document appear benign. The defense is to inspect what the machine receives, preserve provenance, and avoid granting hidden text authority.

Stored Prompt Injection

Stored Prompt Injection writes hostile instructions into persistent application data so a later retrieval can steer an agent in another session. The text may live in a note, profile, ticket, saved conversation, or vector record. Persistence separates the attacker from the eventual victim and turns an ordinary data store into a long-lived instruction channel.

Memory Poisoning

Memory Poisoning deliberately corrupts an agent's stored context so future conversations retrieve attacker-authored guidance and change behavior. The target may be a memory record, profile, summary, embedding, or knowledge base. It is persistent indirect injection with a special focus on how memory selection and reuse can bias later planning or tool use.

Orchestrator Impersonation

Orchestrator Impersonation forges a planner, scheduler, supervisor, or control-plane message so an agent treats attacker text as a trusted workflow instruction. Structured labels and task identifiers can look official to a model, but prose cannot prove origin. Secure multi-agent systems authenticate handoffs and validate permissions outside the model.

Cross-Agent Prompt Propagation

Cross-Agent Prompt Propagation carries an injected instruction from one agent into another through a summary, plan, tool result, or handoff. The downstream agent may trust peer output more than user text and continue the instruction into later stages. The risk is compounded when provenance is lost across an automated multi-agent workflow.

Exploiting Agent Tool Chains

Exploiting Agent Tool Chains steers an agent into combining individually permitted tools to reach a sensitive outcome no single tool was meant to enable. The model becomes a confused deputy with useful capabilities but weak composition checks. Defenses minimize tools and permissions, validate each step, and require approval before consequential combinations.

Server-Side Request Forgery (SSRF) via LLM Agent

Server-Side Request Forgery (SSRF) via LLM Agent occurs when a model-controlled fetch or browser tool is persuaded to request a destination from the server's network position. The attacker may use direct instructions or poisoned content. The model is only the interface; network allowlists, identity-aware proxies, and response filtering enforce the boundary.

Sandbox Escape via Code Execution

Sandbox Escape via Code Execution attempts to move an agent's interpreter beyond its intended isolation into host files, networks, devices, or neighboring workloads. The model may be manipulated into generating code, but containment is an infrastructure responsibility. Strong isolation, minimal capabilities, resource limits, and explicit brokered access matter more than a warning prompt.

Covert Channel Data Theft

Covert Channel Data Theft moves sensitive information through an allowed-looking path such as names, timing, formatting, errors, or ordinary tool activity. An agent with private context and communication capabilities may encode data where normal monitoring sees routine traffic. Defenses combine data minimization, egress policy, anomaly detection, and review of unusual cross-channel correlations.

System prompt

A system prompt is trusted task and behavior guidance supplied to a model by the application. It can shape responses, but it is not authentication, authorization, or a secret container. Keep sensitive policy out of context when possible and enforce permissions in application code.

Guardrail

A guardrail is a control that limits unsafe model inputs, outputs, tools, or actions. Effective guardrails combine deterministic checks, least privilege, provenance, and human review where needed. A natural-language instruction alone is a weak guardrail because a model can misunderstand or prioritize competing text.

Jailbreak

A jailbreak is an attempt to bypass a model or application's safety behavior so it produces a restricted response. Jailbreaks overlap with prompt injection, but the term emphasizes evading safety constraints. The business impact still depends on context, connected tools, data access, and controls outside the model.

Retrieval-Augmented Generation

Retrieval-Augmented Generation, or RAG, retrieves documents or records and places selected content into a model's context before generation. It can improve factual grounding while expanding the trust boundary. Retrieved material remains untrusted data and must not silently become an instruction or permission.

Model Context Protocol

Model Context Protocol, or MCP, is a standard way for AI applications to connect models with tools and context providers. It can simplify integrations, but each server, resource, and tool adds a trust and permission boundary. Authenticate components, scope capabilities, and validate data and actions independently.

Tool call

A tool call is a structured request for an external function such as searching, fetching, writing, or sending. Models may propose tool calls, but an application should validate the arguments, user authorization, destination, and side effects before execution. Tool descriptions are not a permission system.

Exfiltration

Exfiltration is the unauthorized movement of information from a protected system to an outside party or location. In AI workflows it may happen through text, tool results, messages, URLs, files, or subtle side channels. Reduce model context, restrict egress, and correlate data access with outbound events.

Provenance

Provenance records where a piece of content came from, how it changed, and which component supplied it to a model. Labels and hashes help investigators, but provenance is useful only when downstream policies honor it. Untrusted source text should remain data even when it appears in a trusted workflow.

Least privilege

Least privilege gives a component only the data, tools, permissions, and destinations needed for its current task. For an agent this includes limiting functions, arguments, identities, network access, retention, and autonomy. It reduces the impact of a model error or successful injection without requiring perfect detection.

Human-in-the-loop

Human-in-the-loop review places a person at a meaningful decision point before a high-impact AI action occurs. The reviewer needs enough context to understand the proposed action and must be able to reject it. A rubber-stamp prompt or review after an irreversible side effect is not an effective control.

Red teaming

Red teaming is a structured attempt to find security failures by acting like an adversary within an agreed scope. For AI systems it should cover direct and indirect inputs, tools, memory, multi-turn behavior, and side effects. Use fictional data, isolated environments, clear rules of engagement, and actionable mitigations.

Prompt firewall

A prompt firewall is a filtering or policy layer that inspects model inputs and outputs for suspicious content or prohibited flows. It can add visibility and reduce common attacks, but it cannot replace authorization, isolation, or tool controls. Attackers can vary wording, language, format, and conversation history.

Data minimization

Data minimization means giving a model only the fields, records, and context required for a defined task. It limits what an injection can reveal and reduces the value of accidental logs or outputs. Combine it with object-level authorization because removing some data does not make remaining data public.

Output validation

Output validation checks model-generated text or structured data before it is displayed, stored, executed, or passed to another system. Use schemas, allowlists, authorization checks, and destination validation. Validation should inspect meaning and side effects rather than trusting a model because its response matches a requested format.

Agent

An agent is an AI application that can plan across steps, use tools, retrieve context, or act on behalf of a user. Its risk comes from the combination of model behavior and real capabilities. Secure agents need bounded autonomy, authenticated tools, explicit state, and policy checks outside natural language.

Trust boundary

A trust boundary is the point where data, identity, or authority moves between components with different security assumptions. In AI systems it may sit between a user and model, a retrieved document and prompt, or an agent and tool. Marking these crossings makes it possible to apply the right validation and authorization.

Prompt leakage

Prompt leakage is the disclosure of hidden system or developer instructions that an application intended to keep out of the user-facing response. Leaked text can reveal design details, but it is not itself an authorization bypass. Protect secrets by design and keep sensitive policy out of model context when possible.

Safe sandbox

A safe sandbox is an isolated environment for testing AI behavior with synthetic data, restricted network access, disposable state, and no production credentials. It lets a team explore injections and tool failures without turning an educational exercise into an incident. Isolation must be enforced by infrastructure, not only described in a prompt.

Confused deputy

A confused deputy is a program that holds real permissions and is tricked into using them for someone who should not have them. An AI agent becomes one when text from a user, page, or document persuades it to read, send, or change something using its own access instead of the requester's.

Context window

The context window is all the text a model can see when it writes a reply: the app's instructions, the conversation so far, retrieved documents, and tool results. The model has no built-in way to tell which parts are trusted, which is why anything placed in the window can be repeated or obeyed.