LLM · AI SECURITY · OWASP

OWASP Top 10 for LLM Applications

Understand the top risks for large language model applications and how to mitigate them.

The OWASP Top 10 for Large Language Model Applications project aims to educate developers, designers, architects, managers, and organizations about the potential security risks when deploying and managing Large Language Models (LLMs).

LLM01

Prompt Injection

Crafted inputs manipulate an LLM into ignoring system instructions or taking unintended actions. Direct injections overwrite the system prompt; indirect ones hide instructions in external content the model later reads.

Examples
  • Direct Prompt Injection A malicious user injects prompts that extract sensitive information or override safety rules.
  • Indirect Prompt Injection Untrusted webpage or document content steers the model when it is summarized or retrieved.
  • Scam Through Plugins Websites or plugins abuse tool-calling to run scams or exfiltrate data.
Prevention
  • Privilege Control Limit what the LLM can access and apply role-based permissions to tools.
  • Human Approval Require explicit user consent before privileged or irreversible actions.
  • Segregate Content Keep untrusted retrieved content clearly separated from system and user prompts.
  • Trust Boundaries Treat model output as untrusted and visually mark when content may be unreliable.
Attack scenarios
  • Chatbot Remote Execution Injection leads to unauthorized tool use through a chatbot.
  • Email Deletion Indirect injection causes destructive mailbox actions.
  • Exfiltration via Content Embedded prompts push private data into attacker-controlled channels.
LLM02

Insecure Output Handling

Downstream systems accept LLM output without validation. That can turn model text into XSS, SSRF, privilege escalation, or remote code execution when it is rendered or executed as code.

Examples
  • Remote Code Execution Model output is passed to a shell or interpreter without sanitization.
  • Cross-Site Scripting (XSS) Generated JavaScript or Markdown is rendered unsafely in a browser.
Prevention
  • Zero-Trust Output Treat LLM output like untrusted user input - validate and sanitize before use.
  • OWASP ASVS Guidance Apply input validation and encoding standards to model-generated content.
  • Output Encoding Encode output for the target context (HTML, URL, SQL, shell) before render or exec.
Attack scenarios
  • Chatbot Shutdown Unvalidated output triggers a plugin to shut down or misbehave.
  • Sensitive Data Capture Generated payloads send secrets to an attacker-controlled endpoint.
  • Destructive Queries Model-crafted SQL or admin commands wipe or alter data.
LLM03

Training Data Poisoning

Tampered training or fine-tuning data introduces backdoors, bias, or unsafe behavior. Sources can include Common Crawl-style corpora, books, crowd-sourced sets, and vendor fine-tunes.

Examples
  • Malicious Data Injection Falsified samples are inserted during training or fine-tuning.
  • Biased Training Outputs The model inherits and amplifies tainted labels or narratives.
  • Content Injection Attackers seed public corpora so later models learn harmful patterns.
Prevention
  • Dataset Provenance Track origin, licensing, and integrity checks for every training set.
  • Filtering and Review Scrub toxic, secret, and adversarial samples before training.
  • Canary and Eval Suites Test for backdoors and bias regressions after every train or fine-tune.
Attack scenarios
  • Misleading Outputs Poisoned models promote biased or harmful content.
  • Toxic Data Injection Malicious contributors push hate or unsafe material into open datasets.
LLM04

Model Denial of Service

Attackers force expensive inference (huge context, recursive tools, adversarial loops) so availability drops or cloud bills spike. LLM cost and latency make this easier than classic DoS.

Examples
  • High-Volume Queuing Flood the model with many concurrent expensive requests.
  • Resource-Consuming Queries Unusual prompts force heavy context or tool recursion.
  • Continuous Input Overflow Oversized inputs exhaust context windows and workers.
Prevention
  • Input Validation Cap length, reject pathological patterns, and filter abusive content.
  • Resource Caps Limit tokens, tools, and concurrency per user or API key.
  • Rate Limits and Quotas Throttle bursts and alert on anomalous spend or latency.
Attack scenarios
  • Resource Overuse One tenant starves shared hosted capacity for others.
  • Request Amplification A page or bot multiplies costly model calls through automation.
LLM05

Supply Chain Vulnerabilities

Third-party models, datasets, plugins, and libraries can ship vulnerabilities or backdoors into your LLM stack - similar to classic software supply-chain risk, with model artifacts added.

Examples
  • Package Vulnerabilities Outdated Python or JS deps used by the app or serving stack.
  • Vulnerable Models Risky pre-trained checkpoints used for fine-tuning or inference.
  • Poisoned Data Crowd-sourced or vendor datasets that were already compromised.
Prevention
  • Supplier Evaluation Vet model and plugin vendors, SLAs, and update policies.
  • Plugin Testing Only ship tested, pinned, least-privilege plugins and tools.
  • Dependency Hygiene SBOM, SCA scanning, and pinned versions for serving and training code.
Attack scenarios
  • Library Exploitation Known CVEs in ML or web libraries become RCE in your stack.
  • Malicious Plugin A marketplace plugin exfiltrates prompts or credentials.
LLM06

Sensitive Information Disclosure

Models may leak PII, secrets, or proprietary text from training data, RAG stores, or system prompts. Weak filtering turns chat into an unintended data-exfil channel.

Examples
  • Incomplete Filtering Responses still contain secrets that simple denylists missed.
  • Overfitting / Memorization The model reproduces sensitive strings seen in training.
Prevention
  • Data Sanitization Scrub secrets and PII from training and retrieval corpora.
  • Input and Output Filters Detect and block sensitive patterns on both sides of the model.
  • Least Data in Context Only retrieve fields the task needs; never dump full records.
Attack scenarios
  • Unintentional Exposure One user receives another user’s private context.
  • Filter Bypass Adversarial phrasing extracts PII past naive guards.
LLM07

Insecure Plugin Design

Plugins and tools expose powerful actions with weak parameter validation or authz. Once the model can call them, prompt injection becomes remote control of your systems.

Examples
  • Single Field Parameters Plugins accept free-form strings instead of typed, separated fields.
  • Configuration Overrides Config strings let callers widen privileges or change destinations.
Prevention
  • Parameter Control Enforce schemas, types, allowlists, and a validation layer on every tool.
  • OWASP Guidance Apply ASVS-style checks to plugin interfaces as if they were public APIs.
  • AuthZ per Action Authorize each tool call against the end user, not the model identity alone.
Attack scenarios
  • URL Manipulation Attackers inject destinations via manipulated tool arguments.
  • Recon and Exploitation Over-broad tools become scanners or exploit runners.
LLM08

Excessive Agency

Agents get more functions, permissions, or autonomy than the task needs. When something goes wrong - or gets injected - the blast radius is large.

Examples
  • Excessive Functionality Agents expose unused tools that attackers can still invoke.
  • Excessive Permissions Plugins run with admin-like scopes “just in case.”
Prevention
  • Limit Plugin Functions Ship only the tools required for the product surface.
  • Plugin Scope Control Restrict each tool to the minimum data and actions.
  • Human-in-the-Loop High-impact steps need confirmation, not full autonomy.
Attack scenarios
  • Tricked Assistant A malicious email or page steers an over-privileged personal assistant.
LLM09

Overreliance

People or systems trust model output without review. Hallucinations become policy, insecure code gets shipped, and legal or security decisions rest on fluent nonsense.

Examples
  • Misleading Info Teams accept incorrect claims because the answer sounds confident.
  • Insecure Code Suggested snippets introduce vulnerabilities that nobody reviews.
Prevention
  • Monitor and Validate Review outputs with consistency checks and domain experts where it matters.
  • Cross-Check Sources Verify critical claims against trusted documentation and tests.
  • Safe Defaults Do not auto-merge or auto-execute model-generated changes.
Attack scenarios
  • Disinfo Spread Attackers exploit LLM-reliant publishers to amplify falsehoods.
  • Plagiarism / IP Risk Unchecked generation creates compliance and copyright exposure.
LLM10

Model Theft

Unauthorized access, copying, or extraction of proprietary models - via APIs, insider leak, or side channels - destroys competitive advantage and may expose embedded sensitive knowledge.

Examples
  • Vulnerability Exploitation Weak auth on model hosting or registries enables download.
  • Central Model Registry Risk A single poorly locked registry becomes a high-value target.
Prevention
  • Access Control Strong authn/authz, short-lived credentials, and audited downloads.
  • Network Restrictions Private endpoints, egress controls, and no public weight hosting.
  • Monitoring Detect unusual inference volume that looks like extraction.
Attack scenarios
  • Model Theft Stolen weights are reused by competitors or attackers.
  • Insider Leak Employee access expands the chance of exfiltration.

Source project: OWASP Top 10 for Large Language Model Applications. Also see the LLM AI Security Checklist for practical controls.