LLM Data Leakage, Model Extraction and Tenant Isolation

Use these techniques only on systems you own or are explicitly authorized to test. Run high-impact checks in an isolated environment with synthetic data, test credentials, controlled callback services, and rollback procedures.

Data leakage tests

• Use two controlled tenants with unique canary records. Probe tenant A for tenant B’s canary through search, paraphrasing, completion, citations, conversation IDs, exports, and shared caches.
• Inspect browser storage, analytics, traces, support tools, model-provider logs, and error messages for prompts, API keys, personal data, or retrieved documents.
• Probe system prompt and hidden context disclosure, but prioritize secrets and user data over prompt text that has no security impact.
• Check whether deleted conversations, files, embeddings, and memory remain retrievable after the documented retention period.

Model abuse and extraction

• Test whether anonymous or low-cost users can make unbounded requests, recover logits or excessive metadata, or systematically clone behavior.
• Measure per-user and per-tenant token, request, concurrency, and spend controls.
• Look for membership-inference risk when the application exposes confidence, nearest-neighbor scores, or overly specific completions.

Evidence to collect

• Use synthetic canaries, never real customer secrets, while proving cross-boundary access.
• Record tenant, user, conversation, document, cache key, vector namespace, model, and provider request ID.
• Confirm whether leakage occurred in retrieval, prompt construction, model generation, logging, or rendering so the right control is fixed.

Practical test examples

Canary plan

Tenant A secret: TA_CANARY_6f81c2
Tenant B secret: TB_CANARY_9a347e

Probe paths:
1. direct question and paraphrases
2. guessed conversation/document IDs
3. semantic retrieval synonyms
4. export and citation endpoints
5. cache behavior before and after logout
6. deleted-data retest

Useful tools

• Burp Suite or Caido – intercept chat requests, alter hidden parameters, replay conversations, and compare authorization behavior.
• curl, HTTPie, or Postman – replay API calls with different users, model IDs, tool settings, and conversation identifiers.
• mitmproxy – inspect streaming APIs, WebSocket traffic, agent callbacks, and model-provider requests in a controlled test.
• Cloud and application log search – trace canaries across prompts, retrieval, caches, analytics, and provider calls.
• Presidio or another DLP scanner – detect personal data and secrets in model inputs and outputs.

How to fix and retest

1. Scope retrieval, caches, memory, logs, and exports by server-verified tenant and user identity.
2. Minimize data sent to models, redact secrets and personal data, encrypt storage, and enforce deletion and retention.
3. Apply quotas, abuse detection, response shaping, and rate limits to reduce extraction and denial-of-wallet risk.

Retest checklist

• Repeat the original proof with the same identity, state, model version, and input source.
• Try equivalent variants through every other input channel and fallback model.
• Confirm the control is enforced by the backend or runtime, not only by prompt wording.
• Check logs and alerts, then add the case to a versioned regression suite.

Conclusion

AI security issues are easiest to manage when every piece of content is treated according to its real trust level and every consequential action is controlled outside the model. Keep the test evidence reproducible, limit blast radius, and rerun the suite whenever the model, prompt, retrieval pipeline, tool, server, or policy changes.