LLM08: Vector and Embedding Weaknesses

Here is an example of Python code that is vulnerable to vector and embedding weaknesses in RAG:

🥺 Vulnerable Code

def answer(question, user):
    query_vector = embed(question)

    # Vulnerable: one shared index, no tenant filter, similarity decides everything
    chunks = index.query(vector=query_vector, top_k=8)

    context = "\n\n".join(chunk["text"] for chunk in chunks)
    return llm.complete(f"Context:\n{context}\n\nQuestion: {question}")

Similarity search does not know about permissions. Ask about pricing and the nearest neighbours may come from another tenant's contract, because nothing in this query restricts the search to documents this user is allowed to read. Two other problems hide in the same code: deleting a source document does not delete its embedding, and embeddings themselves can be inverted well enough to reconstruct sensitive text.

😎 Secure Code

Here is a version of the same code that is secured against vector and embedding weaknesses in RAG:

def answer(question, user):
    query_vector = embed(question)

    chunks = index.query(
        vector=query_vector,
        top_k=8,
        namespace=f"tenant-{user.tenant_id}",              # hard partition
        filter={"tenant_id": {"$eq": user.tenant_id},
                "trust": {"$eq": "curated"}},
    )

    # Independent check against the source of truth, after retrieval
    allowed = acl.filter_documents(user, [c["metadata"]["doc_id"] for c in chunks])
    chunks = [c for c in chunks if c["metadata"]["doc_id"] in allowed]

    context = "\n\n".join(f"[{c['metadata']['doc_id']}] {c['text']}" for c in chunks)
    return llm.chat(messages=to_messages(context, question), citations_required=True)

A per-tenant namespace makes cross-tenant retrieval structurally impossible rather than filter-dependent, and re-checking document ACLs after retrieval catches permission changes the index has not caught up with yet. Delete embeddings when the source document or its permissions change, encrypt the store, keep untrusted chunks in a separate trust tier, and plant a canary document per tenant so a leak shows up in testing rather than in a customer's chat window.