RAG Pipeline Prompt Injection Tests Before Launch
A RAG pipeline is secure only if retrieved documents stay in the data lane. If a retrieved page can tell the model to change goals, reveal secrets, or call tools, the retrieval layer is part of the attack surface.
Quick Answer
- Seed hostile documents into the corpus.
- Test direct and obfuscated override instructions.
- Check whether retrieved text can trigger tool calls.
- Verify citations point to the hostile source.
- Log failures as security bugs, not prompt quirks.
Test Document Patterns
Add documents that try to:
- override system instructions
- ask for hidden prompts or credentials
- redirect the answer to a competitor
- instruct the model to call a tool
- bury malicious text in tables, comments, or footnotes
What Success Looks Like
The assistant should answer the user's question using the retrieved facts while ignoring instructions inside the document. It can mention that a document contains suspicious instructions, but it should not follow them.
Tool-Use Boundary
RAG gets much riskier when the same agent can send email, edit records, or write files. Retrieved text should never be enough to authorize a tool call.
FAQ
Is prompt hardening enough?
No. You need retrieval labeling, tool permission boundaries, validation, logging, and adversarial tests.
Should hostile documents be in CI?
Yes. Keep a small regression suite so future prompt or retrieval changes do not reopen old failures.
What is the most common failure?
The model follows an instruction in a retrieved document because the context did not clearly separate data from instructions.