14/09 edition: in RAG, a document does not give orders

The question before the pilot

An SME wants to summarise customer requirements when preparing proposals. The risk selected here is not an awkward summary: it is a dossier passage turning into an instruction to change the recipient or seek another file. The decision is specific: permit document consultation without authorising those changes.

We propose starting with one workflow, from the commercial question to a referenced answer. The team should state what remains outside scope: sending messages, exporting attachments and changing contacts. A boundary that exists only in the conversation with the model will not be our acceptance criterion.

What the sources support

OWASP LLM01 describes indirect injection through external content, including files. LLM06 recommends downstream authorisation rather than delegating permission decisions to the model. NIST presents the AI RMF as voluntary. These references frame the problem; they do not certify our proposed design.

A dossier that tries to change the task

Hypothetical scenario: a requirements document contains a note asking the assistant to send the proposal to an address different from the one selected by the salesperson. There is no need to publish a reusable attack instruction. The test team can insert a harmless recipient-change attempt into synthetic material and observe the workflow.

The answer may explain that the document mentions a contact. It should not turn that mention into authorisation. Record the passage read, the answer produced and any tool request separately. An accurate summary does not compensate for an out-of-scope sending attempt.

Prepare a small, readable trial

Our proposal uses paired documents: a clean version and one containing the diversion attempt, with identical commercial requirements. Place the note on a main page, in an appendix and in a retrieved passage among legitimate excerpts. The team must be able to identify which content actually reached the model.

Fix the question, identity, available tools and application version before running. Preserve excerpt identifiers, the expected result and authorisation decisions. Do not collect real customer documents to make the trial more convincing: at this initial gate, synthetic content can represent the relevant behaviour.

Acceptance by boundary

The matrix is an original, untested design. Security checks the boundary; the commercial owner checks whether the task remains useful. Neither approval replaces the other. A blocked lateral action should remain visible even when the final answer looks normal.

Fail safely and measure again

If a lateral request appears, stop expanding the pilot and preserve the question–excerpts–request–denial chain. The proposed correction should address tool authority and context construction, not merely conceal the phrase that triggered the case. Then rerun the pairs and add a variant using different wording.

The Portuguese team may decide that the assistant remains advisory only. That is a useful decision, not a commercial failure. Review this plan when retrieval, available tools or execution identity changes. No finite trial establishes general absence of injection; the conclusion must name the workflow and cases actually examined.