batteriesincluded.com · Questions & Answers

Our AI tool reads supplier emails and attachments. How do we stop instructions inside those documents from making it take the wrong action?

Treat supplier emails and attachments as untrusted data, not as instructions the AI is allowed to obey. Ask your forward-deployed engineer to design that boundary into the application before connecting it to actions such as updating vendor records or preparing payments.

This risk is often called prompt injection. A document might contain text that tells the model to ignore its rules, disclose information, or use different payment details. The text can be malicious, but ordinary quoted instructions can also confuse an insufficiently constrained workflow.

A warning in the AI prompt is not enough. Require controls outside the model:

- Limit extraction to defined fields and validate their format.
- Verify the sender's authority through trusted records, not claims inside the attachment.
- Keep sensitive updates behind an authenticated approval step.
- Restrict available actions and the records each action can affect.
- Retain the source document and a record of proposed and approved changes.

Have the engineer test adversarial examples alongside ordinary supplier correspondence. The test should establish whether the application blocks unauthorized actions, not merely whether the AI recognizes suspicious wording.

Keep EOS as the operational backbone: existing approval authority should remain authoritative. An email must not acquire decision-making power simply because an AI has read it.

For a tool built with Lovable or Replit, review the backend permissions and action handlers, not just the interface. The recommendation is to let AI interpret incoming information while deterministic controls and authorized people govern consequential changes.

Category: Forward-Deployed Engineering

← All questions