batteriesincluded.com · Questions & Answers

Three months after our embedded AI build, how do we know whether it actually improved the business?

Compare the result with the business outcome you intended to improve, not just the fact that the tool was delivered or people logged in. Ask the process owner and the engineer to review both the numbers and the experience of the employees doing the work.

Start with the closest useful measure. Depending on the workflow, that could be time spent on a recurring task, the amount of rework, or the time work waits for a decision. Compare it with a reasonable baseline from before the change, using the same definition and period where possible. If no baseline exists, say so and start measuring now rather than inventing a return.

Also check for costs the headline measure misses. Has someone else inherited extra review work? Are errors being caught before they affect customers? Are employees using a workaround because the tool does not fit the process? A small sample of actual work and conversations with the people involved can reveal issues that usage counts cannot.

Bring the findings into the company's normal review rhythm. If the outcome matters to an existing EOS Scorecard measurable, use that connection, but do not assume every project needs a new measurable. Decide whether to keep, adjust, or stop the workflow, assign an owner for the next action, and review again. Structured systems can always be improved, including after a build is complete.

Category: Forward-deployed engineering

← All questions