Which model do you use?
+
Whichever fits the task, and the choice stays reversible. The loop is written against a provider-portable interface, so switching is a configuration change plus a re-run of the evaluation set. Locking a system to one provider is a decision worth avoiding while it is still cheap to avoid.
Does our data go to a model provider?
+
Only what you decide to send. Retrieval runs in your infrastructure and access rules are applied there, so the model sees the passages your policy allows and nothing else. Where nothing may leave at all, open models run on your own hardware — it costs quality, and it is sometimes the right trade.
What stops it doing something expensive or irreversible?
+
It cannot call anything that has not been registered as a tool, and every tool declares whether it runs unattended or waits for a person. The boundary is data the system enforces, not an instruction in a prompt — a prompt is a request, and a request can be argued with.
How do we know it is getting better rather than different?
+
An evaluation harness with a fixed regression set runs in CI, and a change that lowers the score fails the build. Without one, every prompt change is a matter of opinion and the opinion belongs to whoever demoed it most recently.
We already have a RAG prototype that underperforms. Can you take it further?
+
Usually. The fix is rarely the model: it is chunking, re-ranking, and evaluating retrieval separately from generation so you can see which half is wrong. That diagnosis is normally the first week.