Find what will break before your customers do.

No anonymous path reaches the model or the tools. Service accounts have their own identities, not a shared key.

Permissions are checked at retrieval time with the user's identity, not assumed from the prompt. A user cannot ask their way into someone else's records.

Prompt templates, tool lists and model settings are changed through a controlled path with an audit trail, not from the chat window.

API keys and connection strings live in the runtime's secret store. A search of the built front end and git history finds none.

A written inventory: source, owner, sensitivity, how it is refreshed, and whether customer or employee personal information is in it.

Personal, confidential and public information are labelled, and the system's access to each tier is deliberate rather than "everything it can reach".

For a set of real questions, someone has looked at what was retrieved and confirmed it was relevant, current and complete enough to answer from.

When the system answers from an outdated document, there is an owner and a process to correct the source, not just the answer.

You know which providers receive prompts and documents, where they process them, how long they keep them, and whether they train on them. This is written down and matches your privacy commitments.

Someone has written why this is a model task rather than a rule, a search or a form, and what simpler approach was considered.

Prompt changes are reviewed and can be rolled back. You can say which version answered a given request.

The system declines, asks or escalates in ways you have specified and tested, rather than improvising.

The model version is pinned. Upgrades go through the evaluation set before reaching users.

Each run records the steps taken, the tools called and their inputs and outputs, so a person can follow what happened.

Each tool exists because a workflow needs it. Nothing is exposed "in case".

A tool checks the acting user's rights itself. The model's request is not treated as authorisation.

Sending, paying, deleting, publishing and changing records go through a confirmation step or a human approval, with the rule written down.

Amounts, recipients, record identifiers and query scopes are checked against limits before anything runs.

No tool grants permissions, creates credentials or changes the tool list. Untrusted content (documents, web pages, emails) cannot inject instructions that reach a tool unchecked.

Structured outputs are validated against a schema. Free-text outputs that drive decisions are checked by rules, a second model pass, or a person, and you know which.

Dozens of real or realistic inputs with expected outcomes, including hard cases and cases the system should refuse. It runs before every release.

Hallucinated facts, wrong retrieval, partial answers, refusals, timeouts and tool errors each have a defined user-facing behaviour.

Tests show the system does not reveal other users' data, secrets, or internal instructions when asked directly or indirectly.

Uncertainty is surfaced to the user in words, not hidden behind a confident answer.

Inputs, retrieved context, model version, tools called, outputs, latency and cost are recorded, with personal information handled per your retention rules.

A sample of real interactions is reviewed on a schedule, and the results are compared with the pre-launch evaluation.

A feedback control exists, flags reach a named owner, and recurring problems feed the evaluation set.

Error rate, latency, refusal rate, tool failures and cost per day have thresholds that page a person.

You can see which workflows and which accounts drive spend, and you would notice a runaway loop within the hour.

Every user-facing flow has a route to human help that does not depend on the model working.

A short written list of decisions the system may only draft or suggest, never make. The interface reflects it.

When the model, retrieval or a tool is down, users get a clear message and a working alternative, not a spinner or an invented answer.

Escalated cases go to a queue a named person watches, with a response time you have agreed.

You know the typical and worst-case response time for real requests, and the interface handles the slow ones honestly.

A budget per user or per day exists, with rate limits and caps that stop runaway spend.

Repeated work is cached where safe, retries are bounded, and a failure cannot multiply cost.

The system has been run at a multiple of expected concurrency, and you know where it degrades.

A release is a versioned artefact, deployed the same way each time, and can be rolled back in minutes.

Authentication, authorisation, data handling and the tool layer have automated tests that run before release.

Someone has read the authentication, authorisation and data-access code the tools generated, and dependencies are pinned and checked for known issues.

One person is accountable after launch, with hours each month for monitoring, fixes and improvement, and a backup when they are away.

Where it is used, people are told an AI system is involved, what it can do, and how to get help.