Deploying a website AI assistant is more than adding a chat widget. This guide provides a reusable production checklist covering architecture, API key protection, authentication, rate limiting, streaming, error handling, human escalation, analytics, data retention, and the browser-to-backend workflow.
Overview
A reliable website AI assistant usually has four layers: the browser interface, an application backend, an AI model or orchestration service, and the business systems that supply trusted information or perform actions. Keeping these layers separate makes the system easier to secure, test, monitor, and change.
The browser should handle presentation and user interaction. It should not contain provider API keys, privileged credentials, internal prompts, or unrestricted access to business APIs. Instead, the browser sends a user message and relevant session information to your backend over an encrypted connection. The backend authenticates the request, applies policy and rate limits, retrieves approved context where needed, calls the model, and returns a controlled response.
For a retrieval-augmented generation (RAG) chatbot, the backend can also search an approved knowledge base before generating an answer. For a customer support chatbot, it may create a ticket or transfer the conversation to a human. Each action should have a defined permission, validation rule, timeout, and fallback.
Before deployment, define what the assistant is allowed to do. Write down its supported tasks, prohibited requests, escalation conditions, data sources, retention period, and success measures. This operational definition is as important as the model selection. For prompt structure and evaluation ideas, see the prompt engineering playbook for customer support chatbots.
Checklist by scenario
For a public, unauthenticated assistant
- Protect credentials: store AI provider keys and other secrets on the server or in a managed secrets store. Never place them in browser JavaScript, HTML, or a public repository.
- Limit the surface area: expose only the backend endpoint and operations the widget needs. Do not proxy arbitrary URLs, tools, or model parameters from the browser.
- Apply rate limits: limit requests by IP, session, account where available, and sometimes by route. Add request-size and conversation-length limits to control abuse and unexpected cost.
- Set content boundaries: provide a clear system instruction, restrict the knowledge sources the assistant can use, and test attempts to override its instructions.
- Offer a fallback: show a useful error message, contact route, or human support option when the model or a dependency is unavailable.
- Minimise data collection: do not request personal information unless it is necessary for the task. Tell users what information is stored and why.
For an assistant with signed-in users
- Authenticate on the backend: do not trust a user ID, role, or account number supplied only by the browser. Derive identity from a validated session or token.
- Enforce authorisation: check permissions for every retrieved record and every action. A model should not be treated as an access-control layer.
- Separate tenant data: include tenant or account boundaries in database queries, retrieval filters, logs, and support workflows.
- Control user context: pass only the account information needed for the current task. Avoid sending complete profiles or private records to the model by default.
- Audit sensitive actions: record who requested an action, what was attempted, whether it was approved, and what result was returned.
For an assistant that uses tools, APIs, or webhooks
- Define strict schemas: validate tool arguments for type, format, length, and permitted values before calling an external service.
- Use least privilege: give each integration only the credentials and operations it needs. Prefer read-only access where writing is not required.
- Require confirmation: ask the user to confirm consequential actions such as cancelling an order, sending a message, changing an account, or submitting a form.
- Make actions idempotent: use an idempotency key or equivalent protection so a retry does not create duplicate tickets, payments, or messages.
- Handle dependency failure: define timeouts, retries with limits, clear error states, and a route to human support. More detail is available in Connect an AI Chatbot to APIs and Webhooks.
For a streaming chat experience
- Choose a supported transport: use the streaming method supported by your backend and hosting environment, and verify how it behaves through proxies and content delivery layers.
- Stream only safe output: do not expose internal tool arguments, hidden instructions, stack traces, or unfiltered intermediate content.
- Support interruption: allow the user to stop generation and ensure the backend can cancel or abandon work where practical.
- Finish states clearly: distinguish a completed answer from a timeout, connection drop, moderation block, or partial response.
What to double-check before launch
Browser-to-backend flow: confirm that the widget sends messages only to your intended origin, uses appropriate cross-origin rules, and does not expose secrets in source maps, network requests, or client-side configuration. Check that session cookies or tokens use secure settings suitable for your authentication design.
Prompt and context controls: define token budgets, maximum history, and retrieval limits. Long conversations can increase latency and cost while making relevant instructions harder to preserve. The LLM token limits guide provides a useful review point for these decisions.
Security testing: test prompt injection, data exfiltration attempts, malicious links, oversized inputs, repeated requests, and tool misuse. Include cases where retrieved documents contain instructions intended to manipulate the assistant. Use the prompt injection testing checklist as part of pre-launch review.
Observability: measure request volume, latency, errors, token usage where available, fallback frequency, tool failures, escalation rate, and user feedback. Redact secrets and unnecessary personal data from logs. Create an alert for the failures your team can act on rather than collecting every possible field. See AI chatbot observability for a broader monitoring workflow.
Human escalation: specify when the assistant should stop trying to answer. Escalation may be appropriate after repeated misunderstanding, for sensitive account issues, when confidence is low, or when the user explicitly requests a person. Pass the human agent a concise, relevant transcript while respecting retention and access rules.
Release controls: keep prompts, retrieval settings, tool schemas, and application code versioned. Test changes against a fixed set of representative conversations before release, and maintain a rollback path. A prompt versioning workflow can help separate a prompt change from an application change.
Common mistakes
- Calling the model directly from the browser: this exposes credentials and makes policy enforcement difficult. Put the provider call behind your backend.
- Trusting model output as a command: generated text is not validation. Validate every argument and enforce permissions in application code.
- Logging everything: full transcripts can contain personal, confidential, or authentication data. Define a retention policy, redact sensitive fields, and restrict log access.
- Launching without an escalation route: a chatbot that cannot resolve an issue should make the next step obvious rather than repeating itself.
- Ignoring partial failures: model calls, retrieval services, authentication systems, and business APIs can fail independently. Test each failure path and return a stable user-facing response.
- Changing several variables at once: if the prompt, model, retrieval index, and interface all change together, it becomes difficult to identify the cause of a quality or cost change.
- Treating launch as the end: production conversations reveal new questions, abuse patterns, and integration edge cases. Reserve time for review and improvement.
When to revisit
Review this checklist before each major launch, after moving hosting providers, when adding authenticated user context, and whenever the assistant gains a new tool or data source. It is also worth revisiting before seasonal planning cycles, when traffic or support volume is expected to change, and after a security or reliability incident.
Set a recurring review at a practical interval, such as quarterly, and assign an owner for each area: application security, data governance, support operations, infrastructure, and product quality. Recheck provider limits, browser and hosting behaviour, authentication libraries, retention requirements, and internal escalation routes whenever those workflows change.
For an action-oriented final pass, ask the team to demonstrate five journeys in a staging environment: a normal answer, an authenticated request, a rate-limited request, a failed dependency, and a human escalation. Confirm that secrets remain server-side, permissions are enforced independently of the model, logs are appropriately redacted, and the user always receives a clear next step. Once those checks pass, deploy gradually, monitor the agreed signals, and keep the previous version available for rollback.