GPT-Powered Website Chatbots, Engineered Past the Demo

Following one question through a GPT-powered chatbot

From outside, a GPT chatbot looks like a text box that talks back. Inside, one visitor message passes through several stages, and nearly every quality problem can be traced to one of them.

1. Widget. The browser script sends the message, a session ID and the current page URL to our endpoint. It holds no secrets and never calls the model provider directly.
2. Gateway. The server checks the session, applies per-visitor rate limits, rejects oversized input and screens for content the chatbot is set to refuse outright.
3. Routing. A fast classifier or small model labels the turn: a question needing knowledge, a step in an intake flow, a request for a person, or small talk.
4. Retrieval. The knowledge index is searched for passages relevant to the question, filtered by metadata such as location or service line.
5. Prompt assembly. System instructions, retrieved passages, a compact summary of earlier turns and the definitions of any permitted tools are combined into one request.
6. Model call. The request goes to the GPT model chosen for answering, and output streams back so words start appearing before the full reply is finished.
7. Post-checks. Links must point to your domain, any quoted figure must match an approved value, and a promised action must correspond to a tool call that actually succeeded.
8. Logging. The turn, the passages used, any tool calls and the timings are stored so each answer can be traced to its sources later.

That final step is the one teams skip and later regret. When a customer insists your chatbot said you were open on Sunday, you want to see the exact passage it relied on rather than guess.

Retrieval sets the ceiling on answer quality

Most wrong answers from a GPT-powered chatbot are not pure invention. They are the model faithfully summarizing the wrong passage: an old blog post about a discontinued service, a page for an office that closed, a policy updated in one place but not another. Retrieval is where those errors start, so it gets the most engineering attention.

Choosing sources deliberately

We don’t index the entire website by default. We build an explicit source list, mark each item as authoritative or supporting, and leave out archives, press releases and anything with a date-sensitive claim unless someone maintains it.

Chunking by idea

Documents are split into passages sized around a single idea, and each passage carries its headings, so a fragment such as “Yes, including weekends” still knows which question it answers.

Hybrid search

Vector embeddings find passages that mean the same thing as the question even when the wording differs. Keyword search catches exact terms like model numbers, part names and ZIP codes, which embeddings can blur. We run both, merge the results and re-rank them before anything reaches the model.

Metadata filters

Every passage is tagged for location, service line and audience, so a question asked on the commercial services page never pulls a residential policy into the answer.

A threshold for silence

If nothing relevant scores high enough, the model is told explicitly that no approved information exists for this question. It then says so and offers a person, instead of filling the gap from general knowledge.

Function calling without handing over the keys

GPT models can call functions. Given a list of tool definitions, the model decides when one is needed and writes structured arguments for it. That is how a chatbot moves from answering to doing, and it is also where a careless build creates real risk, because those arguments are shaped partly by whatever the visitor typed.

Our rule is simple: the model proposes and the server decides. Every tool is narrow, every argument is validated, and nothing irreversible happens until the visitor confirms it in the chat.

The model never receives database credentials, never writes free-form queries and cannot call a tool that isn’t registered for the page it’s on. When it sends bad arguments, the server returns an error message the model can read, and the chatbot asks the visitor to clarify instead of failing silently.

Picking the GPT model and tuning its settings

Which GPT model to use is a genuine engineering choice, and the newest, largest one is not automatically right for a website chatbot. These are the settings we decide on for each build:

We also keep the model replaceable. OpenAI’s GPT models are the default in many of our builds, but retrieval, tools, tests and instructions sit behind an interface that can point to another provider when there is a good reason, such as a customer requirement about where data is processed.

Testing software that never phrases things the same way twice

Conventional tests check that one input always produces one output. A GPT-powered chatbot will word a correct answer differently from run to run, so tests must judge outcomes rather than exact strings. Before launch we assemble an evaluation set from your real inquiries, then add cases your inbox is unlikely to contain:

• Fact checks: questions with one correct answer in the knowledge base, graded on whether the reply contains that fact and nothing contradicting it.
• Should-decline cases: requests for advice, unpublished prices or services you don’t offer, graded on whether the chatbot declines and routes properly.
• Adversarial prompts: attempts to extract the instructions, swap the persona or bait the chatbot into criticizing a competitor.
• Multi-turn flows: complete intake conversations, including visitors who change answers midway or give a ZIP code outside your area.
• Tool behavior: cases where the only correct response is a specific function call with specific arguments.

Grading mixes automated checks, model-assisted scoring against a written rubric, and human review of anything borderline. The full set reruns on every change, whether that is a new knowledge source, an edited instruction or a model upgrade. If a change fixes one answer and quietly breaks three others, the regression run catches it before your customers do.

Running it in production after launch

A GPT-powered chatbot is a live service that depends on an outside API, so it needs the same operational care as any production system.

What we monitor

• Latency, including the time until the first words appear
• Error rates from the model provider and from each integration
• Tokens per conversation, which exposes runaway sessions and prompt bloat
• Questions that fell below the retrieval threshold, grouped by topic
• Handoff volume and reasons, and how many captured leads your team actually contacted

Provider outages

Every large API has bad days. The chat window detects failures and switches to a plain fallback: a short message, your phone number and a simple form that still delivers the lead. Visitors get a slower path, not a broken widget.

Spending controls

Usage limits on the API account, alerts at thresholds you choose and per-session caps turn an unexpected traffic spike into a notification rather than a surprise invoice.

Change management

Instruction and knowledge updates travel through the same pipeline as code: edited in version control, run against the evaluation set, then deployed. Staff can propose changes through a simple content sheet without touching the prompt itself.

Where a GPT chatbot is the wrong tool

Some jobs are better served by a deterministic flow with fixed questions, fixed branches and fixed wording, and we recommend one when that is the case:

• Taking payments or handling card data, which belongs on a payment processor’s hosted page.
• Regulated disclosures that must be delivered word for word.
• Short, predictable choices, such as picking one of three appointment types, where a few buttons beat typing.
• Situations where an occasional misunderstanding is unacceptable and no human reviews what the chatbot does.

In practice the strongest builds are hybrids. GPT handles what forms are bad at, which is understanding a messy, free-text description of a problem, and a deterministic flow takes over for the steps that must be exact. You get natural conversation where it helps and predictable behavior where it matters.

What EVOTECH hands over when the build is done

You should be able to take your chatbot to another developer without starting from scratch. At handover you receive:

• Source code for the chat window and the server, in a repository you own
• System instructions and tool definitions, with their change history
• The approved knowledge base and its source list
• The evaluation set and the most recent test results
• A runbook covering deployment, key rotation, fallbacks and adding knowledge
• Admin access to the model provider account, hosting and analytics, all in your company’s name

Work begins with a no-charge call by phone or video, and whatever scope we agree on is written into a fixed quote. For the business-side view of the same project, see our custom ChatGPT chatbot for your website page. If booking is the main job, our page on an AI chatbot with calendar booking goes deeper, and AI integration covers connecting models to the rest of your software.

Related services

Frequently asked questions