AI Chatbot Knowledge Base Setup

A chatbot knowledge base is a search library, not a training run

Most people picture setting up a chatbot’s knowledge as teaching the AI, as if your documents get absorbed into the model. A well-built business chatbot does not work that way. The language model stays exactly as it is. What we build is a searchable library of your approved content, and every time a visitor asks something, the system looks up the most relevant passages and hands them to the model with a standing instruction: answer from these, and admit it when they do not cover the question.

The pattern is usually called retrieval-augmented generation, or RAG. It matters for three practical reasons. You can change an answer by editing a document, with nothing retrained. Each reply can point back to the page or file it came from, so staff can check it. And when the library holds no relevant passage, the bot has a principled reason to say it does not have that information instead of improvising.

It also means the chatbot is mostly as good as its library. A strong model reading a messy, contradictory pile of files will give confident, contradictory answers. Everything described below is the unglamorous work that prevents that outcome: choosing sources, cleaning them, splitting them sensibly, labeling them, keeping them current and testing the whole arrangement against real questions.

Step one: an inventory of what the bot is allowed to know

Before anything is uploaded, we list every candidate source with you and decide, one at a time, whether it belongs. Businesses tend to over-include. The instinct is to hand over everything so the bot can answer anything, but each extra file is one more chance for an outdated price sheet or an internal note to reach a customer.

Each source also gets an owner: the person at your company who confirms it is accurate and who hears about it when it drifts. Documents without an owner are the first to go stale, so we would rather leave a file out than load one nobody is responsible for.

Cleaning documents before they are loaded

Do-it-yourself setups usually skip this step, and it is where most wrong answers originate. These are the problems we look for and fix before a single passage is indexed:

• Superseded versions. An old return policy and the current one sitting side by side. Retrieval will happily surface the older document if its wording happens to match the question more closely. We keep one version in the index and archive the rest outside it.
• Contradictions between sources. The website says same-week service while a PDF says two to three weeks. We flag every conflict and ask you which is true; no retrieval setting can settle a business fact on your behalf.
• Boilerplate noise. Navigation, sidebars, legal footers and related-post blocks come out. Left in, the same footer text shows up in hundreds of passages and crowds genuine content out of the search results.
• Image-only files. Scans need optical character recognition, and OCR output needs a human spot-check, particularly for part numbers, dates and figures.
• Tables and diagrams. A compatibility or rate table extracted as raw text often loses which value belongs to which row. Important tables are rebuilt as structured text, one row per record, so the meaning survives.
• Remarks meant for staff. Notes like “only offer this if they push back” have a way of hiding inside otherwise public documents. They are removed.

You receive a short cleanup report listing what was removed, merged or rewritten and why, so nothing vanishes from your content without a record.

How the library is split, labeled and searched

Chunking

Documents are divided into passages, called chunks, because retrieval returns passages rather than whole files. Chunks that are too small lose context: a sentence saying “this does not apply to commercial accounts” is useless if the chunk never says what “this” refers to. Chunks that are too large dilute the match and waste the model’s limited context window. We split along the document’s own structure, such as headings, numbered steps and table rows, instead of cutting every so many characters, and we carry the section heading into each chunk so it stays self-explanatory. A small overlap between neighbouring chunks keeps a sentence sitting on a boundary from being orphaned.

Metadata

Every chunk carries labels alongside its text. The ones we find most useful are the source URL or file path, the document owner, an effective or review date, the intended audience (public, customers or staff) and business tags such as product line, location or service type. Metadata lets the system filter before it searches: a question asked on your commercial pages can be restricted to commercial content, and material past its review date can be excluded or flagged for its owner.

Search method

Pure vector search, which matches meaning, is good with paraphrased questions but can miss exact strings such as model numbers, SKUs and policy codes. Keyword search catches those strings but stumbles on paraphrase. For most business libraries we use a hybrid of both, and where the volume justifies it, a reranking step that re-scores the top candidates before they reach the model. Where the index lives, whether that is a hosted vector service, a Postgres database with a vector extension or the retrieval feature built into your existing chatbot platform, is decided during scoping based on where your data should reside and what you already run.

Keeping the knowledge base current after launch

A knowledge base that was accurate on launch day and never touched again becomes a slow-moving liability. Freshness has to be designed in from the start:

1. Change detection. Where the source system can announce changes, and many document and help-desk platforms offer change notifications or feeds, updates flow in as they happen. Where it cannot, a scheduled re-crawl compares a fingerprint (a hash) of each document against the previous run and re-processes only what changed.
2. Deletions propagate. Removing a document from the source must remove its chunks from the index. It sounds obvious, yet it is a frequent gap: the file is gone from the drive while its passages keep answering questions for months.
3. Review dates. Documents with a review date send a reminder to their owner, and overdue material can be down-ranked automatically so fresher sources win.
4. A gap log. Each question the library failed to support gets written down. Reviewed regularly, that list is the most reliable guide to which content you should write next.

We also record which document version each answer drew on. If a customer disputes something the bot told them, you can see precisely which source it relied on at that moment.

Public answers and staff-only material live in separate indexes

Some clients want one assistant for customers and another for employees. The safe design keeps two separate indexes, not a single index with a staff-only tag that a prompt is expected to respect. Instructions given to a model can be talked around by a determined user; a document that was never loaded into the public index cannot leak out of it.

Internal assistants built on a shared drive raise a second question: per-user permissions. If a manager can open a folder and a technician cannot, the assistant should not hand the technician a tidy summary of it. Depending on the platform, that means filtering retrieval by the asking user’s group memberships, or limiting the internal assistant to content everyone in the company can already see.

Personal data deserves the same caution. Customer records, signed contracts and HR files rarely belong in a chatbot library at all. When a question needs account-specific information, such as the status of an order, a live lookup against the system of record with proper authentication is the better tool; our work on chatbot database integration covers that pattern.

Proving the answers before a customer sees them

We do not judge a knowledge base by chatting with it for ten minutes. Before launch we assemble an evaluation set: real questions drawn from your inbox, call notes, reviews and sales conversations, each paired with the correct answer and the document that should support it. The set deliberately includes awkward cases, such as badly phrased questions, questions that mix two topics and questions your library genuinely cannot answer.

Each test question is scored on separate points, because each one fails for a different reason:

The same set is re-run after every significant content or configuration change, so fixing one area cannot quietly break another. It is part of what you own at handover, and your team can keep adding questions to it.

How EVOTECH sets up a chatbot knowledge base

1. Free consultation by phone or video about what the bot should answer, who will be asking and where your information lives today.
2. Source inventory, with owners, inclusion decisions and access boundaries agreed in writing.
3. A fixed-scope written quote naming the sources, the sync method, the retrieval stack, the size of the test set and what is handed over.
4. Cleanup and ingestion, with the cleanup report delivered to you.
5. Evaluation against the agreed question set, results shared, fixes made before launch.
6. Launch with the gap log switched on, then a review of unanswered and low-confidence questions once real traffic has arrived.

Behind the work is EVOTECH IT LLC: a Houston-area team in business for 20+ years, rated 5.0 by Google reviewers, serving clients in every state over phone and video. We can set up the knowledge base inside a chatbot platform you already pay for, or build it into a custom assistant through our AI chatbot development service. Call (832) 359-2425 and we can walk through your sources together.

Related services

Frequently asked questions