®A general AI model knows a lot about the world but nothing about your prices, policies or products. To build an AI chatbot that answers from your own data, you connect the model to your content so it can look things up before it replies. Done well, it answers common questions instantly and passes the rest to your team.
How a chatbot uses your data
Most business chatbots today use a method called retrieval-augmented generation, usually shortened to RAG. Instead of retraining a model on your documents, you keep the documents separate and fetch the relevant parts at the moment a question is asked.
The flow looks like this:
- Prepare the content. Help articles, product sheets, policy documents and past support answers are cleaned up and split into short passages.
- Index it. Each passage is turned into a numeric representation called an embedding and stored in a vector database or a search index.
- Retrieve. When a user asks a question, the system finds the passages that are closest in meaning.
- Answer. Those passages are given to the language model with instructions to answer only from them, and to say so when the answer is not there.
This approach has practical advantages. You can update an answer by editing a document, with no retraining. You can show the user which source the answer came from. And you can control exactly what the bot is allowed to know.
Fine-tuning a model has its place, for example to teach a particular tone or output format. For factual answers about your business, retrieval is usually simpler, cheaper and easier to keep accurate.
Tools and building blocks
You do not need to build every piece from scratch. A typical stack includes:
- A hosted language model from a provider such as OpenAI, Anthropic or Google, called through an API. Open models that you host yourself are an option when data must stay on your own servers.
- An orchestration library such as LangChain or LlamaIndex, which handles splitting documents, calling the model and chaining steps together. Some teams prefer plain code with a few small helpers, which can be easier to debug.
- A vector store. This can be a dedicated service or an extension to a database you already run, such as pgvector for PostgreSQL.
- Connectors to the places your content lives, such as a help centre, a shared drive, a CRM or a product database.
- A chat interface on your website, inside your app, or in a channel like WhatsApp or Slack.
| Approach | Best for | Main trade-off |
|---|---|---|
| Off-the-shelf chatbot platform | Simple FAQ bots, quick launch | Less control over data flow and behaviour |
| Custom RAG on a hosted model | Most business assistants | Needs development and ongoing tuning |
| Self-hosted open model | Strict data residency needs | More infrastructure to run and maintain |
Content quality matters most
The chatbot can only be as good as the material it searches. Before writing any code, look hard at your content.
Outdated policies, duplicate pages that disagree with each other and answers that live only in someone's head all lead to poor replies. Assign an owner for each content area. Remove or archive old documents. Write short, direct answers to the questions customers actually ask, because those make the best source passages.
Also decide what the bot must never see. Internal pricing notes, personal customer data and staff records should be kept out of the index unless there is a clear reason and proper access control.
Hand-over to a human
A good assistant knows its limits. Plan the hand-over from the start rather than adding it later.
- Low confidence. If retrieval finds nothing relevant, the bot should say it does not know and offer a person.
- Sensitive topics. Complaints, refunds, legal or medical questions can be routed straight to staff.
- User request. A visible option to talk to a person should always be available.
- Context transfer. When the conversation passes to an agent, the full chat history should go with it, so the customer does not have to repeat themselves.
Connecting the bot to your existing help desk or CRM makes this smooth. The agent sees the conversation, the bot's suggested answer and the sources it used.
Testing before and after launch
AI answers vary, so testing needs more structure than clicking around.
Build a test set of real questions, collected from support emails, chat logs and sales calls. For each one, write the expected answer or the expected behaviour, such as hand-over. Run the full set every time you change the prompt, the model or the content, and compare the results.
Check for these failure types in particular:
- Answers that sound right but are not supported by any source
- Answers drawn from an outdated document
- Replies that go beyond the bot's role, such as giving legal advice
- Attempts by users to make the bot ignore its instructions
After launch, review a sample of real conversations every week. Track how often people ask for a human, which questions get no answer, and which answers users rate poorly. These gaps tell you what content to write next.
Checklist before you start
- Have you picked one clear job, such as answering product questions or order status?
- Is the source content current, owned and free of conflicting versions?
- Do you know which data may be sent to an AI provider under their terms?
- Is there a defined path to a human, with context passed across?
- Do you have a test set of real questions and a way to rerun it?
- Have you agreed how success will be measured?
Frequently asked questions
Do I need to train my own AI model?
Usually not. Most business chatbots use an existing model and supply your content at question time through retrieval. Training or fine-tuning is only worth it for specialised needs.
Can the chatbot give wrong answers?
Yes. Retrieval, clear instructions, visible sources and careful testing reduce the risk a great deal, but they do not remove it. Keep a person involved for anything with financial, legal or safety impact.
Is my data safe with an AI provider?
It depends on the provider's terms and your settings. Read their data use and retention policies, send only what is needed, and consider a self-hosted model if your data cannot leave your own systems.
For more on picking a first use case, read our guide on adding AI to your app. If you are ready to build, our AI development and automation team can design, test and connect a chatbot to the systems you already use.