AriesDevs engineering team Engineering notes
7 min read
A useful business chatbot is not a general-purpose assistant that knows everything. It is a focused tool that answers specific questions from your own content, hands over to a person when it is unsure, and gets measurably better over time. Here is how to build one that your customers and staff will trust.
Step 1: pick one narrow use case
Start where questions repeat: customer support, internal HR and IT policy, product documentation, or sales qualification. A narrow scope makes the assistant more accurate and the project easier to finish. You can widen it later.
Step 2: prepare the knowledge
The assistant is only as good as the content behind it. Collect the current versions of your help articles, policies and product sheets, remove outdated copies and decide who owns each source. Most chatbot problems that look like AI problems are content problems.
Step 3: choose the approach
- Retrieval-augmented generation (RAG): the assistant looks up passages from your documents and answers from them, with citations. This is the usual choice for company knowledge.
- Prompt engineering: clear instructions and examples, often enough for simple, rule-based conversations.
- Fine-tuning: training a model on your examples. It is rarely needed to start and adds cost and maintenance.
Step 4: design for trust
- Show the sources behind each answer so users can check.
- Say "I am not sure" and offer a human hand-off instead of guessing.
- Keep personal and confidential data out of prompts unless it is needed, and understand where your provider stores it.
- Log conversations so that you can review failures, with consent and clear retention rules.
Step 5: measure before and after launch
Write 50 to 100 realistic questions with the expected answers, and score the assistant against them before launch and after every change. Track how many conversations are resolved without a person, how often users rate answers as helpful, and how often the assistant correctly declines. Review a sample of real conversations every week.
What it costs
A focused first version, with content preparation, retrieval, a chat interface, evaluation and a hand-off flow, typically takes four to eight weeks. Ongoing costs include model usage, hosting and a small allowance for content updates and evaluation. Costs depend on traffic and the model you choose, so ask for a range with the assumptions written down.
If you would like an honest view on whether a chatbot is the right fit for your use case, book a call: we will tell you when a simpler solution would serve you better.
