An AI assistant that does not invent

We put an AI assistant on this site a few weeks ago. It answers questions about our services, our process and the markets we work in, in English and Arabic, and it can hand you to a booking link or take your details.

The model was the easy part. The hard part — and the only part worth writing about — is that a language model will confidently tell a visitor about a client who never existed.

The failure mode

Ask an assistant about a company with no grounding, or loose grounding, and it will produce a fluent paragraph. It will invent a case study. It will invent a percentage improvement. It will invent a certification, because certifications are exactly the kind of thing a language model has read about many times.

The output is not obviously wrong. It has the cadence of a real answer. And it reaches a prospect.

That matters far more than it usually would, because this particular site is built on not overclaiming. We removed invented client names from an earlier build. We publish a floor price rather than a rate card specifically so we are not competing on numbers we cannot substantiate. And then the obvious thing to do is bolt on a chatbot that will confidently invent a case study on a Tuesday.

Four rules, in the system prompt

The knowledge base is generated at build time from the same files that render the pages, so it cannot drift from the site. Everything the bot knows about us, it reads from our own content. Then four rules do the rest.

Answer only from the reference. If it is not there, say so and offer to pass the question on. This is the rule that does most of the work.

Never invent a fact. No invented clients, projects, metrics, percentages, prices, discounts, timelines or certifications. The instruction names the specific things, because a general "be accurate" instruction is much weaker than a list of the things the model is likely to invent.

Never quote a price. We publish a £1,000 floor and no rate card. The bot is instructed to hold that line and offer a call instead. This one matters more than it sounds: asked directly for a price, a model will happily interpolate between numbers it has read elsewhere.

Describe what was built, not what was certified. Never state or imply a legal guarantee or a compliance certification. Our own services pages are careful about this — we build things to help with SOC 2 evidence, we are not SOC 2 certified — and the bot has to be careful too.

Temperature, and why it is set to 0.2

Temperature controls how willing a model is to improvise past its source. Higher values read as more fluent and more engaging.

They are also more likely to produce a confident claim that did not happen.

We set it low and accepted slightly less sparkle in exchange for answers that stay inside the source. For a site whose whole position is restraint, that is not a trade-off worth making the other way.

It can still be wrong

This is the part we put in writing rather than leaving implied. The widget carries a permanent note under the transcript saying it answers from this site's own content and that anything involving scope, price or a commitment should go to a human.

We are not claiming the bot cannot be wrong. We are claiming you will have been told. A language model that gets it wrong without saying so is a liability; one that gets it wrong with a caveat beside it is a tool.

What we would tell you to do with the same approach

Ground it in content you control. Make the "I do not know" path easy and explicit rather than hoping the model stays in bounds. Enumerate the specific things it must not invent, because a general accuracy instruction is weak. Set the temperature lower than feels comfortable. And tell the reader what it is.

The model was never the problem. The problem is that an ungrounded assistant is a confident fiction generator with a chat interface, and everyone knows it.