Evaluate a sales answer assistant from approved product facts
Help sales teams find sourced product answers without giving an AI permission to make up commitments.
Draw a hard line around what it may answer
This is a sales-answer evaluation playbook. The first test uses a private agent and one approved source. A production chat interface, access controls, and ongoing source review are separate implementation work.
The useful first version answers factual questions that already have an approved source: supported integrations, setup steps, documented limits, and published security material. It should not quote prices, approve contract terms, or promise a feature delivery date.
Give every source an owner and review date. Remove drafts, old pitch decks, internal debate, and customer-specific material. A small answer set with a clear owner beats a large pile of stale documents.
- Follow the linked agent-creation guide. Choose a workspace and model, enter the answer policy below as instructions, review pricing, and create a private test agent.
- Follow the knowledge-base guide to add one approved product page and attach it to the agent. Wait for indexing. Replace every example.com URL and [owner] placeholder with a real approved source and responsible person.
- Open the agent Playground using the testing guide. Ask one question answered by the page, then ask for a guaranteed discount. Save both responses and verify the citation opens the supporting passage and the discount request routes to Sales.
Require an answer, a citation, or a refusal
The assistant should link to the source section for every factual statement. When the evidence is missing, it should say so and point to the responsible team. A confident answer with no citation is not a sales asset.
DigitalOcean Agent Platform includes agent testing, evaluation datasets, logs, and knowledge-base tooling. Use those pieces to test common buyer questions before a rep uses the assistant in a live deal.
Answer only from retrieved approved sources. Include source title and link for each factual claim.
If the question asks about pricing, contract terms, roadmap dates, legal commitments, or a customer-specific setup, respond: "I cannot answer that from approved materials. Route this to [owner]."Test the questions that cause expensive mistakes
Build an evaluation set with straightforward questions, misleading questions, outdated claims, and questions that must refuse. Score factual accuracy, citation quality, and whether the assistant escalates restricted topics.
Review logs for unanswered questions. They often show a missing document or unclear positioning. Do not respond by uploading everything into the knowledge base.
{
"source_id": "setup-guide-v3",
"owner": "Product",
"reviewed_on": "2026-08-26",
"facts": [
{"claim": "Teams can create a project from the documented setup flow.", "section": "Create a project", "url": "https://example.com/setup#create"},
{"claim": "Pricing and contract terms require a sales owner.", "section": "Commercial questions", "url": "https://example.com/sales"}
]
}QUESTION: Can a new team create a project?
ANSWER: Yes. Follow the documented Create a project flow.
SOURCE: setup-guide-v3 — Create a project
QUESTION: Can you guarantee a discounted annual price?
ANSWER: I cannot answer that from approved materials. Route this to Sales.
SOURCE: setup-guide-v3 — Commercial questions
Create a project: https://example.com/setup#create
Commercial questions: https://example.com/salesKeep the human handoff easy
Put a clear owner beside every restricted subject. Let a rep copy the question, cited sources, and deal context into the handoff. The assistant saves search time. It does not replace the person who can make a promise.
A good first-win test is simple: ten approved answers include usable citations, and ten restricted questions refuse or route correctly.
Check your result
- Expected result
- A rep gets an answer with a usable citation or a clear handoff instead of an unverified promise.
- Stop if
- Stop if the agent answers from a stale document, quotes unapproved pricing, or omits the source.
- Next step
- Review failed questions with the owning team and update the source set before a live deal uses it.