Service guide · AI assistants

AI chatbot development company: scope it before you hire

A chatbot demo is easy to buy. A chatbot that gives cited, permission-aware answers on your real content is a data and evaluation project. The brief decides which one you get.

The short verdict

Hire for a scoped assistant over named content, with citations, a refusal path and an evaluation set agreed before build. If the assistant must take actions, scope it as an agent with approval gates — not as a chatbot with extra features.

  • The content the assistant should answer from exists and is current
  • Someone owns answer quality after launch
  • You can name the cases where escalation is required
Assistant decision pathA question is answered from approved content, or escalated to a person. Action-capable assistants add tool permissions and approval.ANSWER · ESCALATE · ACTUser questionRetrievalcited passagesAnswerwith sourceRefusalor escalateHuman reviewTool actionapproval + auditNo action without a named approver.
  • ScopeAnswer-only or action-capable
  • GroundingCitations and refusal behaviour
  • EvidenceTest set and thresholds before build

Three things people call a chatbot

They differ in permissions and failure modes, not in how impressive the demo looks.

Knowledge chatbot

Answers from approved content, no side effects

  • Scripted or guided conversation over a fixed content set
  • Cheapest to operate and easiest to test
  • Cannot complete a task beyond answering or routing
Best when
  • Questions are predictable
  • The content is already written and approved

Retrieval assistant

Grounded answers with citations

  • Embeds and retrieves from documents or policies
  • Must show sources and refuse when unsupported
  • Quality depends on chunking, permissions and evaluation
Best when
  • Answers live across many documents
  • Trust requires a visible citation

Tool-using agent

Takes actions across systems

  • Plans steps and calls tools with defined scopes
  • Needs approval gates, audit trails and rollback
  • Evaluation covers action sequences, not only answers
Best when
  • A task spans several systems
  • You can name who approves each write

Build the chatbot brief

Select the dimensions that apply. The output is a scoping checklist for partners — not a quote, an architecture decision or a vendor recommendation.

01 · Content and permissions

Most failed chatbots are content or access problems.

02 · Behaviour and boundaries

Decide what the assistant must refuse before you see a demo.

03 · Delivery and evidence

The acceptance plan is part of the scope, not an afterthought.

AI chatbot scope0 selected

Scoping guidance only. It does not measure quality, recommend a model or promise a launch outcome.

What to demand from a chatbot partner

A polished demo proves presentation, not accuracy. Ask for the rows below in writing.

Demo, evidence and contract expectations for an AI chatbot partner.
CriterionDemoWhat most vendors showEvidenceWhat a serious partner providesContractWhat protects you
AccuracyCurated questionsVersioned test set with per-slice resultsThresholds and re-test triggers written in
GroundingAnswers sound rightCitations to source passagesAcceptance test for unsupported answers
PermissionsSingle demo accountRole-based retrieval testSecurity review and access model
Operations'We monitor it'Named owner, cadence, dashboardsSupport and re-evaluation commitment

Scope the assistant before the architecture

Four decisions that make proposals comparable.

  1. Pick the job, not the technology

    Answer, retrieve or act. Each implies a different permission model and evaluation method.

  2. Name the content and the gaps

    List sources and owners. If the answer is not written down, the project is a content project first.

  3. Write the refusal policy

    Define unacceptable answers, escalation paths and who reviews them.

  4. Agree the evaluation plan

    Test set, thresholds, approver and the date of the launch decision.

Chatbot failure modes

Most failures are visible in the brief if you look for them.

  • Confident wrong answer

    Early signalUsers stop checking citations, or citations do not support the sentence

    ResponseCitation-first design and a tested refusal path

  • Permission leakage

    Early signalA user can retrieve content they should not see

    ResponsePer-user retrieval rules tested before launch

  • Demo-derived scope

    Early signalThe test set is made of questions the vendor already showed

    ResponseCollect cases from real work and hold some back

  • No post-launch owner

    Early signalQuality is assumed to hold after the model changes

    ResponseName the monitoring owner and re-test triggers

Straight answers

Questions buyers ask

What does an AI chatbot development company do?

It designs, builds and tests an assistant around a defined job: the approved content it answers from, the systems it may read or change, when it must hand over to a person, and how answer quality is evaluated before and after launch. The model usually comes from a provider; the engineering is everything around it.

Do we need an AI chatbot or an AI agent?

Choose a chatbot when people need answers from content you control. Consider an agent when a task takes several steps across your systems and you can define its permissions and approvals. AI agent vs chatbot walks through the decision.

How do I evaluate an AI chatbot development partner?

Ask how they test answer quality on your real questions, how the assistant behaves when it does not know, how private data and permissions are handled, and who maintains the content after launch. A polished demo on their own data is not evidence it will work on yours; see AI evaluation.

What should an AI chatbot brief include?

The top questions or tasks, where correct answers live today, what the assistant may and may not do, when it must hand over to a person, privacy constraints and how you will judge success. The checklist on this page turns those choices into a brief you can share with partners.

Evidence, limits and next decisions

What this guidance is based on, what it cannot tell you, and where to go next.

Sources used for method and context

An editorial buying guide based on public AI-risk and agent-design guidance. It does not claim Scopeivo builds AI systems or that any vendor is endorsed.

Decide, then scope

Write the assistant brief before you compare demos.

Name the job, the content, the refusal policy and the evidence you expect. Then evaluate partners against the same scope.

Scope My ProjectRead the AI evaluation guide

No account needed. Your answers stay in this browser.