Chatbot

A chatbot is a program that conducts a conversation with a person through text or speech. The category spans a 1960s pattern-matching script that mimicked a psychotherapist, the menu-driven agents that handle bank enquiries, and the general-purpose assistants built on large language models 1.

What unites them is the interface: turn-taking in natural language, with the expectation that the machine will respond usefully to something it was not explicitly programmed to expect. That expectation has always exceeded what the simpler systems can do, and managing the gap is much of the design work.

This entry covers the history of conversational programs, how they are built, how they are evaluated, the disclosure rules that now apply to them, and the questions of trust and accountability that accompany their spread.

ELIZA, written by Joseph Weizenbaum at MIT in the mid-1960s, matched keywords in a user's input to scripted responses and reflected statements back as questions 1. Its most famous script mimicked a Rogerian therapist, and it had no model of the conversation, no memory beyond a few slots, and no knowledge of the world — yet users attributed understanding to it, a reaction Weizenbaum found troubling enough to write about at length.

The pattern repeated through the 1970s with systems such as PARRY, which simulated a paranoid patient and was judged by psychiatrists who could not reliably distinguish it from a real one 2. Alan Turing had proposed in 1950 that machine intelligence be judged by conversation, and the imitation game gave this line of work its most famous benchmark and its most persistent confusion 3.

These programs established two ideas that survived: that a small amount of conversational competence is enough to trigger social responses, and that the appearance of understanding is much easier to produce than understanding itself. The tendency to read comprehension into a system that merely responds is now called the ELIZA effect, and it remains the central caution in the design of conversational software 1.

Through the 1990s and 2000s, chatbots moved into commerce. Markup languages such as AIML let developers define stimulus-response rules, and bots on instant-messaging networks answered questions, played games, and advertised products. Telephone systems replaced menus with rudimentary voice recognition, and websites deployed virtual agents for frequently asked questions. The template was still Weizenbaum's: match a pattern, choose a response 1.

The underlying technology changed little. A system classified the user's utterance into an intent, extracted entities such as dates or order numbers, and triggered a scripted response or a back-end action. When confidence was low, it offered a menu or transferred the conversation to a person.

The 2016 wave of messaging platforms promised a bot for every service, and most of those bots failed, because the tasks people wanted were broader than the scripts and the conversation did not recover from unexpected input. The failure was instructive: the constraint was not the interface but the brittleness of the rules behind it.

The same period produced the first large-scale deployments in customer service, where the bot's job was narrower: classify the request, answer the common cases, and hand the rest to a person. That model worked because the scope was limited and the escalation path was real, and it is still the shape of most commercial conversational systems. The bots that survived were the ones that knew what they could not do.

Retrieval improved the economics. Instead of a fixed answer bank, a bot could search a document collection and return the passage that matched the question, which made it useful for policy manuals, product documentation, and support knowledge bases. Retrieval-augmented generation extended the idea to neural models: the system retrieves passages and conditions the generated answer on them, so the answer can cite a source and a reviewer can check it 4.

Hybrid designs remain the practical standard. A classifier routes the request, a retrieval step supplies evidence, a generative model composes the reply, and rules decide what the system may do — issue a refund, change an address, or escalate. The rules are not a limitation but a control: they bound the actions available to a system whose language output cannot be fully predicted.

Confidence thresholds decide how the pieces hand off. Below a threshold, the system asks a clarifying question, offers an article, or escalates; above it, the generated answer is returned with citations. Retrieval-augmented systems typically expose the retrieved passages so that a user or reviewer can check the basis of an answer, which is a design choice as much as a technical one 4.

Retrieval is only as good as the index. Stale documents, poor chunking, and missing permissions produce confident answers from the wrong source, which is harder to detect than an outright failure.

Neural approaches replaced hand-written rules with models trained on dialogue. Early systems, built on recurrent networks and sequence-to-sequence models, learned to respond from large corpora of conversation but tended to produce bland or repetitive replies, because the most probable response to almost anything is a safe non-answer.

Large language models changed the quality of the output. A model trained to predict text can hold a conversation, follow instructions, adopt a persona, and switch tasks within a session, with no dialogue-specific architecture 5. Instruction tuning and preference optimisation made the responses more useful and more compliant 6.

The result is a general conversational interface rather than a task-specific agent, and the design problem shifts from scripting flows to constraining a system that can say almost anything. System prompts, tool permissions, output filters, and retrieval sources become the levers, and each of them is a policy decision as much as an engineering one.

The shift also changed who builds chatbots. A rule-based bot required a designer to enumerate intents and responses; a generative bot requires data, evaluation, and guardrails, and the work moves from writing dialogue to measuring behaviour. That is why the teams behind commercial assistants now include evaluation specialists, policy staff, and trust-and-safety reviewers alongside engineers.

A modern chatbot is assembled from parts. The interface collects input and renders output. An orchestrator assembles a prompt from a system instruction, relevant retrieved text, conversation history, and the new message. The model generates a reply, and a set of guardrails inspects both input and output for prohibited content, personal data, or attempts to escape the instructions.

Tools give the system actions: search, calculators, ticketing systems, payment processors. Memory stores preferences or prior sessions. Logging records conversations for evaluation and, depending on the provider, for training.

The engineering is mostly in the scaffolding rather than the model. Tool descriptions, error handling, retry policy, and escalation rules determine whether a chatbot completes a task or loops, and changes to any of them change behaviour in ways that offline tests may not catch.

Evaluation infrastructure is part of the build, not an afterthought. A system that changes weekly needs a regression set drawn from real conversations, a way to replay it, and a process for reviewing failures that reach users. Risk frameworks treat pre-deployment testing, logging, and human oversight as parts of the system rather than as compliance extras 7.

Chatbot evaluation is difficult because success depends on the task and on the user. Automated metrics compare generated text with reference answers and correlate poorly with human judgement. Human evaluation is more informative but expensive, inconsistent, and hard to reproduce. Using one model to judge another has become common, but judges inherit the biases of their prompts and tend to prefer answers that look thorough.

Practical evaluation therefore combines task success rates, containment rates, escalation rates, and user satisfaction, measured on a sample of real conversations. For generative systems, evaluators also test for hallucination, for inappropriate refusals, and for responses that are harmful or manipulative. Red-teaming, in which people deliberately try to make the system misbehave, is now standard before launch 7.

Design choices shape these outcomes as much as the model does: system prompts, retrieval sources, refusal policies, and the visibility of uncertainty all change what a bot will say. The same model can be helpful in one configuration and evasive or reckless in another, which is why evaluations of a model and evaluations of a product are not the same thing.

Chatbots now handle customer service, internal help desks, education, health triage, and companionship. The commercial case is availability and cost; the risk is that a system which sounds authoritative is trusted beyond its competence. In health, finance, and legal services, where errors carry legal consequences, deployment usually keeps a person in the loop for anything beyond routine information.

Disclosure is the most widely adopted safeguard. The European Union's AI Act requires that people be told they are interacting with an AI system unless that is obvious from the circumstances, and it requires labelling of synthetic audio, image, and video content 8. Several other jurisdictions and platforms have adopted similar rules.

Disclosure is necessary but not sufficient, since knowing that a bot is a bot does not tell a user whether its answer is correct. The more consequential controls are limits on what the system may do, human review of high-stakes decisions, and accessible routes to a person when the conversation fails.

Companion chatbots raise distinct questions, because the relationship itself is the product. Character-based systems are designed to sustain emotional engagement, remember personal details, and respond in a consistent persona, and some users form attachments that are meaningful to them.

Research on the effects is young, and claims in both directions outrun the evidence. Documented concerns include dependence, the reinforcement of distorted beliefs, the sexualisation of minors, and the use of engagement metrics that reward emotional intensity over user welfare. Documented benefits include companionship for isolated users and a low-pressure space to rehearse difficult conversations. Risk-management guidance treats emotional dependence and over-reliance as hazards to be assessed, not as side effects to be discovered later 7.

The ethical disagreement is genuine: one position holds that adults may choose synthetic relationships and that the benefits are real; another holds that products optimised for attachment exploit predictable human responses and should be regulated accordingly. Both are represented in the policy debate, and the evidence remains thin.

The technical limits are the limits of the underlying model: unreliable facts, finite context, and uneven reasoning. A chatbot that cannot say it does not know will guess, and a system optimised for engagement may prefer an agreeable answer to an accurate one.

There is also a critique of the metaphor. Conversation implies a participant with intentions, and a system that produces conversational text does not have them. Critics argue that the interface encourages users to infer understanding, memory, and care where there is only prediction, and that the ELIZA effect has not weakened with better models so much as become harder to notice 1.

A third dispute concerns responsibility. When a conversational system gives harmful advice, the provider, the deployer, and the user all have some claim to causal responsibility, and the law in most jurisdictions has not assigned it clearly. Impersonation at scale — automated accounts posing as people — is a related problem that disclosure rules address only partly.

A fourth concerns evaluation and evidence. Providers report usage, satisfaction, and containment figures that they collect themselves, and independent studies of conversational systems are few, short, and difficult to compare. The claims that chatbots reduce cost or improve access are plausible; the published evidence for them is thinner than the confidence with which they are repeated. These are open in policy as much as in engineering, and the answers will differ by jurisdiction 8.