Will an AI chatbot actually save my business money?

Possibly. The useful question is whether it lowers the full cost of helping a customer while keeping the help good. Faster replies can free up time. They can also leave the wage bill unchanged, add software fees or create more work when an answer needs correcting.

For a small business, start with one task with a clear finish: answering a delivery question, explaining a return or helping someone fix a familiar problem. Decide what a successful outcome looks like before choosing the tool.

What has actually been measured?

Two useful studies tested AI suggesting responses to human support staff.

A 2025 peer-reviewed study examined 5,172 agents at one software company. Researchers estimated that AI assistance increased resolved issues per worker-hour by about 15%, with larger benefits for less experienced workers. The headline estimate comes from a staggered rollout, rather than a randomized comparison. It measured work output, without establishing an equivalent reduction in spending. Final QJE paper

A four-week randomized Taobao experiment covered 5,940 agents in China. Offering AI help cut issue-identification time by 8.2%, but total chat duration by only 1.1%, or 5.7 seconds. Customer ratings rose 0.042 points on a five-point scale. Repeat contact about the same issue within three days showed no statistically significant overall improvement. Top-performing agents experienced worse quality outcomes. July 2026 working paper

The Taobao manuscript remains under revision at Management Science. These results concern human-agent assistance; they do not establish what happens when a chatbot serves customers alone. Neither study measured a complete small-business cost comparison. Publication status

Where does saved time go?

Imagine a shop owner answering messages between packing orders. Finishing earlier could mean an evening off, quicker dispatch or room for more orders. Those are different benefits, requiring different measures.

For an employer, quicker chats may leave staffed hours unchanged. Someone still needs to cover opening hours, lunchtime rushes and difficult cases.

Measure both active handling time and paid hours keeping support available. If overtime or an outside support bill falls, record the cash saving. If the same team handles more work, record extra capacity. Keep the owner’s time visible, too. Avoid counting one saved hour as both a wage saving and time available for another task.

Count the work around the answer

Keep a list of everything needed to run support:

• Software subscriptions, usage charges and minimum commitments
• Setup, system connections, help-page cleanup and staff training
• Ongoing information updates, answer checks and mistake correction
• Human handovers, including customers repeating their story
• Repeat contacts, complaints, and extra refunds or replacements caused by errors

Show setup costs alongside recurring costs. Spreading setup over a year may help compare a continuing service, but keep the initial cash and time visible and state the period you chose. A cheap subscription can hide an owner’s unpaid evenings maintaining it.

Test whether the problem stays fixed

Bright’s suggested measure is the full cost per customer issue resolved and still resolved after a follow-up period chosen in advance. Count each issue once, including all its chats and handovers.

A customer who stops replying may have given up. Check a sample of outcomes: did the promised refund arrive, was the delivery answer correct, did the fix work? Track complaints and ratings alongside repeat contacts. No single indicator tells the whole story.

Start with a narrow trial and an easy route to a person. Compare similar requests during similar periods; avoid giving AI only easy questions while the comparison group handles the hardest cases. Random assignment can help where practical. Record the tool version, its permissions and required staff review. Check customer-data handling before connecting it to real records.

Decide beforehand what would justify continuing: lower costs, useful time returned, better service or some combination. Set stopping conditions, such as repeated wrong promises or customers being unable to reach someone. A short trial may leave the financial answer uncertain.

If a vendor promises “50% savings,” ask whose costs fell, over what period, against what comparison, and which expenses were included. This is a hypothetical claim; unrelated research cannot verify it.

The result worth paying for is a service that works for customers and leaves you better off after all the work and costs are counted.

Evidence note

The QJE study was published on 4 February 2025 and concerns an earlier rollout at one firm, with agents mainly in the Philippines serving US customers. It involved collaboration with the unnamed AI vendor. The Taobao experiment ran from 23 January to 19 February 2024 among agents with less than a year’s tenure; its v2 manuscript was posted on 31 July 2026. Two authors are affiliated with Alibaba. Ratings cover a self-selected subset of chats. The practical cost checklist and trial design above are Bright’s analysis, not a validated savings benchmark. No product was tested for this article.

Sources: QJE DOI, final author-hosted paper, Taobao version record, author’s current publication status.

Related Bright reading

How small businesses are using AI to develop their own ideas. That story concerns individual projects and vendor-published business accounts. This explainer adds a full-cost purchasing test rather than repeating those examples.