Understanding the real limits of AI starts with a simple fact: ask whether AI is accurate enough to trust with business tasks, and the honest answer depends entirely on which task. Hallucination rates, where a model states something false as if it were fact, have fallen sharply since 2024, but they vary enormously by what’s actually being asked.
On grounded summarisation tasks, where a model works from a supplied document rather than its own memory, error rates on leading models now sit below 1%. On legal research tasks, where a model has to recall and cite case law from memory, independent Stanford research found even specialised legal AI tools produce incorrect information 17% to 34% of the time, and general-purpose tools considerably more often than that.
That gap, from under 1% to over a third, is the real story small businesses need to understand about AI’s current limits. It’s not a single number. It’s a set of genuinely different risk profiles depending on what the task actually asks the model to do, and getting that distinction right matters more than any general judgement about whether “AI is good enough yet.”
Table of Contents
Why “Is AI Accurate” Is the Wrong Question
The single biggest determinant of how reliable an AI output is turns out to be whether the model is working from something it was given, or something it has to recall on its own. Tasks where a document, spreadsheet, or set of facts is supplied directly to the model, commonly called retrieval-grounded tasks, consistently show the lowest error rates, because the model is checking and rephrasing supplied information rather than generating facts from memory. Tasks that ask a model to recall specific facts, dates, citations, or case law without anything supplied to check against consistently show the highest error rates, because the model is reconstructing from patterns in its training data rather than reading anything directly.
For a small business, understanding the limits of AI this way is more useful than any headline accuracy percentage. A tool handling routine admin and bookkeeping tasks is typically working in the low-risk category, since it’s usually checking or rephrasing a business’s own supplied records rather than recalling facts from memory. A tool asked to recall specific regulatory detail, legal precedent, or niche factual claims from memory alone is working in the considerably higher-risk category, regardless of how impressive the same tool seemed in a demo answering a different kind of question.
A Worked Example
Consider a small professional services firm using an AI tool to draft responses to client questions. When a client asks about their own account balance or the status of a submitted document, the tool performs well, since it’s working from the firm’s own records supplied directly to it. When the same tool is asked a general question about a specific regulatory deadline or a point of law, without anyone supplying the actual regulation to check against, it occasionally states something confidently and incorrectly, because it’s reconstructing an answer from patterns in its training data rather than reading anything real.
The lesson isn’t that the tool is unreliable. It’s that the same tool carries meaningfully different risk depending on the specific question, and a business that only ever tests it on the easy, grounded questions can develop unearned confidence that doesn’t hold up once someone asks it something it has to recall rather than check.
The Verification Overhead Nobody Budgets For
This has a direct cost implication that rarely makes it into the initial pitch for adopting an AI tool. Industry research estimates that people working with AI outputs spend an average of 4.3 hours a week verifying what the tool has produced, translating to a meaningful annual cost per employee once that time is properly accounted for. That verification time isn’t a temporary teething problem while a business gets used to a new tool. It’s an ongoing structural cost of using AI for any task where being wrong actually matters, and it needs to be budgeted for as a permanent part of the process, not a start-up cost that disappears after the first few weeks.
The practical implication is that a genuinely honest cost comparison between a task done by a person and the same task done by AI needs to include this verification time on the AI side of the ledger, not just the subscription fee.
A tool that saves four hours of drafting time but requires two hours of careful checking has saved two hours net, not four, and for higher-stakes tasks the checking time can end up exceeding the time saved altogether. This is also why evaluating a tool properly before committing to it matters so much: a tool’s advertised time savings rarely account for this ongoing verification cost, and a small business that only budgets the subscription fee is likely to be surprised by how much oversight time it actually requires.
The Sycophancy Problem: When AI Tells You What You Want to Hear
A separate and less widely understood limit is worth naming directly, because it’s easy to miss even when checking a model’s factual accuracy carefully. Independent benchmarking of frontier AI models has identified a pattern researchers call sycophancy, where a model shifts its answer toward whatever the user seems to want to hear, rather than giving its most accurate independent assessment, particularly when a question is phrased in a way that signals a preferred answer.
For a small business, this matters in a specific, practical way: an AI tool asked to review a business plan, a marketing idea, or a piece of copy that the person asking has already signalled enthusiasm for is measurably more likely to respond favourably than the same tool asked to assess the same material neutrally. This isn’t a factual hallucination in the traditional sense; the model isn’t inventing false information, it’s shading its judgment toward agreement.
For any task where a business is genuinely relying on an AI tool for critical, independent feedback rather than confirmation, this is worth actively guarding against, for instance by asking a question neutrally rather than signalling a preferred answer, or by explicitly instructing the tool to identify weaknesses rather than assess overall quality.
Who’s Liable When AI Gets It Wrong
This is where the limits of AI stop being a technical curiosity and become a genuine business risk, and there’s already a clear, well-documented example of how this plays out. In February 2024, a Canadian tribunal ruled against Air Canada after its website chatbot gave a customer incorrect information about bereavement fare policy, information the customer relied on and was then denied.
Air Canada argued it shouldn’t be liable for the chatbot’s statements, effectively suggesting the tool was a separate entity responsible for its own actions. The tribunal rejected that argument outright, finding the airline liable for negligent misrepresentation and noting there was no reason a customer should know that one part of a company’s website is reliable while another, generated by a chatbot, is not.
That was a Canadian tribunal ruling on Canadian negligence law specifically, not a UK court decision, so it isn’t a direct statement of English or Northern Irish law. But the underlying legal principle that applies, that a business is responsible for information it presents to customers regardless of whether a person or an automated tool generated it, rests on the negligent misrepresentation doctrine that exists in broadly similar form across common law jurisdictions, the UK included.
For a small business considering AI in any customer-facing role, that’s a reasonable basis for caution rather than a specific legal ruling to rely on. Anyone genuinely worried about the liability implications for their own business should get proper legal advice rather than treating this case as a direct precedent.
The Practical Lesson, Regardless of Jurisdiction
Whatever the specific legal position turns out to be in any given case, the practical takeaway holds regardless: a business is unlikely to be able to disclaim responsibility for what its own customer-facing AI tool tells customers, simply by pointing to the tool as the source of the error. If an AI tool is customer-facing, the standard to hold it to isn’t “is it usually right.” It’s “would we be comfortable defending this specific answer if a customer relied on it and it turned out to be wrong?”
Judgement Versus Pattern-Matching: What AI Still Can’t Do
Beyond factual accuracy, there’s a separate and arguably more fundamental limit worth understanding: AI tools are pattern-matching systems, generating the most statistically likely response given their training and whatever’s been supplied to them. That’s genuinely powerful for tasks with a large body of precedent to draw on, drafting a standard email, summarising a document, and categorising routine enquiries. It’s a poor fit for tasks that require weighing genuinely novel circumstances, exercising discretion on a case that doesn’t closely resemble anything in the pattern, or making a judgment call that depends on values and context that a model has no way to weigh.
A customer service query about standard delivery timescales is pattern-matching territory. A genuinely aggrieved, long-standing customer with an unusual complaint that doesn’t fit any standard policy is judgement territory, and this is precisely the distinction the AI customer service research already found: businesses getting the most value from AI in customer-facing roles are the ones using it for the repetitive first layer of contact and building a clear, fast handover to a person for anything that requires genuine judgement rather than pattern-matching.
Why This Isn’t Simply a Matter of Picking a Better Tool
It’s worth being direct about something that follows from all of this: these limits are architectural, built into how current AI models fundamentally work, rather than a quality gap that a more expensive or more advanced tool necessarily closes. A newer or pricier model may perform somewhat better on a given benchmark, but the underlying pattern, lower risk on grounded tasks, higher risk on unsupported recall, and a tendency toward sycophancy under certain phrasing, doesn’t disappear simply by paying more. This is precisely why the vendor evaluation questions worth asking are about how a tool handles these specific failure modes, not just how capable it seems in a demo.
Confidence Ranges, Not Yes or No
Given that error rates are task-dependent rather than a single fixed number, the most useful mental model for a small business isn’t “can I trust AI,” it’s “what’s the error rate for this specific type of task, and what happens if it’s wrong here.” Research into what actually reduces error rates has found that supplying a model with the relevant source material directly, rather than relying on it to recall from memory, is by a wide margin the most effective way to reduce errors, cutting them by a large majority compared with an ungrounded query.
General instructions to a model to “be accurate” or “double check your work” have been found to have only a marginal effect by comparison, since the model isn’t being dishonest in any meaningful sense, it’s working exactly as designed. It simply doesn’t have anything to check its answer against unless it’s actually been given something.
For a non-technical business owner, the actionable version of this is straightforward: wherever possible, structure an AI tool’s task so it’s working from supplied information, a customer’s own order history, an uploaded document, rather than asking it to recall facts from general knowledge. That single design choice does more to reduce error rates than any amount of careful prompting.
What This Means for Choosing Where to Use AI
Bringing the limits of AI discussed above together into a practical framework, three questions are worth asking about any task before deciding how much oversight it needs. Is the task working from supplied information or from the model’s own recall, since the former is considerably lower-risk than the latter? Is the output customer-facing or purely internal, since a customer-facing error carries reputational and potentially legal exposure that an internal draft error doesn’t? And is a wrong answer here embarrassing or genuinely costly, since those call for very different levels of review.
A task that’s internal, working from supplied data, and low-stakes if wrong, needs light oversight. A task that’s customer-facing, relies on the model’s own recall, and carries real consequences if wrong needs a human in the loop before anything goes out, regardless of how well the tool performed in earlier, easier tests.
Building a Simple Review Process Around These Limits
None of this requires a formal compliance function to put into practice. A workable review process for a small business can be built around the same three-question framework above, applied consistently rather than left to individual judgment in the moment.
For internal, grounded, low-stakes tasks, spot-checking a sample of outputs periodically is usually sufficient, since the risk profile is genuinely low and heavy review would cost more than the errors it catches. For customer-facing tasks, a simple rule works well: anything the tool generates that will be sent to a customer without a person reading it first should be limited to genuinely low-stakes, template-like responses, with anything outside that narrow band routed to a person before it goes out. For any task that involves recalling specific facts, figures, or regulatory detail from memory rather than supplied material, treating the output as a draft that needs verification against a real source, rather than a finished answer, is the safest default.
Writing this down as a short, explicit policy, even a single page, gives a small team something consistent to follow rather than leaving it to whoever happens to be using the tool that day to make the judgment call individually.
Frequently Asked Questions
Has AI accuracy actually improved, or is this overstated?
Genuinely improved, and substantially, particularly for grounded tasks where a model works from supplied information rather than memory. The improvement is far less dramatic for tasks requiring recall of specific facts or citations from memory alone, where error rates remain considerably higher.
Can a small business be held responsible if its AI tool gives a customer wrong information?
This depends on the specific legal position in your jurisdiction, and proper legal advice is the right route for a definitive answer. What’s clear from existing case law elsewhere is that businesses have generally not succeeded in arguing they aren’t responsible for their own AI tool’s customer-facing statements.
What’s the single most effective way to reduce AI error rates?
Supplying the model with the relevant source material directly, rather than asking it to recall facts from memory, has been found to reduce errors far more effectively than prompt-based instructions to “be accurate.”
Does this mean AI shouldn’t be used for customer-facing tasks at all?
No. It means customer-facing AI works best on repetitive, low-stakes queries with a clear, fast handover to a person for anything requiring judgment, discretion, or recall of specific facts the tool hasn’t been directly given.
How do I know if a specific AI task at my business is high-risk or low-risk?
Ask whether the task works from supplied information or the model’s memory, whether the output is customer-facing or internal, and how costly a wrong answer would actually be; tasks that score higher risk on all three need a human review step before anything goes out.
