When AI is the wrong answer: rules, plain automation and thin data

AI is the wrong answer in three situations. The first is a task that has one right answer every time. The second is data behind the task that is thin or inconsistent. The third is a process that keeps changing. In each case a plain workflow rule, a fixed calculation or a scheduled report inside the system you already run does the job more reliably, and you can test it more easily.

The public sector gives a useful test. Defra runs a scoring panel for AI requests, and it is blunt about fixed-answer tasks: "A task with one right answer every time scores red, because AI would make it less reliable." The same panel says data readiness scores red more often than any other criterion.

This guide is for owners and managers who want to tell the difference before anyone writes a prompt or signs a contract. It covers four things:

• the three alternatives to AI, defined
• the warning signs that a task belongs to one of them
• a worked example scored the way Defra scores requests
• a decision table you can use on your own task list

A separate section covers what UK data protection law adds to the choice.

Plain automation is a fixed instruction that a system carries out the same way every time. When a record meets a condition, the system performs a set action. A rules engine is the part of a business system that stores those conditions and actions and runs them. In Zoho CRM, plain automation might assign a new lead to an owner or send a reminder before a renewal date.

A report is a saved query that counts, sums or lists records, often on a schedule. A tool such as Zoho Analytics exists to answer questions like "how many orders shipped late last month" from the data you already hold. A report gives the same answer every time it runs on the same data.

AI, in this guide, means a large language model such as Claude reading and writing text. Its output is a prediction, not a lookup. The ICO, the UK data protection regulator, says organisations should record AI outputs as "statistically informed guesses rather than facts".

Anthropic's own documentation is just as frank. A hallucination is text from a language model that is factually incorrect or inconsistent with the context it was given. Anthropic notes that even its most advanced models "can sometimes generate text that is factually incorrect". A rule cannot hallucinate. It can only be written wrongly, and then it is wrong in the same way every time, which makes it easy to find and fix.

A task with one right answer every time is the clearest case where AI is the wrong answer. Defra's panel scores such a task red for AI fit because a model "would make it less reliable". Calculating a price from a price list is a fixed-answer task. So is checking whether a contract has passed its renewal date, or sending a lead to the owner for its region.

These tasks have three things in common:

• the inputs are structured fields, not free text
• the logic can be written as "if this, then that"
• two people given the same record would always agree on the result

Eligibility rules deserve special care. The Department for Education design team wrote about AI summaries of its services. It warned that the nuance most at risk includes "eligibility rules, safeguarding considerations, rights and entitlements that depend on specific conditions". The team adds that the usual risk is not that AI answers are wrong but "that they are incomplete in ways that matter". A refund policy, a discount threshold or a credit limit is exactly that kind of conditional rule.

At Svennis we ask a client to write the decision down as a rule before anyone talks about models. If the rule fits on a page and needs no judgement, we build it as plain automation in the existing system and stop there.

Thin data is the reason AI requests fail most often. Defra reports that "data readiness scores red more often than anything else". Thin data means too few records, categories that were applied inconsistently, or free-text fields that each person fills in their own way. An AI system that sorts tickets by learning from past ones, or summarises history, inherits every one of those gaps.

The process underneath is the second most common problem, according to the same panel. If your team handles the same request three different ways depending on who picks it up, a model has no stable pattern to follow. Neither does a rule. The fix in both cases is to agree the process first.

A report is the cheapest way to see whether your data is ready. Count how many records have the category field empty. Check how many distinct values people have typed into a field that should have five options. Then compare this year with last year. If the counts look poor, the next step is cleanup and a required field, not a model.

Data also changes over time. The ICO defines concept drift, also called model drift, as an AI system becoming less statistically accurate as the populations or behaviours it is applied to change. The ICO says you should "decide and document appropriate thresholds for determining whether your model needs to be retrained". A rule that reads a field does not drift. It needs updating only when you change the policy behind it.

A single accuracy percentage tells you little about whether AI suits a task. The ICO gives a plain illustration in its guidance on statistical accuracy. If 90% of the emails reaching an inbox are spam, a classifier that labels everything as spam is 90% accurate. It is also useless.

The ICO splits accuracy into two measures that matter more:

• Precision is the percentage of cases flagged as positive that really are positive. If nine of 10 emails labelled spam are spam, precision is 90%.
• Recall is the percentage of real positives the system catches. If 10 of 100 emails are spam and the system finds seven, recall is 70%.

The regulator concludes that "overall statistical accuracy is not a particularly useful measure". It also states that "not all AI systems demonstrate a sufficient level of statistical accuracy to justify their use". When a supplier quotes a single figure, ask which errors it hides and what each one costs you.

The ICO also says you "should examine and test any claims made by third parties as part of the procurement process". For a fixed-answer task, a rule makes that test simple. You can run it against every past record and confirm it gives the expected result every time. An AI system can only be sampled and measured. It cannot be proven correct for every input.

AI adds running costs that plain automation does not. The first is usage: every AI step is a model call that costs money to run. Anthropic's models overview lists tiers from Claude Haiku 4.5, the fastest, to Claude Opus 5.5, built for long-running agentic work. A workflow rule makes no model call at all.

The second cost is checking. Anthropic's guide on reducing hallucinations recommends several techniques:

• giving Claude explicit permission to admit uncertainty
• restricting it to the documents you provide
• asking it to cite a supporting quote for each claim
• running the same prompt several times and comparing the outputs

Each technique works. Each also adds prompt design, testing and sometimes extra calls.

The third cost is oversight. The ICO says monitoring frequency "should be proportional to the impact an incorrect output may have on individuals". Some AI systems make decisions about people with legal or similarly significant effects. For those, the ICO also says you are "required to keep a record of all decisions made by an AI system". Someone has to read those records.

That someone faces a known trap. The ICO describes automation bias as people routinely relying on a decision-support system and ceasing to question whether its output might be wrong. If staff will approve whatever the model suggests, a fixed rule with a clear exception queue is often safer.

AI earns its place on work that involves language, searching and sorting. Defra puts it directly: "Language, searching and sorting are what AI is good at, and that work earns a green." Reading a free-text email and deciding which team should handle it is sorting. Summarising a long document is language. Finding the right clause across a folder of contracts is searching.

The ICO gives a public sector example. In its guidance on freedom of information and AI, it says AI "could speed up triaging incoming requests, identify relevant information and help monitor response times". The triage and the search suit a model. The legal deadline does not need one, because a date calculation has one right answer.

Most real requests mix the two kinds of work. Defra's panel handles this explicitly. When only part of a request suits AI, "we take that part and refer the rest". That is the habit worth copying.

Split the job into steps. Give the language steps to a model and the fixed steps to rules.

If you want to see which business tasks usually fall on which side, the overview of AI by business task sorts them by function. The guide on how any company can use AI walks through the same split for a whole organisation.

Defra scores every AI request against 8 criteria grouped under 4 questions. The questions are whether the problem is worth solving, whether it is ready to build on, whether AI is the right answer and whether it will help anyone else. The panel talks each criterion through and agrees a colour.

Take a company that runs customer service in Zoho Desk. A manager asks for AI to do two things. The first is to route incoming emails to the right team. The second is to decide whether each customer qualifies for a refund. Scored separately, the two parts look very different.

The pattern decides the outcome, not the count of greens. Defra says so: "The pattern matters more than the count." Routing goes forward to AI once the categories are cleaned and the time saved is measured. Refund eligibility becomes a plain rule in the helpdesk, with exceptions queued for a person.

The money question deserves weight. Business value scores green only "when a team can show in numbers what AI would save them". Defra reports that about half of teams manage this.

The table below applies the same tests to tasks that come up in most small and mid-sized companies. Use it as a first sort. Then score your own borderline cases with the four questions above.

Two rows need a note. For long documents of more than 20k tokens, Anthropic recommends asking Claude to extract word-for-word quotes before doing the task. That grounds the summary in the actual text. For drafted replies, Anthropic suggests explicitly instructing Claude to use only the documents provided and not its general knowledge.

The last row is a common trap. A prediction sounds like an AI job. With thin data, though, a report showing the customers who cancelled and what they had in common is more honest and costs far less.

UK law does not forbid AI, but it adds duties that shape the choice. The ICO says "data protection law does not stop you from using AI tools", as long as you apply data protection principles. Under the Data Protection Act 2018, most processing of personal data is subject to the UK GDPR.

The law is moving. The Data (Use and Access) Act 2025 (chapter 18) received Royal Assent on 19 June 2025, and its data protection changes came into force in stages. From 5 February 2026 it replaced Article 22 of the UK GDPR and section 14 of the 2018 Act with new Articles 22A to 22D. These allow solely automated decisions in more cases, as long as safeguards are in place.

The ICO notes that its AI accuracy guidance "is under review and may be subject to change" because of that Act. Check the current text before you rely on any summary, including this one.

Several duties bear on the choice between a rule and AI:

• for solely automated decisions with legal or similarly significant effects, the ICO's guidance on individual rights in AI systems expects you to keep a record of each decision, noting whether the person asked for human review or contested it, and whether you changed the decision
• a decision counts as solely automated under the new Article 22A if there is no meaningful human involvement, so a person who only rubber-stamps the output does not change that
• for solely automated decisions with legal or similarly significant effects, Article 22C requires you to tell people about the decision, let them make representations and contest it, and let them obtain human intervention

Telling people about a decision and handling a challenge is easier when the logic is a written rule. If a decision about a person can be made with a rule, that is often the simpler position to defend. The page on AI law in the UK for business covers the wider picture.

You can test most AI ideas in an afternoon, before any money is spent. Work through these checks in order:

1. Write the decision as a rule. If it fits on a page and two colleagues would always agree on the result, build it as plain automation in your existing system.
2. Run a report on the data. Count empty fields, inconsistent values and the number of records. Poor counts mean cleanup comes first.
3. Score the remainder against Defra's four questions. Put a number on the time or money AI would save. Without one, business value stays amber.
4. Split mixed requests. Give the language, searching and sorting steps to a model. Keep the fixed steps in rules, and ask any supplier for precision and recall rather than one accuracy figure.

Whatever passes all four checks is worth building. If you want to see how the AI part sits next to your rules inside the systems you already use, read the post on building AI into the systems you already run. When you are ready to scope it, the page on AI automation for growing businesses explains how a project is set up.