When a language model is the wrong tool

Use a language model when the input is genuinely unstructured and the output can tolerate variation. Use ordinary software when the answer is deterministic — when the same input should always produce the same output, and the rule can be written down. A great deal of what currently gets proposed as AI work is a database query with a budget attached.

Getting this wrong is expensive in a particular way: the system appears to work, and fails occasionally and unpredictably.

What is the actual test?

Two questions, in order.

Can you write down the rule? If you can express the logic as conditions — if the order is over this value and the customer is in that category, apply this discount — then write it as code. It will be faster, cheaper, correct every time, and you will be able to explain any output to a customer or an auditor.

Must the same input always give the same output? If yes, a language model is the wrong tool. Same question, same day, slightly different phrasing, potentially different answer. For a support explanation that is acceptable. For a price, a tax calculation or an eligibility decision, it is not.

Task	Right tool	Why
Calculate tile quantity with wastage	Ordinary code	Deterministic arithmetic — must be identical every time
Answer “how do I calculate wastage?”	Language model	Explanation, tolerates phrasing variation
Check dealer credit limit	Database query	One correct answer exists
Summarise a month of support tickets	Language model	Unstructured input, no single right output
Apply a scheme discount	Rules engine	Must be explainable and auditable
Extract line items from a supplier PDF	Language model, with validation	Unstructured input, but check the numbers after
Decide which team gets a ticket	Language model	Classification from free text
Confirm whether an invoice is overdue	Date comparison	Trivially deterministic

Why does the wrong choice hurt so much?

Because it does not fail cleanly.

Badly written code throws an error. Someone sees it and fixes it. A language model given a deterministic job produces an answer that looks right, is usually right, and is occasionally wrong in a way nobody detects — because there is no error, just a number that is different from the one it should have been.

Applied to a discount calculation, that means some customers get the wrong price. Applied to stock availability, it means some orders are accepted that cannot be fulfilled. You find out from complaints, weeks later, and reconstructing which answers were wrong is often impossible.

There is a second cost. Every one of those calls has a price and a delay attached, so you are paying more, waiting longer, and getting a less reliable answer than a query would have given instantly and free.

Where do language models genuinely earn their place?

Four situations, and they share one property: the input is messy in a way rules cannot anticipate.

Unstructured input. A customer message, a scanned document, an email thread, a supplier PDF laid out differently every month. Rules break on the first unanticipated format. Models cope.

Language itself. Understanding intent, handling multiple languages, generating a readable explanation, summarising. This is what they are actually for.

Classification with fuzzy boundaries. Routing a message by topic, judging sentiment, spotting an unusual complaint. Rules can approximate this and need constant adjustment.

Long-tail cases not worth coding. Where two dozen edge cases each occur twice a year, writing rules for all of them is uneconomic.

What about combining both?

This is usually the correct architecture, and the one good implementations converge on.

The model handles the language; ordinary software does the work. A customer asks about their order in whatever words they choose — the model interprets intent and extracts the order number, a database query retrieves the actual status, and the model phrases the reply. The facts come from the system. Only the understanding and the wording come from the model.

Same principle for document extraction: let the model read the messy PDF, then validate every number it produced against rules you control. Totals must reconcile. Dates must be plausible. Anything failing validation goes to a person.

The model should never be the last thing between a decision and a customer.

How do you tell whether a vendor understands this?

Describe a deterministic problem and see what they propose.

Tell them you need to calculate scheme entitlements across a dealer network. If the answer involves a language model reading invoices and working out the discount, they are selling AI rather than solving your problem. The right answer is a rules engine, possibly with a model helping read whatever arrives as unstructured documents.

A vendor willing to tell you a project does not need AI is considerably more useful than one who finds a use for it in everything. It is also the cheaper relationship, which is rather the point.

We say this on our own OpenAI integration page, and mean it: a good deal of what gets proposed as AI work is better served by a database query.

Common questions

Can a language model be made deterministic?

You can reduce variation with configuration, but you cannot guarantee identical output for identical input the way a function does. For anything that must be reproducible and auditable, do not rely on it.

What if our data is too messy for rules?

Then a model is a reasonable way to clean it up — but validate the output before it enters a system of record. Use it as an extraction step feeding rules, not as a replacement for them.

Are we falling behind if we do not use AI?

Not by declining to use it where it does not fit. You may be falling behind if your catalogue is inconsistent and your transaction history is thin — those are the things that make AI work later, and they are worth fixing regardless.

How do we evaluate an AI proposal?

Ask what happens when it is wrong, how you would know, and how you would explain a specific output to a customer six months later. Vendors who have run these systems answer immediately. Vendors who have demonstrated them do not.

More on this: should your business use AI for support and what RAG actually is. See also OpenAI integration.