How we started
When large language models got good enough for real work, the obvious move was to hand them the whole job. Give the model instructions, a few tools and the data, and let it work out every step on every run. We built automations like that too. In a demo they look great.
Then the same process runs for the thousandth time. A field arrives under a different name, a date comes back in a new format, and a step that worked yesterday gets skipped today. Nobody can tell you why, because there is no line of code to point at. There is only a prompt and a model that made a different choice this time.
Probabilistic by design
An LLM is probabilistic. For a given input it predicts a range of possible outputs and picks one. You can lower the temperature, pin the model version and ask for structured output, and the results get much more stable. They still aren't guaranteed. A new model release, a slightly longer prompt or one unusual input can change the answer.
Code is deterministic. The same input gives the same output today and in a year. When it fails, it fails on a line you can find, and you can write a test so it never fails that way again.
For most business processes, predictability is the whole point. A workflow that moves money, updates customer records or files documents has to do the same thing every time.
"Nobody wants a creative invoice workflow."
What code first means
We still use AI every day. We just use it at a different moment. Instead of asking a model to perform the process on every run, we use it to help write the code that performs the process. The AI does its work once, at build time. A developer reviews the result, tests cover it, and from then on it runs like any other software.
The difference shows up quickly:
- A run costs almost nothing in compute, instead of paying for tokens every time.
- It finishes in milliseconds, so you can run it as often as you like.
- When something breaks, you get an error and a failing test. With a model in the loop you often just get a slightly different answer.
- Anyone can read what the process does, including an auditor.
Where the model still belongs
Code has one big weakness: messy input. A customer email, a scanned contract, a support ticket that mixes three problems in four sentences. Writing rules for all of that never ends, and the rules break the first time someone phrases things differently.
This is where an LLM is really useful. It can read the input and work out the context: what the message is about, who sent it, which category it belongs to and which values matter. So we put the model at the edge of the process, where unstructured text comes in, and keep it out of everything after that.
The trick is to make its job as small as possible. Don't ask the model to "handle the ticket". Ask it to return the ticket type, the customer number and the urgency in a fixed JSON schema. Then validate that output in code and let code do the rest.
The yes/no step
The smallest version of this is my favourite: one question, two possible answers. Is this email a complaint? Does this document have a signature? Is the customer asking for a refund? Several automation tools now offer this as a ready-made building block, an AI condition that reads the input and answers yes or no. You put it in front of an ordinary if/else branch and the rest of the flow stays plain code.
It works because the output is tiny. A yes or no is easy to check and easy to log. Before it goes live, you can run a few hundred real examples through it, count the mistakes and decide whether that error rate is acceptable for this step. When the model isn't sure, the case goes to a person instead of getting a guess.
Test the model like any other component
If an LLM is part of a process, treat it as a component with an error rate, not as a colleague. Keep a set of real examples with the correct answers and run them every time you change the prompt or the model. Log every input and output. Decide what happens when the model's answer doesn't pass validation. Pin the model version and re-run your tests before you upgrade. None of this is new; it's the same discipline we already apply to any other dependency.
My rule of thumb
When I design an automation now, I ask two questions about every step. Can I write the rule down? Then it's code. Does it need to understand language or context? Then it's a small AI step with a fixed output and a test set behind it.
Most steps end up as code and only a few end up as AI. The process as a whole becomes predictable, and the parts that aren't are small enough to measure.
AI first wasn't a mistake
I don't regret the AI first years. They showed us quickly what models can do, and some of those prototypes became real products. But "the model can do it" and "the model should do it on every run" are two different statements. Today I use AI to build the process and code to run it, and I keep the model where it adds something code can't: understanding what came in.