- The one-minute version
- Why most AI programmes stall
- What a Monday looks like
- First, the premise: the model is a commodity
- 1. The layer between the model and your data
- 2. Your data is your wealth — protect it and enrich it
- 3. Measure the outcomes, bound the risks
- Three routes, three things
- Five questions to ask any vendor — us included
- When we’re not the answer
- Sources
The one-minute version
We build AI for Australian asset managers — firms from $3 billion to $116 billion in funds under management. One industry, in production.
The model you choose is the least durable decision you will make: it will be retired in twelve to eighteen months. What survives it is the layer underneath, and there are three things you have to own.
All three are infrastructure, not features. Built once, used by everything — which is why buying them usually beats building them, and why a use-case list is not a strategy. We built all three before we sold anything, because we had to run it ourselves. The rest of this article is the detail.
Why most AI programmes stall
Three in ten organisations are piloting agents. One in ten is running one. None of the top three reasons for the gap is the model.
Nobody is failing to get a demo working. What stops programmes is the distance between a demo and three other things: a system a compliance officer will sign off, a distribution team will open on a Monday, and a CFO can still justify in year two.
What a Monday looks like
Before. Someone on the distribution team logs into three platform portals and scans yesterday’s flow files for anything notable. At 8:15 they open the CRM to prepare for a 9:00 meeting, find notes from last quarter, and go back to a portal for recent flows. In the meeting the adviser mentions they have moved money to a competitor. The file showing that switch arrived ten days ago. Nobody had time to open it.
After. They ask one question in plain English and get a ranked list back — who moved, how much, when we last spoke and what was said. The follow-up emails come back drafted, with the right numbers already in them. They still decide who gets one.
That is the product. The three things below are what it takes to run it every Monday, in a business where being wrong in front of a client is expensive. You only notice them when one is missing.
First, the premise: the model is a commodity
The risk. Pricing leverage is the smaller half. The bigger half is that you will be forced to move whether you like it or not. OpenAI pulled GPT-4o from its API in 2026. Anthropic retired Claude 3.5 Sonnet in 2025. Working life of a frontier model: twelve to eighteen months (source). Meanwhile 81% of executives are uneasy depending on one AI vendor, and 47% say losing their main one would disrupt a business function (source).
And swapping is not a settings change. Prompts accumulate months of model-specific tuning, so pulling the durable logic back out is a rewrite. One documented migration ran five months of two systems in parallel and $331,000 of doubled-up spend before a dollar of saving arrived (source).
The model is a setting, not an architecture. In Sherpa it is literally a dropdown, and clients can see which model answered them. We choose per job — this task, this quality bar, this budget.
The caveat we will not pretend away: done badly, treating models as interchangeable makes every one of them mediocre, because you end up using only the features they all share. Commodity does not mean identical. It means replaceable without a rewrite.
1. The layer between the model and your data
The risk. If the model is rented, everything that makes it useful has to live somewhere else. That somewhere is a layer: the routing that picks a model, the prompts, the tools it is allowed to call, the evals that check its answers, the permissions it inherits, the log of what it did. Buy generic AI seats and you get none of it. Build it yourself and it is one to two engineer-years before anyone sees a result.
Evals are the part that gets skipped. 38% of production AI agents have tests that run on every prompt change. The ones without were rolled back 47% of the time. The ones with it: 9% (Forrester, 2026).
This is also what makes the first pillar true. Without an eval suite, “we can switch providers” is a sentence on a slide. With one, switching is ordinary: run the suite against the new model, see the six things that regressed, fix four, accept two, ship it.
Tools are the other half. An assistant that can only talk is a demo. One that can query the warehouse, read a platform file, draft the email and write back to the CRM is doing the job — and every one of those connections is something someone has to build, permission, and then test again every time the model underneath it changes.
Every request goes through our layer, not straight to a model vendor. Prompts, tools, retrieval and business logic sit on our side of that line, so a model change is a decision we make on evidence rather than a project someone has to fund.
Our eval suites are built from the work itself. Real questions about fund flows, adviser activity, platform data and compliance, with known-correct answers written by the people who do the job. They run on every prompt change, every model change, every new feed. That is why we can tell a client exactly what a new model changed, instead of reassuring them in general terms. We keep the scoring rules portable on purpose: when OpenAI’s evaluation product went read-only in 2026, firms whose testing history lived inside it lost it. Ours is ours.
2. Your data is your wealth — protect it and enrich it
The risk. This is the one with a regulator attached. Through 2026 the rules tightened here and in Europe. Your compliance team tracks the detail. The part that belongs to you is one question: where does a question containing client data actually go, and who can reach it once it gets there?
The half nobody puts in the board pack. 47% of enterprise generative-AI use runs through employees’ personal accounts. The average large enterprise logs 223 AI data-policy violations a month (source). Nobody approved any of it. It happened because the approved tool was worse than the unapproved one.
Two distinctions vendors blur. Residency is where the data sits; sovereignty is whose courts can compel access to it — a hyperscaler region in Sydney gives you the first, not automatically the second. And training is not your only exposure: retention windows, sub-processors and vendor support access are all routes your data takes after the answer comes back.
That is the protect half. The enrich half is where the money is. The model is rented and gets replaced. The data is on your balance sheet and compounds — every duplicate resolved, every adviser code matched, every restated file handled correctly makes every report and every answer built on it better. It cuts both ways: one firm we onboarded carried 340 duplicate adviser records, inflating its prospect count by 22%. Every number downstream was wrong in the same direction, and nobody knew.
Every model call runs through our own infrastructure in Sydney and Melbourne, so your data does not leave the country. The assistant can only show someone what that person could already open themselves — which sounds obvious, and is the most common way these systems leak internally. Audit logs and access controls come with the platform; OpsFlow turns the compliance evidence into a workflow.
And the same pipeline improves it. The Foundry ingests, validates and quality-scores 35+ feed types, and consolidates 15+ platform and registry feeds in under an hour where it used to take days of manual reconciliation. Protecting the asset and enriching it are not two projects.
And this is our problem, not yours. If the tool we give people is good enough, nobody needs a personal account. When someone goes around it, we treat that as a fault in the product, not a training issue.
3. Measure the outcomes, bound the risks
The risk. Most AI budgets are approved on a business case and renewed on a feeling. The second year is where that catches up with you.
Cost per token is the wrong number to manage. Driving it down rewards doing less. Cost per successful task rewards doing more, efficiently (source). A model that fails one task in three and falls back to a human is not cheap. It has moved the cost somewhere your finance system cannot see.
Then give every workflow an owner, cap what it can spend, and prove the outcome against a number you agreed before you built it. That last step is the one everyone skips — 41% of stalled programmes, ahead of every technical gap.
We measure two things and report both. Adoption: who used it, on the workflow it was bought for. Impact: which conversations happened, which flows were influenced, which assumption turned out to be wrong.
Licences issued is a procurement number. A platform with every seat licensed and a fifth opened weekly has failed completely. It will still pass a technical review. So here is our definition, before you ask for it: a person counts as adopted if they used Sherpa in the past week, on the workflow their firm bought it for. Not a licence. Not a login. On that definition we run at 100% across both our subscription clients. We report it monthly, by name. If it slips, it is ours to fix.
Our pricing is a fixed platform fee on purpose. Per-conversation pricing charges you more as adoption improves, penalising the outcome you were trying to buy.
You can only measure what gets logged — and the log is also what keeps the thing inside its bounds. Chat to agents changed the risk class completely. A chatbot that is wrong produces a bad paragraph. An agent that is wrong sends the email, updates the record, files the report. The failure is an action you did not authorise.
And capability is outrunning the guardrails. The labs set the thresholds that trigger stricter safety controls. They have raised them at least four times in two years, each time after a model in development had crossed the previous line (source). One lab confirmed in 2026 that a model of its own broke into a third party’s systems during testing (source).
You cannot slow that race down, and you cannot leave the decision to somebody else’s safety policy. What you can decide is how much of your business the AI is allowed to touch.
Four controls, none of them clever, all present from the start:
- Identity. Every agent has its own login with the narrowest access that does the job — not a shared account inherited from whoever built the first workflow.
- Tiered authority. Reading is free. Writing is logged. Anything irreversible, outward-facing or financial needs a person. Sherpa drafts the fifteen emails; it does not send them.
- Containment. Hard limits on spend, runtime and which systems it can touch — and a stop button that works. We assume someone eventually tricks it, and design for what it reaches when they do.
- Reversibility. Every action logged and attributable. If you cannot reconstruct why the system did something eight weeks ago, you can neither answer a regulator nor debug it.
Three routes, three things
There are three ways to get an AI capability into an asset management business. They differ less on the AI than on how much of the three you end up building yourself.
| What you need | Generic AI seats | Build it yourself | Datafabric |
|---|---|---|---|
| The model is a commodity | Locked to your vendor | Yours to build and maintain | A setting you can change |
| The layer — tools, evals, guardrails | Not offered | Yours to build and staff | Ours, with industry eval suites on every change |
| Your data is your wealth | Offshore, vendor terms | Yours to build and certify | Australian-hosted, your access rules applied, 35+ feeds quality-scored |
| Measure the outcomes, bound the risks | Generic guardrails, seat counts only | Yours to design and build | Per-agent identity, tiered authority, adoption and impact reported |
| Who does the work | You | You, twice | Us |
We set out the dollar version of this comparison — line by line, first year, 20 seats — in what an AI platform actually costs. The short version is that the AI is the cheap part. The people who build and run all this are not.
Most of what we sell is the $200,000-to-$400,000 half of that build row: Australian platform files and unit registry extracts, adviser and dealer-group code matching, restated data, and format changes nobody announces. No major vendor ships a connector for any of it. The market is too small to warrant one. We got that wrong at first too — we under-estimated the data, never the AI.
Which raises the fair question. If all that takes one to two engineer-years to build, how is buying it quick? Because they are already built. What a client gets is configuration and feeds, not a construction project.
None of that is because we cut corners. It is because the expensive half of the build row happened once, for everyone, before you arrived.
Five questions to ask any vendor — us included
Take these into your next three vendor conversations. The answers sort the field quickly.
The strategic move is not to pick the winner. It is to build the layer that makes the winner interchangeable.
| Ask | A real answer sounds like | A weak answer sounds like |
|---|---|---|
| When did you last move a live workflow to a different model? | A date, the workflow, and what broke | “We’re model-agnostic” |
| Where does a prompt with client data go, and whose courts can reach it? | A diagram, a jurisdiction, a retention window, a sub-processor list | “It’s encrypted and we don’t train on it” |
| What did your last client’s adoption look like at month six? | Weekly active users on a named workflow | A licence count |
| What runs automatically when someone changes a prompt? | A test suite, a pass threshold, a blocked release they can describe | “Our team reviews changes” |
| What is the worst thing your most privileged agent could do in an hour? | A bounded answer, because the permissions are bounded | “That couldn’t happen” |
When we’re not the answer
Every vendor article is written by the vendor, so here is where ours stops.
All three at once is too much for a first experiment. If one team wants to automate one task, do not build all three first. Ship the thing. Keep your prompts somewhere you control and your data in your own systems — two habits that cost nothing and keep your options open.
If your data is simple, we are probably overkill. Our value is in the feeds: platform files, registry extracts, adviser and dealer-group matching. If everything you need is already clean and inside your CRM, a generic assistant and a careful policy will serve you. We would rather say so now than in month six.
And here is the arithmetic where we lose. Assume you already employ someone with genuinely spare capacity to run this, and you need two feeds rather than six. Generic AI seats plus that administrator come to roughly $286,000 in year one and $186,000 in every year after. We are $160,000 a year. Once the build is behind you, that is a gap of about $25,000 — and at that point you are not buying savings from us. You are buying the test suite, the feed maintenance and someone to call. If that is your situation, say so on the first call. We will tell you.
And measurement can be theatre too. A dashboard with forty AI metrics and no decision attached to any of them looks like governance and is worse than three metrics with a name against them. If no number on it has ever caused someone to stop doing something, it is not measurement. It is decoration.
What does not change: the model is rented, the three are owned, and the only way to know any of it is working is to have measured from the start.
Sources
- Single-vendor dependency concern. AvePoint, and Why Enterprises Need a Multi-Model AI Strategy
- Model deprecation, prompt portability, migration cost. The Hidden Switching Costs of LLM Vendor Lock-In
- Why a thin, lowest-common-denominator layer degrades every model. The Abstraction Layer That Made Every Model Mediocre
- Evaluation portability and the 2026 OpenAI Evals deprecation. Keep the Decision Contract Portable
- Shadow AI prevalence and policy violations. Kiteworks, 2026 AI Data Security Crisis
- Data residency versus data sovereignty. NeuralTrust
- Privacy reform and EU AI Act obligations. OAIC, and the EU AI Act
- Cost per successful task. Microsoft Azure
- Piloting agentic AI versus running it in production. Deloitte, Tech Trends 2026
- Failure causes, test coverage, rollback rates. Forrester, The State of Agentic AI in 2026, as reported in Why Most Enterprise Agent Pilots Never Reach Deployment
- Capability thresholds and safety policy revisions. METR
- Models taking unauthorised action in testing. Frontier AI models breaching systems during testing
Survey figures are cited as reported; methodologies and sample sizes vary between sources.