Scaling an AI MVP that is still delivered by hand

There is a version of an AI product that works perfectly well right up to the moment it succeeds.

The demo was real. The output was good. Somebody bought it, then twenty somebodies bought it. What the buyers do not see is that each delivery still passes through a browser tab, a founder's judgement and a quiet half hour of cleanup. The product sells faster than it runs.

This is not a failure. It is the correct way to find out whether anyone wants the thing. It only becomes a problem on the day the manual step stops fitting in the day, and that day arrives suddenly.

The three failures, and the model is not one of them

When a team in this position asks for help, they usually ask about output quality. That is almost never the first problem. Three others are, and they are all boring.

A person is inside the loop. Not reviewing the output: producing it. The difference matters. A reviewer scales badly. A producer does not scale at all, and worse, nothing about what they do is written down, so the system cannot be tested, measured or handed over.

The integration is a browser, not an API. A surprising number of AI products are built on a tool that has a web interface and no public API, so the team drives the interface with browser automation and a few accounts. This works beautifully until the vendor moves a button. It cannot be made reliable, it usually breaks the vendor's terms of service, and every hour spent hardening it is an hour spent on something that is going to be deleted.

Nobody knows the cost of one unit. Ask what a single delivered output costs, including retries, the failures nobody counts and the manual time, and the answer is often a pause. Without that number you cannot price, you cannot forecast, and you cannot tell whether your next hundred customers make you money or cost you money.

Fix those three and the quality question gets much smaller on its own.

Why better prompts will not fix it

The instinct is to hire a prompt engineer. It is the cheapest-looking option and it is the wrong one.

A prompt is a request. It gives you a distribution of outputs, and you are going to see the whole distribution, including the tail, because you are running thousands of generations a month. Tightening the prompt narrows the distribution a little. It does not put a floor under it.

What puts a floor under it is constraint: reducing the number of things the model is allowed to decide. Every decision you take away from the model is a class of error you no longer have to detect.

Generate the least you can get away with

The single highest-leverage change in most of these systems is this: stop generating things you already have.

Build one record that holds the verified truth about the thing you are producing output about, with a source recorded for every field. Where the value came from, when, and how confident you are. That record is the only input your generation step is allowed to see.

Then split the output into what must be generated and what can be assembled. Assembly is deterministic, nearly free, and never invents anything. Generation is expensive, slow and probabilistic. Most products generate far more of the output than they need to, because generating all of it was the fastest way to build the demo.

When a model writes copy, every claim in it should trace to a field in the record. A validator that rejects any sentence without a source will catch more errors before publication than any amount of prompt tuning catches after.

The pipeline that replaces the person

The manual step is not removed by automating what the person does. It is removed by building the thing that was missing, which is a pipeline with states.

• A durable queue and a state machine per job. Generation calls take seconds to minutes, so everything is asynchronous. Each job has a status you can query, retries with backoff, idempotency keys so a replay does not produce a duplicate, a timeout and a cap on spend.
• Cost accounting per job and per attempt. Written down at the time, not reconstructed from a provider invoice at the end of the month.
• Your own copy of every artefact. Generation providers routinely delete output after a short retention window. Copy it to your own storage on completion or you will lose work you have already paid for.
• An automated quality gate. Programmatic checks against the record before anything ships: does the output contain the claims it should, does it avoid the ones it must not, does it match the source material. Failures regenerate the failing part only, with a cap on attempts.
• A human queue for the disputed cases only. This is the reviewer role, and it is the one that scales. One person can clear the few percent that the gate flags. No person can produce a hundred outputs a day.
• Tenant isolation and per-tenant limits from the first version. Retrofitting this once you have customers is one of the more expensive mistakes available.

None of this is exotic. It is ordinary backend engineering, which is exactly why AI-first teams often do not have it.

What one unit costs

Here is the shape of the decision, as multiples of the most expensive option. The absolute numbers depend on your providers and change every few months; the ratios have been stable for a while.

Architecture	Relative cost per unit	What you trade
Everything generated, frontier model	1.0x	Nothing, and it is the least predictable output you can buy
Everything generated, fast or lite tier	0.2x to 0.4x	Consistency, which is the thing you were short of
Hybrid: generate only what cannot be assembled	0.1x to 0.2x	A little polish
No generation in the output path at all	under 0.1x	The part customers called magic

Two things are usually missing from the first estimate. Retries add twenty to thirty percent, because a system with a real quality gate rejects real work. And the cheapest model tier is not a free saving: it needs an evaluation set to prove it is good enough, and building that set is the step teams skip. The same trap appears on the language model side, where the spread between tiers of a single provider is a hundredfold on input tokens and the cheap wins come before the structural ones. We went through those numbers in self-hosted LLM or API: what AI automation costs.

The limits that are not in your code

The constraints that stop these products scaling are frequently not technical at all, and they have lead times measured in weeks.

Platform review. If your product publishes or messages on someone else's platform, the production API access almost always requires an application review. Before it is granted you may be rate limited, restricted to private visibility, or sandboxed in a way that silently makes the product look like it works when it does not.

Quotas. Published daily limits are usually far below what a few hundred customers generate. Raising them means another review. Spreading the load across several projects or accounts to get around a quota is generally a terms violation and a good way to lose all of them at once.

Disclosure. Major platforms now require realistic synthetic media to be labelled. Do it automatically at publication rather than relying on anyone to remember.

Consent for voice and likeness. If the product clones a real person, you need written permission from that person and a working way for them to withdraw it. Several jurisdictions now legislate on this specifically. Storing those consents is a product feature, not a legal afterthought.

What a takeover audit should answer

If you are bringing in a team to take this over, the first engagement should be a paid audit, not a quote. Anyone who gives you a fixed price for the rebuild before reading the code is guessing, and you will pay for the guess later.

A useful audit answers seven questions in writing:

1. What state is the code actually in: structure, tests, dependencies, the parts that are generated or copied and not understood by anyone.
2. What the target architecture is, and which pieces of the current one survive into it.
3. What the scope of work is, broken into phases that can be stopped between.
4. How long each phase takes, with the assumptions stated.
5. What it costs to build, and separately what it costs to run per unit.
6. What the risks are, technical, commercial and legal, with the ones that have lead times called out.
7. What the migration strategy is: how the new path runs alongside the old one, what gets compared, and when the old one is switched off.

The last point is the one that gets skipped and the one that decides whether the project lands. A rebuild that goes live as a single switch is a rebuild you cannot reverse. Run the new pipeline in parallel on real work, compare quality, cost and reject rate against the current process, and cut over per customer once the numbers hold.

A document like that is worth having even if you then hire someone else to do the work, which is the honest reason to pay for it.

Where to start

Write down the cost of one unit this week, including manual time and failures. Everything else is easier to decide once that number exists.

Get off browser automation before you optimise anything. It is the piece most likely to break without warning, and no improvement built on top of it survives.

Pick one output type and build the full pipeline for it, gate and all, running next to the current process. One type done properly tells you more than six done partially, and it gives you the comparison you need to justify the rest.

Be realistic about the target. Generative systems do not reach zero errors. They reach a low reject rate with a gate that stops the rest reaching a customer. A supplier who promises you the first thing is either not listening or not experienced.

---

This is the work we do. We are a backend and integration team: queues, pipelines, provider APIs and taking over codebases somebody else started, including the ones that were half written by a model. We will also tell you plainly which parts of your product we have done before and which we have not.

The first step is a fixed-price technical audit that answers the seven questions above on your actual code, or an AI process audit if the question is which parts should be automated at all. Both produce a document you own.

If you have an MVP with customers and a person holding it together, tell us where the manual step is and we will tell you what we would do first.