Self-Hosted vs API LLMs: The Real Cost Comparison

Two different questions arrive wearing the same words.

Self-hosted vs API LLMs is asked as a cost question and is usually a control question. Both are legitimate and they have different answers. On cost: at the volumes a European SME actually runs, self-hosting almost never wins, because the dominant expense is not the GPU, it is the engineer who keeps the thing running at three in the morning. On control: self-hosting wins outright, because nothing leaves your infrastructure, no third party processes the data, and no provider retires a model on a schedule you did not choose. Decide which question you are asking before anyone builds a spreadsheet, because a cost case assembled to justify a control decision will be a bad cost case and it will fall apart in front of your CFO.

Where the break-even for a self-hosted LLM actually sits

Two cost curves with different shapes. Paying per token is close to linear: little usage costs little, heavy usage costs a lot, and nothing costs anything while the system is idle. Self-hosting is a step function on top of a fixed floor. A machine with a capable GPU costs the same every month whether it runs one request or a million, and above its capacity you buy a second one.

That shape produces the answer. Break-even sits wherever the linear line crosses the flat one, which means high and sustained usage, not bursty usage and not occasional usage. A document-processing system handling a couple of thousand documents a month generates an API bill in the tens of euros. A dedicated GPU instance capable of serving a mid-sized open model starts in the hundreds of euros a month and is paid in full during the twenty-two hours a day when nobody is using it. On infrastructure alone the crossover is well above where most SME workloads sit.

Then add the term that decides it. Somebody has to run this: patch it, watch it, restart it, be reachable when it stops. Even at a modest allocation that is a recurring cost measured in engineer days per month, and it dwarfs the hardware line for every workload below genuinely industrial volume. Counting that honestly moves break-even up by a large multiple, and it is the line most self-hosting business cases quietly omit. The same omission shows up in build quotes generally, which is why we set out the running costs separately in what a custom AI build costs.

What gets left out when LLM API cost is compared to hardware

A comparison that sets the LLM API cost against a monthly server invoice is comparing a complete price against a partial one. Five lines belong on the self-hosted side and are almost never there.

• Idle capacity. You pay for the machine, not for the requests. A workload with a weekday-daytime shape wastes most of what it rents, and that waste is invisible in a comparison that divides monthly cost by peak throughput.
• Getting the hardware at all. Capable GPUs are still rationed at most providers, and the ones you can reserve on demand carry a premium over the committed-term price the comparison usually quotes. Plan for a reservation and plan for the term.
• On-call. A hosted API has an operations team you are renting by the token. Self-hosting means your own people answer at 3am, and one person answering alone is a bus-factor problem rather than a rota.
• Model updates. Open models improve quickly. Staying still is a choice with a cost, and moving means re-testing everything that depended on the old one.
• Re-evaluation. Every model change, in either direction, needs your evaluation set re-run and the results read by somebody who understands the workload. Without that set you cannot tell a cheaper model from a worse one, which is how self-hosting projects quietly degrade the product they were meant to protect.

The open source LLM gap in 2026: real, and narrower than it was

Reviewed in August 2026, and the honest summary is that the gap is real and that it no longer matters for most business workloads. Open weights handle classification, extraction from your own documents, routing, tagging and summarisation at a quality that is difficult to distinguish from a frontier model in a blind comparison on those tasks. Those tasks are the majority of what companies actually deploy.

The gap shows up in three specific places, and if your workload lives in one of them the cost argument is beside the point. Long-context reasoning over large document sets, where the frontier models remain clearly ahead. Complex tool use across many steps, where reliability compounds and small differences per step become large differences per task. And the smaller European languages, where open models vary considerably and the only way to know is to test on your own text rather than to read a leaderboard.

That last point is the general rule. A leaderboard tells you how a model performs on somebody else’s task. Fifty of your own documents with known correct answers tells you how it performs on yours, and the exercise takes an afternoon. Our own stack runs open models under Docker on Hetzner alongside hosted APIs, and the choice is made per workload on exactly that evidence.

Sovereignty, procurement, and where self-hosting is unanswerable

Self-hosting removes the third-party processor entirely. No transfer mechanism to argue about, no sub-processor list to maintain, no provider policy that might change next quarter. For a public-sector tender that requires processing inside national borders on infrastructure you control, that is not a preference, it is a qualification criterion, and no amount of API convenience substitutes.

Be precise about what it is not, though. Self-hosting is not the only compliant route, and treating it as one leads companies to spend heavily on a problem an EU cloud region already solved. The deployment path decides the data-protection position rather than the provider brand, which is the argument in GDPR and LLMs, and for a large share of European buyers a frontier model called inside an EU region with a proper agreement in place answers the question at a fraction of the effort. Self-hosting is the strongest position available. It is not the only defensible one, and the difference is worth several tens of thousands of euros.

Where it genuinely earns the extra work: special-category personal data, client material under professional privilege, defence and public-sector procurement with explicit infrastructure requirements, and anything where a contractual prohibition on third-party processing already exists. Outside those, the sovereignty argument is usually a preference dressed as a requirement, and it is worth saying so out loud in the meeting.

The hybrid, and why it is the default answer

The arrangement that survives contact with a real business is not a choice at all. Route by data class: the sensitive workloads run on your own infrastructure, everything else goes to an API, and one thin routing layer decides which is which based on rules you can read. You get the compliance position where it is needed and the quality and elasticity everywhere else, and the routing layer is a week of work rather than a strategic commitment.

The precondition is portability, and it has to be designed in from the start. Memórias do Jamor was built fully stateless, with everything held in managed storage rather than on the machine, precisely so the same system could move to cheaper self-hosting later with no downtime. That decision cost nothing at build time and it is what keeps the option open. The rest of our case studies and the capabilities page follow the same principle: keep the model behind an interface you own, so swapping it is a configuration change rather than a rewrite.

Three cases where self-hosting is a mistake

You are doing it to save money at SME volume. Run the arithmetic including the engineer days and it will not close. If it does close, check whether you costed the idle hours and the on-call, because those are the two lines that usually turn a winning case into a losing one.

You have no evaluation set. Without one you cannot tell whether the open model you switched to is as good, and you will find out from a customer instead. Build the evaluation set first. It is worth doing even if you never self-host anything.

One person would own it. A self-hosted LLM deployment with a single administrator is an outage waiting for a holiday. Either resource it properly with two people who can both operate it, or take the managed route and spend the difference on something that moves your business.