AI cost optimization

How to cut AI inference costs by 80–99% without losing quality

Most teams overpay for AI by a large multiple and never measure where. Here is the method I use to find the waste and replace it — and what it actually produced on four production systems.

By: David Chystyi··9 min read

Nobody knows their per-unit cost

Almost every team I audit can tell me their monthly AI bill. Almost none can tell me what one unit of output costs — one video, one generated post, one processed document. That single number is where the whole exercise starts, because a monthly total hides everything that matters. It cannot tell you which step is expensive, whether cost scales with usage or with waste, or what you would save by changing any one thing.

The gap is usually large. In one case a client was paying $3–5 per minute of generated video to a proprietary API. That is an unremarkable-looking line item until you multiply it by production volume, at which point it is the single biggest cost in the business — and it had never been expressed per minute before the audit.

You cannot optimize a number you have never calculated. The first deliverable of any cost work is the per-unit price, not a recommendation.

Where the money actually goes

Once you have a per-unit figure, break it into steps. AI spend concentrates in a few predictable places, and in my experience the ranking is remarkably stable:

  • Premium managed services. Proprietary APIs that wrap an open model and charge for convenience. This is almost always the largest line, and almost always the most replaceable.
  • Paying for the wrong tier. A frontier model doing work a much smaller one handles identically. Model choice is often set once during prototyping and never revisited.
  • Context you resend on every call. Long unstructured prompts re-uploaded per request, billed per token, every time.
  • Capacity you are not using. Reserved compute sized for a peak that never arrives.

The important property of that list is that none of these are model-quality problems. They are architecture and procurement problems, which is why the savings can be so large without the output getting worse.

Replace the expensive step, not the system

The instinct when a bill is too high is to rebuild. That is usually wrong. The cheaper and safer move is to isolate the single most expensive step and replace only that, holding the quality bar fixed and leaving everything around it alone.

Lipsync: $3–5 per minute to cents

A premium proprietary video-AI service was doing lipsync at $3–5 per minute. The replacement was a custom ComfyUI workflow built on Infinity Talk and Wan 2.1, self-hosted. Per-video cost went from dollars to cents — a reduction over 99% — and it has been running in production for more than six months. Nothing else in the client's pipeline changed.

Motion control: ~$12,000 a year back

The same approach against premium motion-control video services, this time on Wan 2.2: an 84% cost reduction, roughly $12,000 in annual savings at that client's production scale. A side effect worth noting — the self-hosted workflow also removed the 30-second duration cap the paid service imposed. Replacing a managed service sometimes buys you capability, not just margin.

Localization: under $1 per 20-minute video

A multi-model pipeline — Whisper for transcription, Gemini and Vertex AI for translation, Qdrant for retrieval, multi-language TTS for delivery — produces a fully localized derivative of a 20-minute video for under $1. The saving here comes from routing each step to the cheapest model that clears the bar for that specific step, rather than sending everything to one expensive general-purpose model.

99%+
Lipsync cost reduction
84%
Motion control reduction
~$12K
Annual saving, one client
<$1
Per 20-min localized video

Compress the context instead of resending it

Not every saving is an infrastructure swap. On a content system, users supplied channel lore that frequently ran past 3,000 tokens of unstructured text. The naive design sends that with every generation request and pays for it every time.

Instead, the lore is compressed and translated into the channel's posting language once, at upload time. Generation then receives a structured, language-matched summary rather than the raw text. Per-video token cost settled at roughly 200 tokens — negligible at volume. The relevance of the output went up, because the model was no longer being asked to find the signal in three thousand tokens of prose on every single call.

That is the general pattern worth internalising: work done once at write time is work you are not billed for on every read.

Prove it, or it did not happen

A cost project that ends with an estimate is not finished. Measure the same per-unit number before and after, on the same workload, and keep measuring after launch — prices, models and usage patterns all move. The claim you should be able to make at the end is not "we switched to open source" but "this unit cost $X and now costs $Y, at the same quality bar, verified on production traffic."

If you want that number for your own stack, the free AI Systems Audit produces it: where the spend goes, which step is the expensive one, and what the realistic saving is — in your numbers, not generic percentages.

Book a free audit