AI Cost Optimization Framework

The AI Cost Optimization Framework is Crescent Capital Advisors' six-layer model of AI spend: API, infrastructure, tooling, people, waste, and risk. Layers 4 through 6 carry 55 percent of spend and are the three rated least visible. Working all six layers, with governance as the detection mechanism, recovers 20 to 34 percent of AI spend in the first year.

Why a Framework and Not a Checklist

A checklist tells a team to switch models and tighten prompts. Both are worth doing, and together they reach about a quarter of the spend. The rest of the money sits in places a checklist has no column for: engineer hours spent on manual review, requests that fail and get retried, a dev environment nobody shut off, an incident that was budgeted nowhere.

Those costs stay invisible because no one instrumented them. Governance is what makes them visible. Once every call carries metadata and every layer has an owner, the waste stops being an argument and becomes a number a CFO can act on.

The framework exists to hold that structure. Six layers, each with a share of spend, a visibility rating, and a named intervention. A portfolio company can score itself against it in an afternoon and know which layer to open first.

The 6 Cost Layers

Every AI dollar lands in one of six layers. Share of spend is the pattern CCA sees across mid-market portfolio companies; visibility is how easily a finance team can see that layer today without new instrumentation.

#	Layer	What It Is	Share of Spend	Visibility	Conventional Approach	Operating-Partner Approach
1	API Costs	Direct inference cost (OpenAI, Anthropic, Azure)	25%	High	Cheaper models and prompt tuning: 10–15%	Confidence-based model cascading: 10–20%
2	Infrastructure	Compute, storage, GPU rental, vector DB	15%	Medium	Not addressed	Batching, quantization, caching: 20–30%
3	Tooling	Monitoring, orchestration, observability platforms	5%	High	Not addressed	Consolidate tools, OSS alternatives: 15–25%
4	People	ML engineers, QA, security review, oversight	40%	Low	Not addressed	Automation, process efficiency: 15–30%
5	Waste	Failed requests, retries, dev/test bleed, duplicates	10%	Very Low	Not addressed	Governance and observability: 40–60% of waste removed
6	Risk	Compliance, incident response, breach remediation	5%	Very Low	Not addressed	Governance and prevention: 80–100% of preventable incidents

The math: multiply each layer's share of spend by its recoverable rate and the six layers sum to 20–34% of total AI spend in the first year. Layers 4 through 6 contribute 14 to 23 of those points, close to two-thirds of the total, while carrying 55% of the spend. Step 5 compounds the result at 20–30% year over year, which is where the higher figures quoted for mature programs come from.

Read the visibility column next to the share column. Layers 4, 5, and 6 carry 55% of the spend and are the three rated Low or Very Low. That correlation is the whole argument.

Layer Detail

Layer 1: API Costs

What good looks like: every call tagged with timestamp, system, user, feature, model, tokens, and cost, so spend rolls up to the feature owner who caused it.

Where firms get it wrong: swapping the default model across every workload and declaring the project finished. Capability gets matched to the loudest use case instead of to each one.

Layer 2: Infrastructure

What good looks like: batching, quantization, and semantic caching in place, with cache hit rate and GPU utilization reported alongside the bill.

Where firms get it wrong: reserved GPU capacity still running for a pilot that ended two quarters ago. Vector databases sized for a traffic forecast nobody revisited.

Layer 3: Tooling

What good looks like: one observability and orchestration stack, chosen deliberately, with open-source alternatives evaluated where the paid tier buys little.

Where firms get it wrong: three teams buy three overlapping monitoring platforms. The layer is small (5%) and highly visible, which is why it tends to be the easiest early win and the least examined.

Layer 4: People

What good looks like: engineer, QA, and security-review hours spent on AI work are costed and reported next to inference spend. Manual review moves to automation where the accuracy case holds.

Where firms get it wrong: the largest layer at 40% of spend sits in headcount lines that nobody thinks of as AI cost. Senior ML engineers spend their weeks on manual QA and access reviews that tooling should carry.

Layer 5: Waste

What good looks like: failed requests, retries, duplicate calls, and dev/test bleed measured weekly, with a named owner and a target.

Where firms get it wrong: no one owns the metric, so no one knows the number. This is the layer with the highest recoverable share and the lowest current visibility, and it is where the operating-partner approach puts most of its effort.

Layer 6: Risk

What good looks like: compliance work, incident response, and remediation are costed in advance, and prevention is funded as a line item.

Where firms get it wrong: risk shows up once, as an unbudgeted remediation in the quarter it lands, and the exit diligence question that follows costs more than the incident did.

Where the Effort Goes

The conventional split puts roughly 90% of effort on layers 1–2, the 40% of spend that is visible and easy to work. The operating-partner split inverts it: about 10% of effort on layers 1–2, 40% on layer 5 (waste elimination through governance), 30% on layer 4 (people and automation), 20% across layers 3 and 6.

The shift underneath: most of the savings come from usage that should never have happened, and only a minority from making necessary usage cheaper.

The 5-Step Methodology

Step 1: Cost Visibility (Weeks 1–2)

Goal: know where every dollar is going.

• Map all AI systems (inference, fine-tuning, embeddings, storage, monitoring)
• Tag every API call with metadata: timestamp, system, user, feature, model, tokens, cost
• Instrument infrastructure (compute, bandwidth, storage logging)
• Aggregate labor costs tied to AI work
• Quantify waste (failed requests, retries, timeouts)

Deliverables: AI cost breakdown by layer; cost by system, team, and feature; waste log; labor allocation analysis.

Step 2: Usage Pattern Analysis (Weeks 3–4)

Goal: identify which costs create business value.

• Which features generate revenue and which are cost centers
• Which users generate value and which are experiments
• Which models are needed at full capability and which could run cheaper
• Pareto analysis: 80% of cost usually traces to 20% of usage
• Correlation check: does higher API spend track with higher revenue

Deliverables: low-value usage patterns identified; cost-to-revenue mapping by feature; failure root-cause analysis; caching opportunity assessment.

Step 3: Model Selection Strategy (Weeks 5–8)

Goal: match model capability to actual need.

Confidence-based cascading runs the cheap, fast model first and measures its confidence. Above 0.8, the response ships. Between 0.7 and 0.8, the request escalates to a mid-tier model. Below 0.7, it escalates to the full-capability model and pays full cost.

In a working cascade, roughly 70% of queries resolve at the bottom tier, 20% at mid-tier, and 10% at full cost, for a net 40–50% reduction in inference cost with accuracy held constant. The accuracy floor is the gate: a cascade that degrades output quality has pushed the cost onto the customer.

Step 4: Waste Elimination (Weeks 6–12)

Goal: prevent expensive calls from happening.

Waste Type	Solution	Savings
Failed requests	Improve error handling, input validation	10–15%
Retries	Better backoff strategy, queue management	5–10%
Duplicate requests	Request deduplication, semantic caching	20–40%
Dev/test bleed	Shut off dev and test environments after hours	10–20%
Low-value features	Disable AI for unprofitable use cases	15–30%
Inefficient prompts	Prompt optimization, remove unnecessary context	5–15%

Expected outcome: layer 5 waste reduced 50–70%. Because layer 5 is 10% of the estate, that lands as 5 to 7% of total AI spend, plus whatever the low-value-feature and prompt work returns in layers 1 and 2.

Step 5: Governance and Continuous Improvement (Weeks 8–16, ongoing)

Goal: embed cost discipline as operational governance.

• Cost budgets and alerts per system, team, and feature
• Cost tied to the product roadmap so feature owners see what their feature spends
• Model performance monitoring per tier, tracking accuracy decay
• Quarterly reviews measured on cost-per-outcome rather than cost-per-token
• Continuous infrastructure optimization: cache hit rates, GPU utilization, latency

Expected outcome: cost optimization becomes an operating procedure, and a 20–30% year-over-year improvement compounds.

Self-Assessment Rubric

Score the portfolio company on each layer. Most mid-market companies land in Weak or Developing on layers 4 through 6 and do not know it, because the instrumentation that would tell them was never built.

Layer	Weak	Developing	Strong
1. API Costs	One vendor invoice, no per-call metadata	Spend split by system, not by feature or user	Every call tagged; spend rolls up to a named feature owner
2. Infrastructure	GPU and vector DB spend unreviewed since procurement	Utilization reported, no action loop	Batching, quantization, caching live; hit rate and utilization on the deck
3. Tooling	Overlapping platforms bought team by team	Inventory exists, consolidation not started	One stack, deliberately chosen, open-source alternatives evaluated
4. People	AI labor cost never separated from general engineering	Hours tracked, not costed against AI outcomes	AI labor costed and reported next to inference spend; review automated
5. Waste	Failed and duplicate calls unmeasured	Waste visible in logs, no owner or target	Weekly waste metric, named owner, target, and a trend line
6. Risk	Compliance and incident cost unbudgeted	Risk register exists, not costed	Preventable incident cost modeled and prevention funded as a line item

For a scored read on a specific portco, book a call.

Illustrative Savings Scenarios

The figures below are modeled from the layer shares above, not results from named CCA engagements. They show the steady-state position after two to three years of the Step 5 loop, which is why they sit above the 20 to 34 percent first-year band.

Company Profile	Pre-Optimization	Post-Optimization	Savings
B2B SaaS, AI support across 50K customers	$178K/year	$116K/year	$62K/year (35%)
HealthTech, AI-assisted diagnosis, 10K requests/day	$368K/year	$220K/year	$148K/year (40%)
FinTech, AI trading, 500K daily API calls	$2.4M/year	$1.03M/year	$1.37M/year (57%)

At portfolio scale the arithmetic carries. Five portfolio companies spending a combined $300K/month, or $3.6M a year, recover roughly $1.08M in the first year at a 30% rate. Held under the Step 5 loop, that run rate approaches $1.8M a year by the second or third year. Against $10M of EBITDA that is a 10.8% lift in year one and around 18% at steady state, worth roughly $9M of enterprise value at a 5x multiple, before a dollar of new revenue. Again, illustrative: the real number depends on the estate, and CCA quantifies it per company during the assessment.

The layer shares themselves draw on CloudZero's AI cost report and on published inference-cost research from OpenAI and Anthropic, tested against what CCA sees in portfolio company estates.

Crosswalk: Cost Layers and AI Investment Categories

The AI Value Creation Framework has its own set of six, the 6 AI Investment Categories. The two sets are axes of one matrix.

Investment categories describe where capital is deployed across the AI lifecycle: Foundational Infrastructure, AI/ML Platform and Tooling, Data Preparation and Engineering, Model Development and Fine-Tuning, Deployment and Integration, Ongoing Operations and Governance. They are independent of accounting type.

Cost layers describe what any one of those categories costs: API, Infrastructure, Tooling, People, Waste, Risk. A single category such as Model Development and Fine-Tuning decomposes into all six layers, and its People layer usually dominates.

Question the executive is asking	Framework that owns it	Verb
Where should the next AI dollar go?	AI Value Creation	Allocate
Which dollars are already being wasted?	AI Cost Optimization	Recover
Did the investment produce anything?	AI Value Attribution	Verify

The vocabulary is locked per CCA decision D-076. Allocation belongs to Value Creation, recovery to this framework, verification to Value Attribution. A row in the investment-category axis and a column in the cost-layer axis intersect at a real number. That intersection is what makes the pair a matrix.

Attribution sits above both axes and depends on this one. Its Level 1 is the same AI cost figure the six layers decompose, so a company that counts only layers 1 through 3 is working with 45% of its true spend as the denominator. Every value claim it makes at Levels 4 and 5 is overstated by the missing 55%, which is why the cost work runs before or alongside the attribution work rather than after it.

Relationship to CLEAR™

Layer 4 and layer 5 savings land in the CLEAR™ Leverage phase: internal, productivity and margin, EBITDA this year. Layer 6 risk reduction lands in Realize, where the value shows up as protected exit value and a diligence answer the buyer accepts.

The detection mechanism sits inside the Enterprise AI Control Plane, in its Enterprise Operations pillar. Logging and monitoring is what makes layer 5 visible; the compliance and assurance work is what prices layer 6. Cost optimization runs as a co-pillar of governance.

The Positioning Line

Smart governance doesn't cost money. It reveals where money is being wasted and creates the mechanism to fix it. Cost optimization is a side effect of good governance, not a separate engagement.

Frequently Asked Questions

What is the AI Cost Optimization Framework? A six-layer model of AI spend (API, infrastructure, tooling, people, waste, risk) with a share of spend and a visibility rating attached to each layer, plus a five-step methodology for working all six. Governance is the detection layer that makes the invisible layers measurable.

How much can a portfolio company actually recover? Multiply each layer's share of spend by its recoverable rate and the six layers sum to 20 to 34 percent of total AI spend in the first year. Conventional practice works layers 1 and 2 only, which are the two already visible on a vendor invoice, and leaves the 55 percent sitting in layers 4 through 6 untouched. Programs that hold the Step 5 governance loop compound from there at 20 to 30 percent year over year.

Which layer holds the most money? People, at 40% of total AI spend. It also carries a Low visibility rating, because that cost sits in engineering headcount lines that finance does not classify as AI spend.

How is this different from a FinOps practice? FinOps disciplines cloud and infrastructure consumption, which maps to layer 2 and part of layer 1. This framework extends the same discipline across labor, waste, and risk, and it detects cost through governance instrumentation, which billing exports cannot see. A mature FinOps function is a head start on two of six layers.

How long before savings show up? Cost visibility takes two weeks. Waste elimination starts producing measurable reduction between weeks 6 and 12. The governance loop that compounds the result runs from week 8 onward and stays on as an operating procedure.

Do we have to switch to a cheaper model? No. Confidence-based cascading keeps the full-capability model available for the queries that need it and routes the rest down. The accuracy floor is set first, and the cascade is tuned to hold it.

What is confidence-based model cascading? A routing pattern: the cheap, fast model answers first and reports a confidence score. Above 0.8 the answer ships, between 0.7 and 0.8 it escalates to a mid-tier model, below 0.7 it escalates to full capability. Typical distribution is 70 / 20 / 10 across the three tiers, for a 40–50% inference cost reduction with accuracy held.

How does governance reduce cost when governance is usually a cost? Governance adds instrumentation, and instrumentation is what surfaces failed calls, duplicate requests, orphaned dev environments, and unowned spend. The controls also stop the low-value usage that generates those costs. The savings in layers 4, 5, and 6 exceed what the controls cost to run.

How much of AI spend is waste? Layer 5 runs about 10% of total spend in a typical mid-market estate, and 40–60% of it is removable once measured. Layer 4 carries a second kind of waste: manual review work that automation can absorb, worth 15–30% of that layer.

How do the 6 Cost Layers relate to the 6 AI Investment Categories? They are two axes of one matrix. Investment categories say where capital is deployed across the lifecycle; cost layers say what any one of those categories costs. See the crosswalk section above.

What does this do to EBITDA and the exit multiple? Savings in layers 1 through 5 drop to EBITDA in the current year, which is CLEAR™ Leverage. Layer 6 protects exit value by removing preventable incidents and producing the governance evidence a buyer asks for, which is CLEAR™ Realize.

Who should own this inside a portfolio company? The CFO owns the number and the CTO owns the instrumentation, with a named owner per layer. Layer 5 in particular fails whenever it is everyone's job, which is the default state in most estates.

What do we need in place before starting? Access to vendor billing, infrastructure logs, and the engineering roster. Everything else gets built during Step 1. A portfolio company with no per-call metadata today can still complete cost visibility inside two weeks.

Does this apply to a company still running AI pilots? Yes, and it is cheaper to apply then. Instrumenting cost while the estate is small means the governance is already there when the estate scales, before it has to be retrofitted across production systems.

Next Step

Bring the AI spend and the thesis. CCA walks the six layers against the real estate, scores each one, and returns the recoverable number per layer with an owner attached. Book a call to run it against a specific portfolio company.

v1.2 · Updated August 2026