Optimize AI economics on Amazon Bedrock with Accenture’s sequenced approach

As enterprises scale AI on Amazon Bedrock, token spend often grows even as per-token prices fall. Without cost allocation tags in place, your cloud invoice shows AI spend as a single line item rather than by team, workload, agent, or model. Cheaper tokens expand adoption, and agentic workloads multiply invocations. As successful AI use cases scale, consumption can rise faster than unit prices drop. The economic question is whether the business value generated grows faster than the cost of the intelligence required to produce it. The Linux Foundation has launched the Tokenomics Foundation to develop open standards for AI cost economics, a sign that managing the economics of AI is now an industry discipline.

In this post, you will learn about six levers for optimizing token spend, how Accenture applied them in production, and how to apply them using Amazon Bedrock.

Accenture applied these levers in four client engagements across telecom, insurance, mining, and a large enterprise deployment. The same levers are available natively on Amazon Bedrock as described in this post. Where savings were measured, token spend fell by about 40–85 percent. Results are specific to each client’s workload and may vary. Client names are withheld by Accenture to protect client confidentiality.

Attribution first: Measuring cost at the business level

Before you optimize anything, you need to see where consumption is occurring, who owns it, and what business value it produces. Cost allocation tags at the team, workload, and agent level are the prerequisite for connecting consumption to accountable owners and, ultimately, to business outcomes. You apply them through Amazon Bedrock application inference profiles, one profile per application or workload, tagged by team, cost center, and environment. After you activate them as cost allocation tags in the AWS Billing and Cost Management console, you can use these tags to trace which teams, applications, and agents are driving spend. They also make it possible to measure the impact of each change you introduce.

Three AWS services provide the underlying telemetry. AWS Cost and Usage Reports (CUR 2.0), delivered through AWS Data Exports, handle financial attribution, breaking down billed inference cost by workload and team after your cost allocation tags are active. Amazon CloudWatch captures operational signals such as invocation counts and latency. Amazon Bedrock model invocation logging records request-level detail, including the model used and the input and output token counts for each call.

Telemetry alone isn’t enough. Engineers can read cost per invocation on a dashboard, but product owners and program leaders need a metric expressed in business terms. Accenture’s Cost per Business Transaction (CpBT) framework bridges this gap. CpBT aggregates all the model invocations, retries, tool calls, and agent handoffs that contribute to one successful business output into a single fully loaded cost figure. It surfaces the true cost of producing a result, including the failed attempts that raw token pricing hides. Tracked against the business value each workflow delivers, CpBT distinguishes a growing bill that reflects scaling success from one that reflects waste.

The six optimization levers

The goal of these six levers isn’t to minimize tokens at all costs. It’s to use the right amount and level of intelligence to deliver each business outcome at the appropriate cost, quality, and risk. The levers are organized into three phases based on the complexity of the change and the investment it requires. AWS and Accenture recommend working through the phases in order, and you can apply each phase independently to the workloads where it will have the most impact.

Phase 1: Configuration only, no infrastructure change

With Amazon Bedrock prompt caching, the model reuses the parts of a prompt that repeat across requests, such as system instructions, tool definitions, and reference documents, so you stop paying full price to process the same content again. Amazon Bedrock stores the key-value state of previously computed prompt prefixes and is designed to skip recomputation on later requests. This is designed to reduce cached input token costs by up to 90 percent for supported models (see Amazon Bedrock Prompt Caching).

Amazon Bedrock batch inference processes non-urgent workloads asynchronously as a group at 50 percent lower cost than on-demand pricing for select models (see Amazon Bedrock Pricing). You submit inference requests as a batch job, and results are written to Amazon Simple Storage Service (Amazon S3) when processing completes. Batch inference suits nightly reports, document processing pipelines, and evaluation runs that don’t need an immediate response.

Both levers add configuration to your existing setup. Prompt caching requires cache checkpoint markers in your prompts, and batch inference requires submitting eligible requests as a batch job instead of calling the on-demand API. Apply caching to interactive, on-demand traffic and batch inference to asynchronous work. Neither requires new infrastructure or architecture changes.

Phase 2: Architectural changes

Amazon Bedrock intelligent prompt routing directs requests across models within the same family, matching each task to the level of intelligence its business value, complexity, and risk require, rather than defaulting every request to the most powerful model. Intelligent prompt routing can reduce costs by up to 30 percent without compromising accuracy (see Amazon Bedrock intelligent prompt routing). Check Amazon Bedrock Pricing for any routing charges. In the following engagements, Accenture’s model tiering and triage routing apply this same routing lever.

Retrieval Augmented Generation (RAG) optimization tunes the retrieval step in the RAG process so that fewer unnecessary context tokens reach the model. Techniques such as hierarchical chunking, metadata-based filtering, and reranking narrow the retrieved context to the most relevant chunks, so you pay primarily for content that improves the answer.

Two complementary Accenture patterns work alongside these AWS native levers. Context scoping limits the data each agent receives to the document slice relevant to its step of the workflow, rather than passing the full case context at every handoff. Context compression summarizes or filters documents before they reach the model. Both reduce input token volume, especially across multi-agent chains.

Phase 3: Infrastructure and training investment

Semantic caching stores responses and reuses them when a new request is semantically similar to one already answered, even if the wording differs. This avoids redundant model invocations and reduces both cost and latency. On AWS, you can implement it with a vector store such as Amazon ElastiCache in front of the model. In an AWS benchmark on 63,796 real chatbot queries, a 0.90 similarity threshold produced a 74.5 percent cache hit rate, 92.3 percent accuracy, and 72.5 percent cost savings (see ElastiCache semantic caching benchmarks). As you apply routing and semantic caching, keep Amazon Bedrock Guardrails on every route, and set semantic cache expiry so that users don’t receive outdated answers.

Model distillation transfers knowledge from a large teacher model to a smaller, faster student model. Distilled models in Amazon Bedrock are up to 500 percent faster and up to 75 percent less expensive than original models, with less than 2 percent accuracy loss for use cases such as RAG. Distillation is best suited to stable, high-volume workloads where the transaction volume justifies the training investment.

In production: Four client engagements

Accenture applied these levers in four client engagements. Each engagement targeted a different cost pattern. In two of them, Accenture applied several changes together, so those results reflect the combined effect rather than any single lever.

The first engagement was a large enterprise deployment where every request went to a frontier model, with no caching, model tiering, or batching. Accenture applied three changes together: model tiering to match each task to the smallest model that could handle it well, prompt caching for repeated context, and batch processing for overnight and non-urgent jobs. Together they cut monthly token cost by 85 percent, and the saving was not separated by change. About 15 percent of tasks still needed the top model for hard, open-ended reasoning, and the rest ran on lighter models. Your split will depend on your workload composition.

The second engagement was a telecom operator with $12 million in annual token spend. Most of that spend came from agents fanning out requests without limits and from every request going to a frontier model regardless of complexity. Accenture applied triage routing, semantic caching, and context compression together. Input tokens fell by 70 percent, and annual token spend fell from $12 million to $3.8 million, a 68 percent reduction. Accenture validated output quality against the operator’s pre-change performance benchmarks.

The third and fourth engagements each applied a single cache layer. An insurer used model-layer prefix caching, reusing the static portions of its prompts across invocations. A mining operator used an application-layer semantic cache, returning stored responses for repeated queries before they reached the model. Across the two engagements, token spend fell by about 40 percent combined, without model routing changes or training investment.

Results

Outcomes ranged from about 40 percent to about 85 percent, reflecting each client’s starting point. The large enterprise deployment began with no optimization in place and reached 85 percent. The telecom operator reached 68 percent on $12 million in annual token spend, and the insurer and mining operator reached about 40 percent combined with one caching lever each. Results are specific to each client’s workload and may vary.

These savings aren’t the end goal. They create capacity to scale AI workloads, reinvest in higher-value use cases, and improve the return generated by each dollar of AI consumption. Two of these engagements applied several changes at once. When you apply one lever at a time, you can attribute each result to a specific change. AWS and Accenture recommend this sequence for your own workloads: connect consumption to owners and outcomes, apply one lever at a time, measure cost, quality, and business value, and build on what works.

Production governance: Four operating tenets

Optimizing AI economics in production requires an ongoing governance cycle spanning cost, quality, risk, and business value, not a one-time technical change. Before you apply any lever, establish a quantitative baseline for both cost and quality so that you can measure the impact of each change against a known starting point. When you validate output quality, use held-out evaluation sets drawn from your actual production traffic rather than synthetic benchmarks. Apply one lever at a time per workload so that you can attribute results to a specific change and roll back cleanly if quality drops below your acceptable thresholds. If you operate in a regulated industry such as financial services, insurance, or healthcare, document your quality test results and rollback procedures as part of your standard change management record.

These four tenets apply regardless of which levers you are using or what phase of the sequencing you are in. Amazon Bedrock AgentCore is an environment where you can build, connect, and optimize agents at scale with any framework or model. It includes AgentCore Evaluations, a capability of Amazon Bedrock AgentCore that provides batch evaluation to support your quality validation cycle as you progress through more advanced levers.

Getting started

You can begin with three steps. First, turn on cost attribution for Amazon Bedrock by applying cost allocation tags at the team and workload level so that your spend is traceable from day one. Second, activate those tags in the AWS Billing and Cost Management console. Then use CUR 2.0 and model invocation logging to identify high-consumption workloads, and assess each one based on spend, business value, quality requirements, risk, volume, and optimization potential. Third, apply Amazon Bedrock prompt caching to that workload’s interactive traffic and Amazon Bedrock batch inference to its asynchronous jobs as configuration-only changes, and measure the result before you move to architectural changes in Phase 2.

The AWS Well-Architected Agentic AI Lens includes model invocation and token cost optimization (AGENTCOST02), a focus area within the cost optimization pillar for agentic workloads. This focus area provides a structured framework you can use to apply these levers at scale across your organization.


About the authors

Ilan Geller

Ilan is a Managing Director at Accenture, focused on artificial intelligence, delivery, and client growth.

Steven Cox

Steven is an Associate Director in the Accenture AWS Business Group and a Principal Architect specializing in AWS. He leads token economics work across Accenture client engagements.

Manoj Ravulapati

Manoj Ravulapati

Manoj is a Senior Customer Solutions Manager at AWS who leads strategic AI initiatives, helping enterprises scale agentic AI.

Rajdeep Banerjee

Rajdeep Banerjee

Rajdeep is a Senior Solutions Architect in the AWS Cloud Center of Excellence. He works with AWS partners to design and optimize generative AI workloads on Amazon Bedrock.

​As enterprises scale AI on Amazon Bedrock, token spend can grow even as per-token prices fall. This post walks through six composable optimization levers, shows how Accenture applied them across four client engagements, and recommends a sequence for cutting AI token spend on your own workloads. Read More

Leave a Reply

Your email address will not be published. Required fields are marked *