Bedrock Intelligent Prompt Routing: how the router picks a model, and when it saves money.

IWhat Intelligent Prompt Routing is

Most apps send every request to one model. Many of those requests would get an equally good answer from a cheaper model. A router makes that choice per request, inside Bedrock.

You create a prompt router and call its ARN where you would put a model ID in a Converse or InvokeModel call. The router looks at each prompt, predicts how good each of its two models would be at answering it, and forwards the request to one of them. The response tells you which model answered.

There is no routing code to write or host. Bedrock runs the prediction and the call. The trade-off is control: the router only chooses between two models in the same family, and you cannot feed it your own performance data.

Routing does not make a model smarter. It stops you paying for a stronger model on prompts that did not need it, or, if you anchor on the cheaper model, it pays for the stronger model only when the prompt seems to need it.

IIHow the router decides: the fallback model and the response quality difference

Two settings drive every decision. Get them right and the rest is measurement.

The fallback model is your baseline. AWS describes it as the anchor: the model that handles a request unless the router predicts the other model will do clearly better. In practice teams often make the cheaper model the fallback and let the router escalate to the stronger one.

The response quality difference sets how much better the other model must be predicted to be before the router switches. AWS's example: with a 10% difference and a smaller model as fallback, the router uses the larger model only when its answer is predicted to be 10% better. A small value means the router switches more often; a large value means it sticks to the fallback more often.

Each request runs the same steps: Bedrock analyzes the prompt, predicts answer quality for both models, applies your criteria, forwards the request to the chosen model and returns the answer with the model it used. You pay a routing fee per request plus the tokens of the model that answered.

IIIDefault routers and configured routers

Start with a default router to see how your prompts route, then configure your own.

Default routers are set up by AWS and use select Amazon Nova and Meta Llama models, depending on the Region. You can try them in the console playground with no setup. Configured routers are yours: you name the router, choose exactly two models in one family, pick the fallback and set the response quality difference, using the console, the CreatePromptRouter API or the AWS CLI. Then you test prompts in the playground and adjust.

• Pick two models with a real price gap: The saving comes from the price difference between the two models. Two similar models save little.
• Choose the fallback on purpose: A cheaper fallback biases toward cost, with escalation for hard prompts. A stronger fallback biases toward quality, with savings only on prompts the smaller model is predicted to handle as well.
• Set the criteria from data: Run a set of real prompts, read which model answered and score the answers. Move the response quality difference one step at a time.
• Swap the model ID for the router ARN: Nothing else in your Converse or InvokeModel call changes. Log the model used on every response.

IVSupported models and limits

Check the AWS list before you design around a model pair. It changes.

On 10 October 2026 the prompt routing page listed these supported models: Amazon Nova Lite, Amazon Nova Pro, Meta Llama 3.1 8B Instruct, Meta Llama 3.1 70B Instruct and Meta Llama 3.3 70B Instruct, each with its own single-Region and cross-Region availability. If you want to route between other models, such as two Claude tiers, check the current list first or write your own routing.

AWS lists three limits: routing is optimized for English prompts; the router cannot adjust its decisions based on your application's performance data; and it may not route well for unusual or specialist use cases, because its quality predictions depend on its training data.

VManaged routing vs. writing your own

The managed router is the lowest-effort option. It is not the only one.

VIWhen routing helps, and when it does not

Ask one question: is my traffic a mix of easy and hard prompts that all hit one model today?

• Helps: mixed-difficulty assistants: A support bot that gets both "what are your hours?" and long troubleshooting questions has an easy majority the smaller model can answer.
• Helps: apps that default to a strong model "to be safe": Most of that traffic may not need the stronger model.
• Little help: uniformly hard traffic: If nearly every prompt needs the stronger model, the router keeps choosing it and there is little to save.
• Little help: already on the cheapest model: There is nothing cheaper to route to.
• Not a fit: routing across providers or non-English traffic: The router works inside one family and is optimized for English. Use your own rules instead.

VIIHow to measure the saving and the quality you keep

A routing saving is only real if answer quality holds. Measure both before and after you change the criteria.

Track four numbers. Routing mix: the share of requests each model answered, from the model-used field. Quality: scores on a fixed evaluation set (automatic metrics, an LLM judge or human review) plus production signals such as retries and escalations. Counterfactual saving: price your logged tokens as routed, then as if every request had gone to the stronger model, and subtract. Routing fee: requests times the per-request price, so the saving you report is net.

If almost everything goes to the stronger model, your criteria may be too loose for switching, or your traffic is simply hard. If quality slips, raise the bar for leaving the stronger model. Keep measuring in production, because traffic changes. See Bedrock model evaluation for building the evaluation set.

VIIIRouting, prompt caching and batch inference together

Routing picks the model. Two other levers lower the price of each call.

Prompt caching bills repeated prompt prefixes at a lower cache-read rate on supported models; see Bedrock prompt caching. Batch inference runs non-interactive jobs on select models at a price the AWS pricing page lists as 50% lower than on-demand. Caching does not work with the batch API, so use routing and caching for interactive traffic and batch for bulk jobs.

IXWho can file AWS funding for a Bedrock build

There are two kinds of AWS money for a Bedrock build. A partner files one. You apply for the other yourself.

Partner-filed funding. An AWS consulting partner can file POC (proof of concept) funding to test a Bedrock workload, whether or not you run on AWS today. If you already run on AWS and are adding new AI work, the partner can file incremental workload (IW) funding instead. For startups this is often $50K to $150K. Depends on the workload; AWS approves. The work has to be real, and the partner has to be the one doing it.

AWS Activate, which you apply for yourself. Activate Founders gives $1,000 to start, up to $5,000, with no investor needed. Activate Portfolio gives up to $200,000 if your VC, accelerator or angel group is an Activate Provider and gives you its Org ID. A consulting partner cannot apply for Activate on your behalf. AWS made Activate credits usable for third-party models on Bedrock in April 2024.

For most startups without a VC, the partner route is the larger of the two. Routing and caching then make whatever funding you get last longer. CloudRoute matches you with one vetted AWS partner for free. Partners pay CloudRoute a monthly membership plus 10% of AWS-approved funding.