GenAI on AWS for startups — the under-$500/mo stack (and how to make it $0).

IWhy startups build GenAI on AWS — and what actually costs money

For an early-stage team, the appeal of building generative AI on AWS is simple: you get access to every major foundation model through one managed API, with enterprise security by default, and you pay only for what you run. There is no GPU procurement, no inference fleet to operate, and no minimum spend. The hard part is not getting started — it is keeping the bill small while you do.

The center of gravity for startup GenAI on AWS is Amazon Bedrock: a fully-managed service that lets you call foundation models from Anthropic (Claude), Meta (Llama), Mistral, Amazon (Nova and Titan), Cohere, Stability AI, AI21, and DeepSeek through a single API, with no servers to manage. Your prompts and outputs are not used to train the base models and stay in your AWS account and Region. For a small team, that combination — many models, zero infrastructure, data governance for free — is why Bedrock, rather than self-hosted inference or a single vendor API, is the default. The complete reference for the platform itself lives at Amazon Bedrock.

The thing to understand before you write a line of code is where the money actually goes. In a typical startup GenAI application there are only a handful of cost lines: model inference (tokens in and out, by far the largest line for most apps), embeddings (cheap, but they add up when you index a large corpus), the vector store behind your retrieval layer, and — only if you choose them — reserved capacity (Bedrock Provisioned Throughput) or any SageMaker endpoints you leave running. Almost every runaway GenAI bill is one of those lines used carelessly: a frontier model where a small one would do, real-time inference where batch would do, or idle reserved capacity nobody turned off.

The good news for a cost-conscious founder is that the levers are few and they are blunt. Pick smaller models, cache repeated context, batch what you can, retrieve instead of stuffing, and reserve capacity only when volume is high and steady. Get those five right and a genuinely useful GenAI feature costs less than a single mid-level SaaS subscription. Get them wrong and the same feature costs five figures a month. The rest of this page is those five levers, the stack that embodies them, the traps that violate them, and the credits that pay for all of it.

IIThe under-$500/month GenAI stack

Here is a concrete, opinionated reference stack a startup can stand up in a day and run for well under $500/month at early-product traffic — a grounded, governed assistant over your own data, on Amazon Bedrock, with cost designed in from the first commit. The dollar figures are representative as of 2026 to show relative scale; always confirm live rates on the AWS Bedrock pricing page.

The architecture is deliberately boring, because boring is cheap and reliable. Documents live in Amazon S3. A Bedrock Knowledge Base turns them into a searchable, grounded retrieval layer for you — it chunks the documents, generates embeddings, stores them in a vector index, and at query time fetches the relevant passages and grounds the model's answer in them, with citations. Answers are generated through the Converse API, which gives one request schema across every model so you can swap models with a one-line change. A Guardrail filters harmful content and redacts PII. And the cost discipline comes from model routing (cheap small model for the easy 90%, frontier model only for the hard 10%), prompt caching (so a long system prompt or retrieved context is not re-billed at full price every turn), and batch for anything offline (embedding the corpus, nightly enrichment).

The single most important decision is the default model. For the bulk of calls — classification, routing, extraction, short answers, drafting — a small, fast model such as Amazon Nova Lite or Nova Micro, Claude Haiku, or a small Mistral is an order of magnitude cheaper than a frontier model and entirely adequate. You escalate to a workhorse like Claude Sonnet or Nova Pro only on the steps that genuinely need stronger reasoning. Because everything runs through the Converse API, that escalation is just a different modelId on the hard path — no second integration. See Amazon Nova for the small-model family and Claude on Bedrock for the reasoning tiers.

IIIThe five levers that keep GenAI burn low

Cost control on AWS GenAI is not a dark art; it is five levers, applied deliberately. A startup that designs all five in from the start rarely gets a surprising bill. These are the same levers a vetted partner would set up for you — there is nothing proprietary about them.

Notice that four of the five levers cost you nothing to adopt — they are choices, not purchases. Model routing is a code branch. Caching is a flag. Batch is an API. Retrieval is a managed feature. Only the fifth lever (reserved capacity) involves a commitment, and the advice there is to delay it. That is the whole reason a real GenAI feature can run for the price of a streaming subscription: the cheap path is the default path, if you design for it on day one.

• Model routing — default small, escalate rarely — The highest-leverage move by far. Send classification, routing, extraction, and short answers to a small model (Nova Micro/Lite, Claude Haiku, small Mistral) and escalate only the genuinely hard reasoning steps to a frontier model. Because the Converse API uses one schema, routing is a branch in your code, not a second integration. This alone is usually a 5–10× difference on the inference line.
• Prompt caching — never re-bill stable context — Agents and chat apps resend the same long system prompt, tool schema, or retrieved document on every turn. Mark that stable context for caching and Bedrock bills the repeated tokens at a steep discount instead of full input price. For a verbose-system-prompt assistant this can cut total input cost substantially.
• Batch — make latency-tolerant work asynchronous — Anything that does not need an instant answer — embedding your corpus, nightly summarization, bulk classification, dataset enrichment — should run as Bedrock batch inference at roughly 50% off on-demand. Treat batch as the default for offline jobs and real-time as the exception.
• Retrieve, do not stuff — managed RAG over giant prompts — Pasting an entire knowledge base or document into the prompt on every call is the most expensive way to give a model context, and it scales linearly with your data. A Knowledge Base retrieves only the handful of relevant chunks per query, so your per-call input stays small no matter how large the corpus grows.
• Reserve capacity last — Provisioned Throughput only when steady — Bedrock Provisioned Throughput (and any SageMaker endpoint) reserves dedicated capacity billed by the hour whether or not you use it. It is cheaper than on-demand only at high, steady volume — and required to serve a fine-tuned model. For a startup with spiky early traffic, on-demand is almost always cheaper. Reach for reserved capacity once volume is genuinely high and flat, not before.

IVThe common cost traps — and the cheap fix for each

Runaway GenAI bills almost never come from one big mistake; they come from a handful of recurring, avoidable patterns. Each maps directly to one of the five levers being ignored. Here are the traps that catch startups most often, and the fix for each.

Two of these deserve emphasis because they are the silent ones. Idle reserved capacity is dangerous precisely because it is invisible in your code — a Provisioned Throughput commitment or a forgotten SageMaker real-time endpoint keeps billing at full rate whether you send it one request or none. For a startup with bursty traffic, that is money set on fire. No spend visibility is the meta-trap: tag your GenAI resources, set an AWS Budgets alert at a threshold that would worry you, and enable Bedrock model-invocation logging so you can see token volume by feature. Catching a cost problem on day two is trivial; catching it on the invoice four weeks later is a board conversation.

VBedrock or SageMaker? The startup decision rule

Startups routinely over-think this. For the vast majority of early-stage GenAI features, the answer is Bedrock, and SageMaker is a later, optional addition for specific needs. Here is the honest decision rule and where each tool actually fits.

Amazon Bedrock answers the question most startups are actually asking: "I want to use existing foundation models through a managed, secure API with the least operational overhead." You make an API call, you pay per token, AWS runs the inference fleet, and your data governance comes for free. For shipping a chat assistant, a RAG application, a content generator, an extraction pipeline, or an agent, Bedrock is the cheaper and faster choice — there is no cluster to size, no endpoint to keep warm, and no GPU budget to defend.

Amazon SageMaker answers a different question: "I need to own the ML lifecycle." That means bringing your own model or architecture, running custom training, controlling the serving infrastructure, or doing classical (non-foundation-model) machine learning — a recommendation system, a forecasting model, a custom vision model, fraud scoring. SageMaker gives you full control of training and deployment, which is exactly what you want for those workloads and exactly the overhead you do not want for a standard GenAI feature. For a startup, the cost caution with SageMaker is real-time endpoints: they bill hourly while running, so an always-on endpoint at low traffic is one of the easier ways to overspend. The full head-to-head is at Bedrock vs SageMaker; pricing detail at SageMaker pricing.

The two are complementary, not competing. A common startup architecture uses Bedrock for the GenAI application layer and adds a single SageMaker model later for the one thing no foundation model covers — both in the same AWS account, both fundable by the same credits. The default for an early-stage team is: start on Bedrock; add SageMaker only when a specific workload genuinely requires owning training or non-FM ML. Do not stand up a SageMaker training pipeline to do something a Bedrock API call already does.

VIBuild it yourself vs route to a vetted partner

A small team can absolutely build the under-$500 stack alone — none of the five levers requires specialist knowledge. But there are two recurring situations where routing to a vetted AWS partner is the faster, cheaper path, and one of them is the reason this whole thing can cost you nothing.

The first situation is capacity. Most early-stage teams are one or two engineers deep on infrastructure, fully allocated to product. Standing up RAG with proper data residency, configuring Guardrails, wiring model routing and caching, and setting spend guardrails is a few days of focused work — days a two-person team often does not have without dropping the roadmap. A partner who has built the same pattern many times does it faster and sets the cost defaults correctly the first time.

The second situation is the credits, and this is the headline. AWS funds generative-AI builds through credit programs that are largely partner-filed and invisible on the public Activate page: Activate Portfolio (up to $100K) for institutionally-funded startups, a dedicated Bedrock/GenAI proof-of-concept track ($10K–$50K) for a defined GenAI build, and the competitive Generative AI Accelerator (up to $1M) for AI-first companies. You generally cannot self-serve the large tiers; they are submitted by an AWS partner through the ACE program or by a VC with Portfolio access. This is precisely what CloudRoute does — we route you to a vetted partner who files the credit application and, if you want hands, builds the workload with you. Because AWS funds both the credits and the partner engagement, you pay $0.

Put the two together and the economics invert. The under-$500/month stack was already cheap. Routed through CloudRoute to a partner who secures the credits, the first many months of that bill are covered by AWS, and the build help is funded by AWS too. The cost-conscious answer to "how do we afford GenAI on AWS?" for most startups is not a smaller stack — it is letting AWS pay for the one you already designed. See AWS credits for generative-AI startups and $100K AWS credits.

VIIA startup's first GenAI build on AWS, step by step

Concretely, here is what the first build looks like — the order of operations to get a grounded, governed, cost-controlled assistant live, with the cost levers baked in from the start rather than bolted on after the first scary invoice.

The whole sequence is a week of part-time work, not a quarter. Critically, the cost levers go in before traffic, not after — which is the difference between a GenAI feature that stays cheap forever and one that has to be re-architected the month it gets popular. And because the credit application runs in parallel, the team's first real Bedrock invoice is often already covered by AWS credits before it arrives.

• Day 0 — enable access, scope IAM — In the Bedrock console, request model access to one small default model (Nova Lite or Claude Haiku) and one embeddings model (Titan or Cohere) in your chosen Region. Attach an IAM policy scoped to those specific model ARNs. Pick the Region for data-residency and latency now.
• Day 0–1 — first Converse call — Make a first call through the Converse API against the small model. One schema, one modelId string. This is your baseline; everything else is added around it. Set maxTokens so output cannot run away.
• Day 1–2 — add grounding with a Knowledge Base — Point a Bedrock Knowledge Base at a folder of documents in S3. It chunks, embeds (run that embedding pass as batch), stores vectors, and retrieves grounded context with citations at query time. You now have RAG without building a retrieval stack.
• Day 2 — wrap it in a Guardrail — Configure a Guardrail for PII redaction and any denied topics, applied consistently across models. Governance is now a config, not a project.
• Day 2–3 — design in the cost levers — Add a routing branch (small model by default, frontier only on hard steps), turn on prompt caching for the system prompt and retrieved context, and confirm offline jobs run as batch. These are the choices that keep the bill flat as traffic grows.
• Day 3 — set spend guardrails — Tag the GenAI resources, set an AWS Budgets alert, and enable Bedrock model-invocation logging so token volume by feature is visible. You will catch any cost problem on day one, not on the invoice.
• In parallel — secure the credits — While you build, a routed AWS partner files the Bedrock/GenAI POC and Activate Portfolio applications via ACE. Founder time is roughly half an hour of inputs; credits typically land in the account within a couple of weeks, covering the bill you just minimized.