How to fine-tune an LLM on AWS (2026).
An end-to-end reference for fine-tuning a large language model on AWS, across all three paths. First the decision that should come before any of them: fine-tune vs RAG vs prompt engineering. Then the three ways to do it (Amazon Bedrock fine-tuning, fully managed; Amazon SageMaker training, full control; SageMaker JumpStart, the guided middle), with data prep, the workflow on each path, the GPU and AWS Trainium cost of the training run, how you host the tuned model afterwards, how to evaluate it, and a decision table. Plus who files AWS funding for a fine-tuning build.
- There are three ways to fine-tune an LLM on AWS, and choosing among them is most of the decision. Amazon Bedrock fine-tuning is fully managed (you supply a JSONL dataset; AWS runs everything) — fastest, least control. Amazon SageMaker training is full control (your own training script on GPU or Trainium instances) — most flexible, most work. SageMaker JumpStart is the guided middle — pre-built recipes that fine-tune popular open models with minimal code.
- Before any of them, run the cheaper decision: most "we need to fine-tune" problems are actually a RAG problem (the model lacks your facts) or a prompt problem (the instructions are weak). Fine-tuning teaches behaviour — a style, a format, a narrow skill — not facts, and it goes stale when your data changes. Exhaust prompt engineering and RAG first; fine-tune only when you have good labelled examples and a behaviour that prompting cannot make reliable.
- The cost that surprises everyone is not the training run; it is hosting the result. Many Bedrock custom models are served on Provisioned Throughput (a flat hourly charge, 24/7), though Bedrock can now deploy some custom models for on-demand inference; a SageMaker-tuned model runs on an always-on endpoint billed by the hour. Hosting usually dwarfs the one-time GPU or Trainium training cost.
- Funding: if the work qualifies, a vetted AWS partner can file POC funding to test a fine-tuned model against your base model, or incremental workload (IW) funding if you already run on AWS and are launching new work. For startups this is typically $50K to $150K. Depends on the workload; AWS approves. You apply for Activate credits yourself; a consulting partner cannot file them.
IFine-tuning an LLM — what it is, and when you actually need it
Fine-tuning continues training a pre-trained LLM on your own labelled examples so it learns a specific style, format, or narrow skill better than the base model does out of the box. The most valuable thing to settle before picking an AWS path is whether you need fine-tuning at all — because most teams who think they do actually have a RAG or prompt problem.
A foundation model arrives fluent in language in general but knowing nothing about your output format, your tone, your domain conventions, or the precise way you want a task done. Fine-tuning closes that gap by showing the model many examples of the input it will see and the output you want, and nudging its weights toward producing that output. The result is a custom model: a private derivative of the base model that behaves the way your examples taught it to. Critically, fine-tuning changes how the model behaves, not what facts it knows.
That distinction drives the whole decision. If your model gives wrong answers because it does not know your content — your docs, your product, last night's data — that is a facts problem, and the fix is retrieval-augmented generation (RAG): retrieve the relevant passages and put them in the prompt at request time. Fine-tuning is poor at teaching facts and goes stale the moment your data changes. If the model can clearly do the task but does it inconsistently, that is usually a prompt problem — better instructions and a few in-context examples — which is free and instant to iterate. Only when you have good labelled examples and a behaviour that prompting still cannot make reliable does fine-tuning become the right tool.
When fine-tuning is right, it tends to be for a narrow, stable, behaviour-sensitive task: a strict JSON output schema the base model keeps subtly breaking, a consistent brand voice, a domain-specific classification or extraction. Those are the cases where locking the behaviour into the weights pays off. The companion sibling rag-on-aws covers the retrieval path in depth; the rest of this page assumes you have made the decision to fine-tune and focuses on how to do it on AWS — across the three paths.
One caveat, stated once and meant throughout: exact dollar figures, the precise list of fine-tunable models, the available training instances, and feature details change frequently across Bedrock and SageMaker. Every number here is representative as of 2026 to convey relative cost and the shape of the work. Always confirm the current model list, instance types, and pricing on the official AWS documentation and pricing pages before committing.
Missing facts → RAG. Weak instructions → prompt engineering. A behaviour (style/format/narrow skill) that prompting cannot make reliable, and you have good labelled examples → fine-tune. Climb the ladder; fine-tune last, and never to add facts.
IIThe three paths to a fine-tuned LLM on AWS
AWS gives you three distinct ways to fine-tune an LLM, trading control for convenience. Picking the right one for your team and task is most of the work — the wrong choice means either fighting a managed service that cannot do what you need, or hand-building infrastructure you did not have to. Here they are, most-managed first.
The honest framing: start as managed as your requirements allow, and move toward full control only when a specific need forces it. Most teams should try Bedrock fine-tuning or a JumpStart recipe before writing a custom SageMaker training job, because those reach a working custom model in a fraction of the time. Conversely, teams with a hard requirement — a model Bedrock cannot tune, a custom training objective, or full ownership of the artifact — will be happier going to SageMaker from the start than fighting the managed path.
Path A — Amazon Bedrock fine-tuning (fully managed)
The least-effort path. You upload a JSONL training dataset to Amazon S3, point a Bedrock model customization job at a supported base model, set a few hyperparameters, and AWS runs the entire training process on managed infrastructure; you never see a GPU, a training loop, or a checkpoint. The output is a private custom model in your account, and your training data and the resulting model stay in your account and region (not used to train the base model). Choose this when: your target model is on Bedrock's fine-tunable list (Amazon Nova, Amazon Titan, and open-weight families like Llama and Cohere have been the most reliable), and you want the shortest path to a working custom model. The trade is the least control and that serving the result often needs Provisioned Throughput, though some custom models can now be deployed for on-demand inference (covered in §VI). The amazon-bedrock-fine-tuning sibling is the deep dive.
Path B — Amazon SageMaker training (full control)
The most flexible path. You write (or adapt) a training script — typically using Hugging Face Transformers, PyTorch, and parameter-efficient methods like LoRA/QLoRA — and run it as a SageMaker training job on GPU instances (the ml.p and ml.g families) or AWS Trainium instances (ml.trn, via the Neuron SDK). You control the model, the objective, the data pipeline, distributed-training strategy, and every hyperparameter; you own the resulting weights as an artifact in S3. Choose this when: you need a model or technique Bedrock does not offer, full ownership of the weights, a custom training objective, or large-scale distributed training. The trade is real ML engineering effort. The amazon-sagemaker sibling covers the platform; aws-trainium covers the cheaper-than-GPU training silicon.
Path C — SageMaker JumpStart (the guided middle)
The middle path: most of SageMaker's flexibility, far less of the boilerplate. JumpStart is a hub of pre-trained open models (Llama, Mistral, Falcon, and many more) with built-in fine-tuning recipes — you select a model, point the recipe at your dataset, set a handful of parameters, and JumpStart handles the training script, the instance selection, and the deployment scaffolding for you. You get a SageMaker endpoint hosting your tuned open model without writing the training loop yourself. Choose this when: the model you want is a popular open one available in JumpStart, you want more control and model choice than Bedrock but do not want to hand-write a training job. It is the pragmatic default for fine-tuning open-weight LLMs on AWS.
Need it managed and your model is on Bedrock's list → Bedrock fine-tuning. Want an open model with minimal code → SageMaker JumpStart. Need full control, custom techniques, or to own the weights → SageMaker training. Move toward control only when a concrete requirement forces it.
IIIThe three paths, compared on what matters
The same three paths, lined up against the dimensions that actually drive the choice: how much you build, how much control and model choice you get, what silicon the training runs on, and — the part teams overlook — how you host the result and what that standing cost looks like.
| Dimension | Bedrock fine-tuning (managed) | SageMaker JumpStart (guided) | SageMaker training (full control) |
|---|---|---|---|
| Effort | Lowest — upload JSONL, run a job | Low–medium — pick a recipe, set params | Highest — write/adapt a training script |
| Control | Minimal (a few hyperparameters) | Moderate (recipe parameters) | Total (objective, data, distribution) |
| Model choice | Bedrock fine-tunable list (Nova, Titan, Llama, Cohere…) | JumpStart open models (Llama, Mistral, Falcon…) | Any open model you can train |
| Training silicon | Managed (abstracted away) | GPU or Trainium (ml.p/ml.g/ml.trn) | GPU or Trainium, your choice (ml.p/ml.g/ml.trn) |
| Methods | Managed SFT (+ continued pre-training on some models) | Recipe-driven (often LoRA/QLoRA) | Anything — full SFT, LoRA/QLoRA, custom |
| Who owns the weights | Private custom model in Bedrock | Model artifact in your S3 + endpoint | Model artifact in your S3 — fully yours |
| How you host it | Provisioned Throughput (hourly), or on-demand for supported custom models | SageMaker endpoint (hourly instance) | SageMaker endpoint (hourly instance) |
| Best for | Fastest path; managed models | Open models with minimal code | Custom needs, full ownership, scale |
IVData preparation — the part that actually determines quality
Fine-tuning quality is mostly a data problem, and this is true on all three paths. The model can only learn what your examples demonstrate, so curation and format matter more than any hyperparameter. The shape differs slightly by path, but the discipline is identical.
The format. On Bedrock, training data is a JSONL file ("JSON Lines") in Amazon S3 — one self-contained JSON object per line, each a prompt/completion pair for supervised fine-tuning (chat-formatted models use a messages-style schema; the exact field names depend on the model). On SageMaker training and JumpStart, the format is whatever your script or the chosen recipe expects — commonly JSONL or CSV with an instruction/response (or chat) structure, also staged in S3. In all cases each example pairs the input the model will see with the exact output you want it to produce.
The examples must mirror production. The single biggest determinant of a good fine-tune is that the training prompts look like the prompts the model will actually receive, and the completions look exactly like the output you want back — same format, same length profile, same tone. If production prompts include a system instruction and retrieved context, your training examples should too. A clean dataset of a few hundred to a few thousand high-quality, representative pairs typically beats a much larger noisy one.
Curation and hygiene. De-duplicate examples, remove contradictions (two near-identical prompts with different desired outputs confuse the model), balance the classes or formats you care about, and strip anything you would not want the model to imitate. Always hold out a validation set the model does not train on, so you can measure generalization rather than memorization. And because the data feeds a training process that bakes patterns into weights, scrub or tokenize PII and secrets you do not want learned. This work is path-independent — it matters exactly as much whether you fine-tune on Bedrock, JumpStart, or a custom SageMaker job.
Practically, getting from raw logs, spreadsheets, and documents to a clean training set is where most of the human effort in any fine-tuning project goes, far more than running the job itself. It is also exactly the kind of work a vetted AWS ML partner does efficiently, (§VIII covers who files AWS funding for it).
- Bedrock: JSONL prompt/completion (or messages) pairs in S3, matching your chosen model's schema.
- SageMaker / JumpStart: JSONL or CSV instruction/response (or chat) data in S3, matching the recipe or script.
- Training examples that mirror real production prompts — including any system instruction and context.
- A separate validation set held out from training, to measure generalization not memorization.
- De-duplicated, contradiction-free, class-balanced examples; PII and secrets removed.
- Everything staged in an Amazon S3 bucket in the same Region as the training job.
VRunning the fine-tune — the workflow on each path
With a clean dataset in S3, the act of running the fine-tune differs by path: a managed job on Bedrock, a recipe on JumpStart, or a training job you script on SageMaker. Here is what each looks like, end to end.
Bedrock fine-tuning — a managed customization job
In the Bedrock console (or via API/SDK) you create a model customization job: choose the base model, point it at your training (and validation) data in S3, name the output custom model, set a few hyperparameters — typically epochs (passes over the data), learning-rate multiplier, and batch size — and grant an IAM role that can read your S3 data and write the result. Bedrock provisions the training infrastructure, runs the job, and reports training and validation loss when it finishes. Jobs run from minutes to hours depending on dataset size and epochs. The output is a private custom model registered in your account, ready to evaluate. No infrastructure to manage.
SageMaker JumpStart — a recipe-driven fine-tune
In SageMaker Studio's JumpStart hub you select a fine-tunable open model (e.g. a Llama or Mistral variant), point the built-in fine-tuning recipe at your dataset in S3, choose a training instance (a GPU ml.g/ml.p or a Trainium ml.trn type, depending on the recipe), and set the exposed parameters — epochs, learning rate, and usually whether to use a parameter-efficient method like LoRA. JumpStart supplies the training script and orchestration; you launch the job and it produces a tuned model artifact plus the scaffolding to deploy it to a SageMaker endpoint. You get most of SageMaker's flexibility without writing the training loop.
SageMaker training — your own training job
For full control you write or adapt a training script (Hugging Face Transformers + PyTorch is the common stack), choose the instance type and count — GPU (ml.p/ml.g) or Trainium (ml.trn) — and launch a SageMaker training job that spins the cluster up, runs your script against your S3 data, writes the model artifact back to S3, and tears the cluster down. You decide the technique: full supervised fine-tuning or, far more commonly for LLMs, a parameter-efficient method (LoRA / QLoRA) that trains a small set of adapter weights instead of the whole model — dramatically cheaper in memory and compute, and the standard way to fine-tune large open models affordably. For multi-GPU or multi-node runs you configure distributed training. The resulting weights are entirely yours.
Full fine-tuning updates every weight in the model — expensive in GPU memory and time for a large LLM. Parameter-efficient fine-tuning (LoRA, and quantized QLoRA) trains a small set of adapter weights instead, cutting training cost and memory by a large factor while keeping most of the quality. On SageMaker and JumpStart it is the default way to fine-tune large open models on modest hardware; it is a big reason the GPU/Trainium bill for a fine-tune is often far smaller than people expect.
VIThe cost: GPU/Trainium training vs hosting the tuned model
Fine-tuning has two very different costs, and confusing them is the most common budgeting mistake. The training run is a one-time charge driven by the silicon and how long it runs. Hosting the result is an ongoing charge that, on every path, usually dwarfs the training.
The training run — GPU or Trainium, billed by the hour
A fine-tuning job runs on accelerated compute, and you pay for the instance-time it consumes. On SageMaker and JumpStart you choose the instance and pay its hourly rate for the duration of the job: GPU instances (the ml.p family — high-end NVIDIA for the largest jobs; the ml.g family — cheaper GPUs for smaller fine-tunes) or AWS Trainium instances (ml.trn), AWS's custom training silicon, which is typically meaningfully cheaper per unit of training throughput than comparable GPUs and is accessed via the Neuron SDK (well-supported by JumpStart recipes and Hugging Face). On Bedrock the training infrastructure is abstracted away — you are billed for the customization itself (commonly priced per 1,000 training tokens × epochs) rather than for instance-hours.
The encouraging part: for most fine-tunes — especially LoRA/QLoRA on an open model, or a managed Bedrock job on a typical dataset — the training run is a modest one-time cost, frequently tens to low-hundreds of dollars. Two levers cut it further: parameter-efficient methods (LoRA/QLoRA) slash the compute needed, and Trainium or EC2 Spot-backed training capacity lowers the hourly rate. Training is rarely the line item that makes or breaks the economics.
Hosting the tuned model — the cost that surprises everyone
The real cost is serving the model, and the mechanism differs by path. Many Bedrock custom models are served on Provisioned Throughput (Bedrock can now deploy some custom models for on-demand inference; check the docs for your model): dedicated capacity billed at a flat hourly rate, continuously, the entire time the model is deployed, regardless of traffic. A SageMaker or JumpStart tuned model is deployed to a real-time endpoint: one or more instances (often a GPU ml.g/ml.p) that bill by the hour for as long as the endpoint is up, whether or not requests arrive. Either way, a tuned model sitting idle still bills every hour.
The consequence is stark: the training might be a few hundred dollars once, but hosting the result can cost far more per month than that — and far more than equivalent on-demand inference on a base model would have. This single fact flips the economics of most casual fine-tuning ideas. The honest default: do not fine-tune-and-host unless the volume and quality gains clearly justify a standing hourly bill. For low or spiky traffic, a base model with good prompting and RAG is almost always cheaper overall. Where fine-tuning does pay off, it is high, steady volume on a narrow task — enough traffic that the quality gain is worth a lot and the reserved capacity stays busy. (Bedrock 1- or 6-month Provisioned Throughput commitments, and SageMaker Savings Plans or endpoint auto-scaling, lower the hosting math.) The amazon-bedrock-provisioned-throughput sibling covers the Bedrock side; amazon-sagemaker-pricing covers endpoints.
Training a tuned LLM is a small one-time charge, smaller still with LoRA/QLoRA and Trainium. Hosting it is the real cost: Bedrock custom models usually need Provisioned Throughput (some support on-demand deployment); SageMaker/JumpStart models need an always-on endpoint; both are flat hourly charges that accrue 24/7 whether or not the model is used. Budget for the standing hosting bill, and only fine-tune-and-host for high, steady volume.
VIIEvaluating the tuned model — before you pay to host it
A finished fine-tune is a hypothesis, not a result. Before you put a standing hosting bill behind it on any path, prove it actually beats the base model on your task — and that the improvement justifies the cost and the operational weight.
Start with the training and validation loss the job reports (Bedrock and SageMaker both surface these): validation loss falling alongside training loss is healthy; training loss falling while validation loss rises means overfitting (memorizing rather than generalizing) — a cue to reduce epochs or get more/cleaner data. But loss is only a proxy. The real test is task performance on a held-out evaluation set the model has never seen.
Run a head-to-head: the same prompts through the base model and through your tuned model, scored on the metric that matters for your task — exact-match or schema-validity for structured extraction, a rubric or LLM-as-judge score for style/quality, accuracy/F1 for classification. On the Bedrock path, Bedrock model evaluation can run automated and human-in-the-loop evaluations; on the SageMaker path, SageMaker Clarify / model-evaluation tooling and open-source frameworks do the same. The bar to clear is not "is the tuned model good" — it is "is it enough better than the base model (with good prompting and RAG) to justify a standing hosting cost."
That framing is the honest test of whether fine-tuning was worth it at all. Tally the full cost: one-time GPU/Trainium (or Bedrock) training + ongoing hosting (Provisioned Throughput or endpoint) + the human effort to build and maintain the dataset (a fine-tune drifts as the task evolves and may need re-training). Set it against the measured gain over the cheaper alternatives. Fine-tuning wins cleanly when the task is narrow, stable, high-volume, and format/behaviour-sensitive; it loses when traffic is low or spiky, the task keeps changing, or the real gap was missing facts (RAG) or a weak prompt (prompt engineering) all along.
- Check loss curves first: Falling validation loss with training loss = healthy. Rising validation loss = overfitting; cut epochs or improve data.
- Evaluate on a held-out set: Same unseen prompts through base vs tuned model, scored on a task-appropriate metric. Use Bedrock model evaluation or SageMaker Clarify to systematize it.
- Compare against the cheap alternatives: The bar is beating a base model with good prompting and RAG — not beating nothing.
- Tally the full cost before hosting: Training + standing hosting (PT or endpoint) + dataset maintenance. Fine-tuning is worth it for narrow, stable, high-volume tasks; rarely otherwise.
VIIIWho pays for it: partner-filed funding and Activate credits
Everything above prices a fine-tune if you pay AWS directly. Two kinds of AWS funding can offset it: funding a partner files for you when the work qualifies, and Activate credits you apply for yourself.
Partner-filed funding. If the work qualifies, your matched AWS partner can file POC (proof of concept) funding to test a fine-tuned model against your base model, or incremental workload (IW) funding if you already run on AWS and are launching new work. If you are moving to AWS from another cloud, it can file MAP migration funding. For startups this is typically $50K to $150K. Depends on the workload; AWS approves. Only a partner can request it, and it is tied to real work the partner does with you. See AWS partner funding.
AWS Activate credits. You apply for these yourself at aws.amazon.com/startups/credits: Activate Founders gives $1,000 to start, up to $5,000; Activate Portfolio gives up to $200,000 with an Org ID from your VC, accelerator or angel group if it is an Activate Provider. A consulting partner cannot file Activate for you. Bedrock usage generally draws down promotional credits, but check your credit terms: AWS promotional credits do not pay Savings Plans or Reserved Instance upfront fees.
CloudRoute matches you with one vetted AWS partner that can file that funding and help build the fine-tune: choosing the path (Bedrock, JumpStart or SageMaker), curating the dataset, running the training and setting up evaluation, or telling you honestly that RAG or prompting is the better answer. Startups pay CloudRoute nothing; partners pay CloudRoute a monthly membership plus 10% of AWS-approved funding. If you only need Activate credits, apply yourself; you do not need us.
Fine-tune vs RAG vs prompt — and which fine-tuning path if you do
Two decisions on one screen. First (top three rows): should you fine-tune at all, or is this really a RAG or prompt problem? Then (bottom three rows): if fine-tuning is genuinely right, which of the three AWS paths fits. Representative 2026 guidance, not quotes.
| Approach | Best when… | Effort | Teaches facts? | Changes behaviour? | Ongoing cost shape |
|---|---|---|---|---|---|
| Prompt engineering | The prompt/examples just are not good enough yet | Lowest | Only inline | Yes (via instructions) | None — free to iterate, no hosting |
| RAG (Knowledge Bases) | The model lacks your facts / docs / latest data | Low–medium | Yes | No | Embeddings + vector store + tokens; no model hosting |
| Fine-tuning (any path below) | Need a locked-in style/format/skill; prompting unreliable; good labelled data | Medium–high | No | Yes | One-time training + standing hosting |
| ↳ Bedrock fine-tuning | Fastest path; model is on Bedrock's list; want it managed | Low | No | Yes | Training charge + Provisioned Throughput (hourly) |
| ↳ SageMaker JumpStart | Popular open model; want control with minimal code | Medium | No | Yes | GPU/Trainium training + endpoint (hourly) |
| ↳ SageMaker training | Custom technique/model; full ownership; scale | High | No | Yes | GPU/Trainium training + endpoint (hourly) |
How a match works (example)
- You tell us about the work
You fill in a short form: what you run today, what you want to move, test or build, your stage and your region.
- We pick one partner
We read your answers and pick one vetted AWS consulting partner that fits the work, the stack and the region. We reply within 24 hours; your matched partner reaches out within 5 business days.
- The partner checks POC funding
Your matched partner scopes the test with you. Then it checks whether the test qualifies for POC (proof of concept) funding. A POC can fit whether or not you run on AWS today.
- The partner files, AWS decides
If the work qualifies, the partner files the funding request with AWS. AWS reviews it and decides. The amount depends on the workload.
- You pay nothing
CloudRoute is free for startups. Partners pay CloudRoute a monthly membership plus 10% of AWS-approved funding.
AWS Activate credits are separate, and you apply for them yourself. Use Activate Founders if no investor backs you. Use Activate Portfolio with the Org ID or invite from your VC, accelerator, incubator or angel group if it is an Activate Provider. A consulting partner cannot apply for Activate on your behalf.
Common questions
How do you fine-tune an LLM on AWS?
Should I fine-tune an LLM, or use RAG or prompt engineering instead?
Bedrock fine-tuning vs SageMaker fine-tuning — which should I use?
What is SageMaker JumpStart, and when is it the right path?
What instances and silicon does LLM fine-tuning use — GPU or Trainium?
How much does it cost to fine-tune an LLM on AWS?
How do I host a fine-tuned model on AWS after training?
What is LoRA / QLoRA and why is it used for fine-tuning LLMs on AWS?
How do I evaluate a fine-tuned LLM to know if it was worth it?
Who files AWS funding for a fine-tuning build?
Fine-tune your LLM on AWS with a partner who can file the funding.
CloudRoute matches you with one vetted AWS partner that can build the fine-tune: pick the path (Bedrock, JumpStart or SageMaker), prepare the data, run the training and evaluate it, or tell you honestly not to fine-tune. If the build qualifies, the partner files POC or incremental workload (IW) funding, typically $50K to $150K for startups. Depends on the workload; AWS approves. You apply for Activate credits yourself. CloudRoute is free for startups.