The stages of an ML pipeline
AWS's Well-Architected Machine Learning Lens names six phases, and describes the lifecycle as cyclic and iterative — phases are not strictly sequential, and feedback loops send you back to earlier ones.
The ML lifecycle (Well-Architected ML Lens)
- Business goalthe problem and how success is measured
- ML problem framingwhat to predict, which metric
- Data processingcollect, pre-process, feature engineering
- Model developmenttrain, tune, evaluate
- Deploymentinference in production
- Monitoringdetect degradation early
retrain when performance drifts
| Stage | What happens | Clue words in a question |
|---|---|---|
| Business goal identification | Define the problem, the value and the success criteria | “ROI”, “success criteria”, “stakeholders” |
| ML problem framing | Decide what is observed and what is predicted (the label), and which metric to optimise | “target variable”, “is this classification or regression” |
| Data collection | Gather data from its sources | “ingest”, “label”, “data sources” |
| Data pre-processing | Clean, fill gaps, remove duplicates, convert formats | “missing values”, “outliers”, “normalise” |
| Feature engineering | Create, transform and select the input variables | “new features”, “select variables”, “encode” |
| Training and tuning | Fit the model; search for the best hyperparameters | “hyperparameter”, “epochs”, “tuning job” |
| Evaluation | Measure on data the model has not seen | “validation set”, “test set”, “precision/recall” |
| Deployment | Serve predictions to applications | “endpoint”, “inference”, “batch” |
| Monitoring | Watch quality, drift and cost in production | “drift”, “degradation”, “retrain” |
Where foundation models come from
- Pre-trained models from a provider, used through a managed API — for example the models offered in Amazon Bedrock.
- Open-source pre-trained models, which you can download, adapt and host yourself — for example through SageMaker JumpStart or on your own infrastructure.
- Training a custom model from scratch on your own data — by far the most expensive option, and rarely the right answer for an AIF-C01 scenario.
Using a model in production: managed API or self-hosted
Managed API service versus self-hosted
Managed API (e.g. Amazon Bedrock)
- No servers or GPUs to manage
- Pay per use (for FMs, per token)
- Fastest route to production
- Less control over the model and hosting
Self-hosted (e.g. a SageMaker AI endpoint or your own instances)
- You choose instance types and scaling
- Full control of model version and configuration
- You pay for capacity whether it is used or not
- More operational work: patching, scaling, monitoring
AWS services by stage
The guide names four services for this objective. Two are ML platforms; two are newer tools that show how AWS now expects AI to be used across a business.
| Service | What it is | Where it fits |
|---|---|---|
| Amazon SageMaker AI | AWS's platform to build, train and deploy custom ML models | Data preparation, training, tuning, evaluation, deployment and monitoring of your own models |
| Amazon Bedrock | Fully managed service offering a choice of foundation models through one API | Using and customising FMs without managing infrastructure |
| Amazon Quick | AWS's agentic AI workspace for business users: chat, research, business intelligence and automation across company data and apps | Putting AI in front of business users, and consuming results |
| Kiro | AWS's agentic development tool (IDE and CLI) built around spec-driven development: requirements, design, then tasks | Building the application code around a model |
MLOps in one page
MLOps applies DevOps habits to machine learning. The guide lists its goals; each has a practical meaning.
| Concept | What it looks like in practice |
|---|---|
| Experimentation | Every training run is tracked — data version, code, parameters, results — so the best one can be found and reproduced |
| Repeatable processes | Pipelines, not notebooks: the same steps run the same way every time |
| Scalable systems | Training and inference that grow with data and traffic without a rewrite |
| Managing technical debt | Avoiding one-off scripts, hidden data dependencies and undocumented features that make models fragile |
| Production readiness | Testing, security, versioning and a rollback plan before a model faces customers |
| Model monitoring | Watching prediction quality and input data for drift after deployment |
| Model re-training | Refreshing the model on new data, on a schedule or when monitoring shows decline |
Model metrics: accuracy, precision, recall, F1
For a classifier, every prediction lands in one of four boxes: true positive, false positive, false negative, true negative. The metrics are ratios of those boxes, and each answers a different business question.
| Metric | Formula | The question it answers | Prioritise when… |
|---|---|---|---|
| Accuracy | (TP + TN) ÷ all | How often is the model right overall? | Classes are balanced and both errors cost the same |
| Precision | TP ÷ (TP + FP) | When it says “yes”, how often is it right? | False alarms are expensive — e.g. blocking a legitimate payment |
| Recall | TP ÷ (TP + FN) | Of all the real “yes” cases, how many did it catch? | Misses are expensive — e.g. failing to flag a disease or fraud |
| F1 score | Harmonic mean of precision and recall | A single balance of the two | Both errors matter and classes are imbalanced |
Business metrics
The guide lists cost per user, development costs, customer feedback and return on investment. A model can improve its F1 score and still lose money. Business metrics answer whether the project is worth running: what it costs to build and operate, what each prediction costs, what users say about it, and whether the value it creates exceeds all of that.
Traps the wrong answers are built from
| Tempting but wrong | Do this instead |
|---|---|
| Treating deployment as the last step. | Plan monitoring and re-training from the start; the lifecycle is a loop. |
| Reporting accuracy on a rare-event problem. | Use recall, precision or F1, chosen by the cost of each error. |
| Training a model from scratch when a pre-trained one exists. | Start from a managed or open-source pre-trained model; train from scratch only with a strong reason. |
| Judging a project by model metrics alone. | Pair them with business metrics: cost per user, development cost, feedback, ROI. |
You should now be able to
- Name and order the components of an ML pipeline, and recognise each from a description.
- Describe where foundation models come from: provider APIs, open-source pre-trained models, custom training.
- Compare a managed API service with a self-hosted model.
- Place SageMaker AI, Amazon Bedrock, Amazon Quick and Kiro in the lifecycle.
- Explain MLOps concepts: experimentation, repeatability, scale, technical debt, production readiness, monitoring, re-training.
- Choose between accuracy, precision, recall and F1, and name business metrics that judge an ML project.