Master Amazon Bedrock model evaluation to choose accurate and reliable AI models

Foundation models have revolutionised AI applications but evaluating them effectively presents unique challenges. While these powerful models can handle diverse tasks, they may produce inconsistent results or exhibit issues like hallucination and toxicity. Even when multiple models perform similarly, determining the optimal choice for specific use cases requires systematic evaluation across multiple dimensions including accuracy, robustness, quality, and responsibility.

This course will explore the fundamentals of model evaluation and why it matters, Amazon Bedrock’s built-in evaluation tools and capabilities, key considerations for evaluating models in different scenarios and best practices for systematic model assessment.

Through hands-on examples and practical guidance, you’ll learn how to leverage Bedrock’s evaluation framework and the right foundation models for your specific needs.

You’ll access learning content and practical activities online through Amazon Web Services (AWS), alongside your FutureLearn course experience, giving you direct access to AWS learning as you progress.

Syllabus

Week 1
Mastering Model Evaluation in Amazon Bedrock: From Basics to Best Practices
- Mastering Model Evaluation in Amazon Bedrock: From Basics to Best Practices
  In this activity, you will learn foundation model evaluation principles in Amazon Bedrock, run automatic evaluations using built-in metrics, and apply best practices to select optimal models for your use cases.

When would you like to start?

Start straight away and join a global classroom of learners. If the course hasn’t started yet you’ll see the future date listed below.

Available now

Learning on this course

On every step of the course you can meet other learners, share your ideas and join in with active discussions in the comments.

What will you achieve?

By the end of the course, you‘ll be able to...

Explain the fundamentals of model evaluation in Amazon Bedrock
Practice hands-on experience running evaluations
Perform practical skills for selecting and implementing models based on evaluation results
Apply best practices for model evaluation and selection

Who is the course for?

This course is ideal for Architects, Cloud Operators, Data Engineers, Data Scientists, Developers. DevOps Engineers, Infraestructure Engineers, Trainers.

Who developed the course?

Amazon Web Services (AWS)

AWS is the world’s most comprehensive cloud, enabling organizations to accelerate innovation, reduce costs, and scale more efficiently.

What's included?

Amazon Web Services (AWS) are offering everyone who joins this course a free digital upgrade, so that you can experience the full benefits of studying online for free. This means that you get:

Unlimited access to this course

Includes any articles, videos, peer reviews and quizzes

A PDF Certificate of Achievement to prove your success when you’re eligible

Learning on FutureLearn

Your learning, your rules

Courses are split into weeks, activities, and steps to help you keep track of your learning
Learn through a mix of bite-sized videos, long- and short-form articles, audio, and practical activities
Stay motivated by using the Progress page to keep track of your step completion and assessment scores

Join a global classroom

Experience the power of social learning, and get inspired by an international network of learners
Share ideas with your peers and course educators on every step of the course
Join the conversation by reading, @ing, liking, bookmarking, and replying to comments from others

Map your progress

As you work through the course, use notifications and the Progress page to guide your learning
Whenever you’re ready, mark each step as complete, you’re in control
Complete 90% of course steps and all of the assessments to earn your certificate

Want to know more about learning on FutureLearn? Using FutureLearn