• FutureLearn logo

Introduction to Data Engineering with Microsoft Azure 1

Discover essential data engineering skills and processes using Microsoft Azure services.

670 enrolled on this course

A male Azure data engineer sat in a sever room, connecting the laptop on his lap to the server tower next to him, to begin his data analytics.

Introduction to Data Engineering with Microsoft Azure 1

670 enrolled on this course

  • 6 weeks

  • 4 hours per week

  • Digital certificate when eligible

  • Introductory level

Find out more about how to join this course

Gain the skills to pass the DP-203: Data Engineering on Microsoft Azure exam

Microsoft logo This course has been created in partnership with Microsoft.

Over recent years, the data generated by systems and devices has increased massively.

On this course, you’ll explore the generation, storage, and management of data using various technologies and platforms and prepare to take the DP-203 exam.

Learn the fundamentals of Azure for the data engineer

Data professionals must understand the evolving data landscape and the roles and technologies that have changed with it.

You’ll investigate data platforms, including cloud technologies, and examine a data engineer’s role in helping organizations benefit from technological advances.

Improve data integration using Azure Synapse Analytics

A data engineer’s responsibilities include building and maintaining secure data processing pipelines, and explaining these processes to stakeholders.

Using Azure Data Factory and Azure Synapse Pipeline, you’ll learn to manage data pipelines and build analytical solutions that align with business requirements.

Identify new organizational opportunities using emerging technologies

Using tools such as Apache Spark, you’ll be able to boost the performance of big-data analytic applications, taking your data visualization and analysis skills to the next level.

With a range of exercises aimed to get you comfortable working across Azure’s suite, you’ll finish this course able to optimize, monitor, and manage your data engineering workload, whatever the scale.

By the end of this course, you’ll have gained the introductory knowledge in preparation for the DP 203 exam. By continuing your learning with Introduction to Data Engineering with Microsoft Azure 2, you’ll equip yourself with all the necessary knowledge to pass the exam and progress your career in data engineering.

Syllabus

  • Week 1

    Azure for the Data Engineer

    • Understand the evolving world of data

      Learn how data systems are evolving and how the changes affect data professionals. Explore the differences between on-premises and cloud data solutions, and consider sample business cases that apply cloud technologies.

    • Survey the services on the Azure Data platform

      Learn about Azure technologies that analyze text and images and relational, nonrelational, or streaming data. See how data engineers can choose the technologies that meet their business needs and scale to meet demand securely.

    • Identify the tasks of a data engineer in a cloud-hosted architecture

      Learn about the responsibilities of a data engineer. Find out how they relate to the jobs of other data and AI professionals. Explore common data engineering practices.

  • Week 2

    Store data in Azure

    • Choose a data storage approach in Azure

      Learn how to use Azure Storage, Azure SQL Database, Azure Cosmos DB, or a combination of Azure resources for a performant solution in your business scenario.

    • Create an Azure Storage account

      Create an Azure Storage account with the correct options for your business needs.

    • Connect an app to Azure Storage

      Create a simple application and add configuration, client library references, and code to connect it to Azure Storage.

    • Secure your Azure Storage account

      Learn how Azure Storage provides multilayered security to protect your data. Find out how to use access keys, to secure networks, and to use Advanced Threat Protection to proactively monitor your system.

    • Store application data with Azure Blob storage

      Build an app that stores user files with Azure Blob storage.

  • Week 3

    Data integration at scale with Azure Data Factory or Azure Synapse Pipeline

    • Integrate data with Azure Data Factory or Azure Synapse Pipeline

      In this module, you will examine Azure Data Factory and the core components that enable you to create large scale data ingestion solutions in the cloud.

    • Petabyte-scale ingestion with Azure Data Factory or Azure Synapse Pipeline

      In this activity, you will learn the various methods that can be used to ingest data between various data stores using Azure Data Factory.

    • Perform code-free transformation at scale with Azure Data Factory or Azure Synapse Pipeline

      In this activity, you will learn how to perform common data transformation and cleansing activities within Azure Data Factory without using code.

    • Populate slowly changing dimensions in Azure Synapse Analytics pipelines

      In this activity, you will learn how to implement Slowly Changing Dimension using Azure Data Factory or Azure Synapse Pipelines.

    • Orchestrate data movement and transformation in Azure Data Factory or Azure Synapse Pipeline

      In this activity, you will learn how Azure Data Factory can orchestrate large scale data movement by using other Azure Data Platform and Machine Learning technologies.

    • Execute existing SSIS packages in Azure Data Factory or Azure Synapse Pipeline

      In this activity, you will see how you can integrate SQL Server Integration Services packages into an Azure Data Factory solution.

    • Operationalize your Azure Data Factory or Azure Synapse Pipeline

      In this activity, you will learn how you can publish your Azure Data Factory work between different environments.

  • Week 4

    Realize Integrated Analytical Solutions with Azure Synapse Analytics

    • Introduction to Azure Synapse Analytics

      Learn the features and components that Azure Synapse Analytics provides to provide a one stop shop for all your analytical needs.

    • Survey the Components of Azure Synapse Analytics

      Learn how to recognize and synthesize speech by using Azure Cognitive Services.

    • Explore Azure Synapse Studio

      Take a tour of the core application used to interact with the various components of Azure Synapse Analytics.

    • Design a Modern Data Warehouse using Azure Synapse Analytics

      Learn how Azure Synapse Analytics enables you to build Data Warehouses using modern architecture patterns.

  • Week 5

    Work with Data Warehouses using Azure Synapse Analytics

    • Design a multidimensional schema to optimize analytical workloads

      In this activity, you will study about designing a multidimensional schema to optimize analytical workloads.

    • Use data loading best practices in Azure Synapse Analytics

      In this activity, you will learn how to use data loading best practices in Azure Synapse Analytics.

    • Optimize data warehouse query performance in Azure Synapse Analytics

      In this activity, you will learn how to optimize data warehouse query performance in Azure Synapse Analytics.

    • Integrate SQL and Apache Spark pools in Azure Synapse Analytics

      In this activity, you will learn about integrating SQL and Apache Spark pools in Azure Synapse Analytics.

    • Understand data warehouse developer features of Azure Synapse Analytics

      In this activity, you will learn how to understand data warehouse developer features of Azure Synapse Analytics.

    • Manage and monitor data warehouse activities in Azure Synapse Analytics

      In this activity, you will learn about managing and monitoring data warehouse activities in Azure Synapse Analytics.

    • Analyze and optimize data warehouse storage in Azure Synapse Analytics

      In this activity, you will learn analyze and optimize data warehouse storage in Azure Synapse Analytics.

    • Secure a data warehouse in Azure Synapse Analytics

      During this activity, you will learn how to secure a data warehouse in Azure Synapse Analytics.

  • Week 6

    Perform data engineering with Azure Synapse Apache Spark Pools

    • Analyze data with Apache Spark in Azure Synapse Analytics

      Apache Spark is a core technology for large-scale data analytics. Learn how to use Spark in Azure Synapse Analytics to analyze and visualize data in a data lake.

    • Ingest data with Apache Spark notebooks in Azure Synapse Analytics

      Learn how to ingest data with Apache Spark notebooks in Azure Synapse.

    • Transform data with DataFrames in Apache Spark Pools in Azure Synapse Analytics

      Learn how to transform data with DataFrames in Apache Spark Pools in Azure Synapse Analytics.

    • Integrate SQL and Apache Spark pools in Azure Synapse Analytics

      Learn how to integrate SQL and Apache Spark pools in Azure Synapse Analytics.

    • Monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics

      Learn how to monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics.

When would you like to start?

Start straight away and join a global classroom of learners. If the course hasn’t started yet you’ll see the future date listed below.

  • Available now

Learning on this course

On every step of the course you can meet other learners, share your ideas and join in with active discussions in the comments.

What will you achieve?

By the end of the course, you‘ll be able to...

  • Explore how the world of data has evolved and how the advent of cloud technologies is providing new opportunities for business to explore.
  • Explore the basics of storage management in Azure, how to create a Storage Account, and how to choose the right model for the data you want to store in the cloud.
  • Create and manage data pipelines in the cloud using Azure Data Factory.
  • Perform different types of analytics through Azure Synapse Analytics components.
  • Explore the tools and techniques that can be used to work with Modern Data Warehouses productively and securely within Azure Synapse Analytics.
  • Perform data engineering with Azure Synapse Apache Spark Pools to boost the performance of big-data analytic applications by in-memory cluster computing.

Who is the course for?

This course is designed for data professionals who want to prepare for the DP 203: Data Engineering on Microsoft Azure exam.

Learners should follow this course with Introduction to Data Engineering with Microsoft Azure 2, to ensure they have all the knowledge required to take the DP 203 exam.

It’s recommended that you already have a solid understanding of data processing languages, as well as parallel processing and data architecture patterns before taking the exam.

Who developed the course?

FutureLearn

FutureLearn is a leading social learning platform and has been providing high quality online courses for learners around the world over the last ten years.

Ways to learn

Buy this course

Subscribe & save

Limited access

Choose the best way to learn for you!

$134/one-off payment

$27.99

For the first two months. Automatically renews

Free

Fulfill your current learning needDevelop skills to further your careerSample the course materials
Access to this courseticktick

Access expires 10 May 2024

Access to 1,000+ coursescrosstickcross
Learn at your own paceticktickcross
Discuss your learning in commentstickticktick
Certificate when you're eligiblePrinted and digitalDigital onlycross
Continue & Upgrade

Cancel for free anytime

Ways to learn

Choose the best way to learn for you!

Subscribe & save

$27.99

For the first two months. Automatically renews

Develop skills to further your career

  • Access to this course
  • Access to 1,000+ courses
  • Learn at your own pace
  • Discuss your learning in comments
  • Digital certificate when you're eligible

Cancel for free anytime

Buy this course

$134/one-off payment

Fulfill your current learning need

  • Access to this course
  • Learn at your own pace
  • Discuss your learning in comments
  • Printed and digital certificate when you’re eligible

Limited access

Free

Sample the course materials

  • Access expires 10 May 2024

Find out more about certificates, Unlimited or buying a course (Upgrades)

T&Cs apply.

Learning on FutureLearn

Your learning, your rules

  • Courses are split into weeks, activities, and steps to help you keep track of your learning
  • Learn through a mix of bite-sized videos, long- and short-form articles, audio, and practical activities
  • Stay motivated by using the Progress page to keep track of your step completion and assessment scores

Join a global classroom

  • Experience the power of social learning, and get inspired by an international network of learners
  • Share ideas with your peers and course educators on every step of the course
  • Join the conversation by reading, @ing, liking, bookmarking, and replying to comments from others

Map your progress

  • As you work through the course, use notifications and the Progress page to guide your learning
  • Whenever you’re ready, mark each step as complete, you’re in control
  • Complete 90% of course steps and all of the assessments to earn your certificate

Want to know more about learning on FutureLearn? Using FutureLearn

Do you know someone who'd love this course? Tell them about it...