Skip main navigation

Real-Time Processing In Azure

In this step, we'll take a look at real-time processing options in the Azure environment.

In the previous step, we saw how we could perform batch processing using some of the Azure resources available to us. In this step, we’ll take a look at real-time processing options in the Azure environment.

Event Hub in Azure

Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters.

The following are some of the scenarios where we can use event hubs:

  • Anomaly detection (fraud/outliers)
  • Application logging
  • Analytics pipelines such as clickstreams
  • Live dashboarding
  • Archiving data
  • Transaction processing
  • User telemetry processing
  • Device telemetry streaming.

Azure Stream Analytics

Azure Stream Analytics is real-time analytics and complex event-processing engine that’s designed to analyse and process high volumes of fast streaming data from multiple sources simultaneously.

Patterns and relationships can be identified in information extracted from a number of input sources including devices, sensors, clickstreams, social media feeds, and applications. These patterns can be used to trigger actions and initiate workflows such as creating alerts, feeding information to a reporting tool, or storing transformed data for later use. Also, Stream Analytics is available on Azure IoT Edge runtime, enabling the processing of data on IoT devices.

The following scenarios are examples of when we can use Azure Stream Analytics:

  • Analyse real-time telemetry streams from IoT devices
  • Weblogs/clickstream analytics
  • Geospatial analytics for fleet management and driverless vehicles
  • Remote monitoring and predictive maintenance of high-value assets
  • Real-time analytics on Point of Sale data for inventory control and anomaly detection
  • Output results to Azure Synapse Analytics, or Power BI.

You can try Azure Stream Analytics with a free Azure subscription.

The rest of this activity will cover processing big data with Azure HDInsight, an easy and cost-effective, enterprise-grade service for open-source analytics.

Join the discussion

Graeme used data from Twitter to demonstrate real-time data and the ‘sentiment’ feature. Would you suggest any modifications to the sentiment algorithm? Or any alternative uses?

Use the Discussion section below and let us know your thoughts. Try to respond to at least one other post and once you’re happy with your contribution, click the Mark as complete button to move on to the next step.

This article is from the free online

Microsoft Future Ready: Fundamentals of Big Data

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education