Big Data in Motion

To finish off our introduction to big data, Graeme will demonstrate some of the options available when working with a high-velocity problem when your data is in motion (when there is an incoming stream of data to be reported).

Data in motion

This data has no beginning or end. When dealing with a stream of incoming information, the first step is to capture the data reliably. Various technological solutions are available to do so within the Azure platform. As we capture, want to perform stream processing.

Aggregations & Real-Time Reporting

Often our aim with real-time data is to draw conclusions and general sentiments rather than specific values. The values may change over time, but the way they change presents information that can be used to establish trends or identify problems.

Storage

We may also want to write to a data store and combine this with historic data. This is generally referred to as Lambda architecture.

If you’d like to know more about best practices when working with big data, take a look at the documents and links in the See also section below.