Cloud Service Components of an IoT Solution

In the last two steps, we explored IoT hardware and software. In this step, we’ll take a look at the cloud components of an IoT solution.

As you’ve learned, the cloud services you choose are an essential part of your overall solution. In fact, the cloud services used in your solution constitutes the ‘I’ in IoT. There are options from many of the larger companies participating in this space as well as offerings from startups and medium-sized businesses. You can explore the individual offerings on your own. In this step, we’ll look at categories of services these companies offer in order to give you an idea of how cloud services fit into an overall IoT architecture.

Cloud Gateways

In an earlier step in this activity, we briefly looked at the concept of a field gateway which is a piece of hardware that brokers communication between IoT devices and cloud services. Cloud gateways do more than broker communication. They provide a set of services that devices can run either locally or in the cloud. Cloud gateways can provide workloads such as:

• Authentication and authorisation
• Message brokering
• Data storage and filtering
• Data analytics
• Functions (discrete code blocks that perform specific tasks).

Data Storage Options

Given the centrality of data in an IoT solution, figuring out the right cloud-based data storage and retrieval options ranks high on the list in terms of importance. IoT devices can generate enormous amounts of data very quickly and storing high volumes of data in the cloud can become expensive but also unwieldy. You need to be able to do something with the data, and too much of it can make analytics and decision-making harder.

Cloud service providers are continually updating their data services to make it easier and more cost-effective for organisations to store, manage, and analyse data. Even so, a thorough analysis of cloud storage technical options and prices should be a fundamental part of any IoT architecture. For example, some architectures may demand a multitiered approach with some data being stored on the device, other stored in on-premise databases and other data stored in the cloud. Depending on the required architecture, make sure the cloud services you choose supports your needs.

There are other concepts to be aware of when considering cloud storage. Data is often time series data and is required to be stored where it can be used in visualisation and reporting, as well as later accessed for additional processing. It’s common to have data split into ‘warm’ and ‘cold’ data stores. The warm data store holds recent data that needs to be accessed with low latency. Data stored in cold storage is typically historical data. Most often the cold storage database solution chosen will be cheaper in cost but offer fewer query and reporting features than the warm database solution.

A common implementation for storage is to keep a recent range of telemetry data (for example, the last day, week, or month) in warm storage and to store historical data in cold storage. With this implementation, the application has access to the most recent data and can quickly observe recent telemetry data and trends. Retrieving historical information for devices can be accomplished using cold storage, generally with higher latency than if the data were in warm storage.

Cloud service providers may supply services to support both types of storage and make managing data across these types much easier.

You can read more about the warm and cold storage different technologies Microsoft Azure provides for managing these storage options in section 3.5 of the Azure Reference Architecture document.

Analytics Services and Data Visualisation

Analytics

Once data is captured and stored, it only becomes useful when it provides insights into the physical world from which your IoT devices have captured the data. This is where analytic services come into play.

Azure Analysis Services, for example, enables architects to use advanced mashup and modelling features to combine data from multiple data sources, define metrics, and secure data in a single trusted tabular semantic data model. The data model provides an easier and faster way for users to browse massive amounts of data for ad-hoc data analysis.

Without analytics, data collected from IoT would be too voluminous and unstructured from which to visualise or gain insights. Analytic services enable architects to build meaningful relationships between sets of data in order to make it easier to manage. For example, Azure Stream Analytics can take stream data from IoT devices and engineers can specify a transformation query that defines how to look for data, patterns, or relationships. The transformation query leverages a SQL-like query language that’s used to filter, sort, aggregate, and join streaming data over a period of time.

Data Visualisation

Stream analytics can help condition data so it’s easier to manage, and provides models that give insight into what you need to understand or learn. Once the data is conditioned and you’ve created the right models, the data can be visualised using tools such as Microsoft’s PowerBI or Tableau.

Data visualisation tools can take input from various data streams and combine them into dashboards that can be used to tell a story about the data collected. Ultimately, this is the goal of IoT.

Machine Learning

Machine learning (ML) is one of the more exciting developments in modern computer science. It’s a complex field but one that’s producing significant positive results with large datasets. As we’ve said throughout this course, IoT devices produce large volumes of data. Analytic systems help engineers model the existing data in meaningful ways. Machine learning takes this a step further and can make predictions about what new data will show and provide insights that wouldn’t be possible without the machine learning algorithms.

As the name states, the technology gives computers the ability to learn (predict) from data by expressing trends or a direction future data will take. This can provide engineers with a powerful mechanism for enabling a wide variety of scenarios.

Using big data and machine learning to predict purchasing decisions is one simple example. Suppose a retailer has warehouse space in various cities and needs to determine which items to stock in those cities in order to be able to get products to customers in the most efficient way.

With machine learning, the retailer can predict that a given set of users that purchase a specific television tend to buy a particular type of cable and other accessories such as tv stands and audio equipment. This allows the retailer to keep those items in the warehouse when television sales are popular so that if a customer orders the cable or other accessory, the item can be shipped quickly to the customer.

Can you think of other, IoT-specific scenarios where machine learning would enable various scenarios that could make the IoT architecture more effective?

Because of the tremendous amount of computing power needed to perform the calculations required for this type of analysis, cloud-based ML technology tends to be the most effective at providing the type of insights machine learning promises.

In this step, we surveyed the various cloud-based services and technologies that make IoT possible. In the next activity, we’re going to explore approaches to IoT Business Solutions.