Using Azure Event Hubs As A Message Hub For IoT Scenarios
When we build apps for larger IoT scenarios or working on Big Data solutions, one major challenge would be to handle millions of messages received from hundreds of systems where these messages will be subscribed by hundreds of other systems. To handle this kind of problem using our traditional enterprise messaging systems and publish-subscribe message brokers would be really a challenge as we need to handle massive amount of stream of data in a second, in which we may need to process the data and events in near real-time. Our most of the existing publish-subscribe message brokers would be failed when we deal with millions of messages in a second. Technologies like Apache Kafka solves this problem by providing a high-throughput distributed messaging system with a distributed commit log. Kafka is designed for cluster-centric design where you may need to leverage distributed coordination tools such as Apache ZooKeeper for coordinating and synchronizing the cluster nodes. This may leads to lot of operational complexity when working with larger distributed systems.
Azure Event Hubs provide easy provisioning of capacity to ingest events from millions of devices while preserving event order on a per device basis. Support for AMQP and HTTP allow many platforms to work with Event Hubs. Native client libraries also exist for popular platforms. Azure Event Hubs is a brilliant technology which can be used as Event Hub / Message Hub for your modern apps, especially for your IoT apps and Big Data solutions. Azure Event Hubs is Cloud powered managed service so that it removes the complexity for provisioning, managing, scaling of your brokered systems.
We have developed an example app for using Azure Event Hubs as a message hub for IoT (Internet of Things) scenarios. The project source is available on github at https://github.com/AccelNA/azure-guidance/tree/master/EventHub.
Here is the details for the example app:
Technologies:
- Azure Event Hub
- Azure App Services Web App
- SignalR
Solution Architecture
Problem Domain
- Build a message hub for IoT scenarios for sending millions of messages and events to the Cloud at massive scale, with low latency and high reliability.
- Process the massive amounts of messages and events in near real-time.
Apps in the Demo:
- Android app representing automotive vehicles.
- Android app for receiving messages from IoT devices and sending it to IoT Message Hub.
- Real-time web app for receiving messages from IoT Message Hub and providing real-time insight into the browser UI.
Solution Workflow
- When automotive vehicles are moving towards roads, notify the GPS Coordinates to Android app for sending these events into an IoT Message Hub powered by Azure Event Hub. (For the sake of the demo, we use an Android app instead of automotive vehicles)
- The Android app which receives the events from automotive vehicles, send the messages and events into IoT Message Hub.
- A real-time web app running on Azure App Services Web App receives the events from IoT Message Hub and provide real-time insight on the data received.
Azure Services
- Azure Event Hub – Working as the IoT Message Hub.
- Azure App Services Web App – Running the real-time web app with SignalR.
Source Code
The project source including two android apps and a ASP.NET SignalR app is available here: https://github.com/AccelNA/azure-guidance/tree/master/EventHub/