Book Image

Stream Analytics with Microsoft Azure

By : Ryan Murphy, Manpreet Singh
Book Image

Stream Analytics with Microsoft Azure

By: Ryan Murphy, Manpreet Singh

Overview of this book

Microsoft Azure is a very popular cloud computing service used by many organizations around the world. Its latest analytics offering, Stream Analytics, allows you to process and get actionable insights from different kinds of data in real-time. This book is your guide to understanding the basics of how Azure Stream Analytics works, and building your own analytics solution using its capabilities. You will start with understanding what Stream Analytics is, and why it is a popular choice for getting real-time insights from data. Then, you will be introduced to Azure Stream Analytics, and see how you can use the tools and functions in Azure to develop your own Streaming Analytics. Over the course of the book, you will be given comparative analytic guidance on using Azure Streaming with other Microsoft Data Platform resources such as Big Data Lambda Architecture integration for real time data analysis and differences of scenarios for architecture designing with Azure HDInsight Hadoop clusters with Storm or Stream Analytics. The book also shows you how you can manage, monitor, and scale your solution for optimal performance. By the end of this book, you will be well-versed in using Azure Stream Analytics to develop an efficient analytics solution that can work with any type of data.

Related Content you might be interested in

Current Title:

Small book image
Stream Analytics with Microsoft Azure
Ryan Murphy | Manpreet Singh
Dec, 2017 | 322 pages
Small book image
Azure IoT Development Cookbook
Yatish Patil
Aug, 2017 | 254 pages
Small book image
The Azure IoT Handbook
Dan Clark
Dec, 2023 | 248 pages
Small book image
Implementing Azure Cloud Design Patterns
Oliver Michalski | Stefano Demiliani
Jan, 2018 | 300 pages
Table of Contents (18 chapters)
Title Page
Title Page
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Customer Feedback
Customer Feedback
Preface
Preface
Free Chapter
1
Introducing Stream Processing and Real-Time Insights
Introducing Stream Processing and Real-Time Insights
Understanding stream processing
Understanding queues, Pub/Sub, and events
What is an event?
Summary
2
Introducing Azure Stream Analytics and Key Advantages
Introducing Azure Stream Analytics and Key Advantages
Services offered by Microsoft
Introduction to Azure Stream Analytics
Microsoft Cortana Intelligence suite integration
Summary
3
Designing Real-Time Streaming Pipelines
Designing Real-Time Streaming Pipelines
Differencing stream processing and batch processing
Canonical Azure architecture
Summary
4
Developing Real-Time Event Processing with Azure Streaming
Developing Real-Time Event Processing with Azure Streaming
Stream Analytics tools for Visual Studio
Defining a Stream Analytics query for Vehicle Telemetry job analysis using Stream Analytics tools
Implementation of an Azure Stream Analytics job using the Azure portal
Summary
5
Building Using Stream Analytics Query Language
Building Using Stream Analytics Query Language
Built-in functions
Query language elements
Windowing
Time management and event delivery guarantees
Summary
6
How to achieve Seamless Scalability with Automation
How to achieve Seamless Scalability with Automation
Understanding parts of a Stream Analytics job definition (input, output, reference data, and job)
How to scale queries using Streaming units and partitions
Sample use case 
Configuring SU using Azure portal
Out of order and late-arriving events
Summary
7
Integration of Microsoft Business Intelligence and Big Data
Integration of Microsoft Business Intelligence and Big Data
What is Big Data Lambda Architecture?
Moving to the streaming-based data solution pattern
Evolution of Kappa Architecture and benefits 
Comparison between Azure Stream Analytics and Azure HDInsight Storm
Designing data processing pipeline of an interactive visual dashboard through Stream Analytics and Power BI
Summary
8
Designing and Managing Stream Analytics Jobs
Designing and Managing Stream Analytics Jobs
Reference data streams with Azure Stream Analytics
Summary 
9
Optimizing Intelligence in Azure Streaming
Optimizing Intelligence in Azure Streaming
Integration of JavaScript user-defined functions using Azure Stream Analytics
Summary
10
Understanding Stream Analytics Job Monitoring
Understanding Stream Analytics Job Monitoring
Troubleshooting with job metrics
Visual monitoring of job diagram
Logging of diagnostics logs
Configuring job alerts
Viewing resource health information with Azure resource health
Exploring different monitoring experiences
Summary
11
Use Cases for Real-World Data Streaming Architectures
Use Cases for Real-World Data Streaming Architectures
Solution architecture design and Proof-of-Concept implementation of social media sentiment analytics using Twitter and a sentiment analytics dashboard
Summary

Sample use case 

In this sample, we have a three automated tollbooth and cars pass through every few minutes. The goal is to calculate how many cars have passed through in a given duration.

  • No Partition:
SELECT COUNT(*) AS Count, TollBoothId
FROM Input1 GROUP BY TumblingWindow(minute, 3), TollBoothId, PartitionId
  • Partitioned Query:
SELECT COUNT(*) AS Count, TollBoothId
FROM Input1 Partition By PartitionId
GROUP BY TumblingWindow(minute, 3), TollBoothId, PartitionId
  • Split query with Partition and Non-Partition:
WITH Step1 AS (
 SELECT COUNT(*) AS Count, TollBoothId, PartitionId
 FROM Input1 Partition By PartitionId
 GROUP BY TumblingWindow(minute, 3), TollBoothId, PartitionId
 )
 SELECT SUM(Count) AS Count, TollBoothId
 FROM Step1
 GROUP BY TumblingWindow(minute, 3), TollBoothId

Summary of the result with the total Streaming Unit:

Previous Section
End of Section 4
Next Section