Book Image

Designing Production-Grade and Large-Scale IoT Solutions

By : Mohamed Abdelaziz
Book Image

Designing Production-Grade and Large-Scale IoT Solutions

By: Mohamed Abdelaziz

Overview of this book

With the rising demand for and recent enhancements in IoT, a developer with sound knowledge of IoT is the need of the hour. This book will help you design, build, and operate large-scale E2E IoT solutions to transform your business and products, increase revenue, and reduce operational costs. Starting with an overview of how IoT technologies can help you solve your business problems, this book will be a useful guide to helping you implement end-to-end IoT solution architecture. You'll learn to select IoT devices; real-time operating systems; IoT Edge covering Edge location, software, and hardware; and the best IoT connectivity for your IoT solution. As you progress, you'll work with IoT device management, IoT data analytics, IoT platforms, and put these components to work as part of your IoT solution. You'll also be able to build IoT backend cloud from scratch by leveraging the modern app architecture paradigms and cloud-native technologies such as containers and microservices. Finally, you'll discover best practices for different operational excellence pillars, including high availability, resiliency, reliability, security, cost optimization, and high performance, which should be applied for large-scale production-grade IoT solutions. By the end of this IoT book, you'll be confident in designing, building, and operating IoT solutions.
Table of Contents (15 chapters)
1
Section 1: Anatomy of IoT
5
Section 2: The IoT Backend (aka the IoT Cloud)
10
Section 3: IoT Application Architecture Paradigms and IoT Operational Excellence

A case study

Despite this book being generically about designing and building an IoT solution, we will look at a case study to be used in the book's chapters where applicable.

A smart city initiative is not one IoT project; it is, in fact, so many different IoT projects and it usually has the following cases or modules to be covered:

  • Waste collection management
  • Street lighting management
  • Digital building management
  • Energy management
  • Green space management
  • Forest fire detection
  • Digital citizenship services
  • Smart parking

In this book, we've chosen the smart parking module of the smart city initiative to be used as a reference or case study:

Figure 1.3 – Parking lots (source: https://wordpress.org/openverse/image/d621a8cf-e786-4877-adea- bc5eb0ce7f87)

Figure 1.3 – Parking lots (source: https://wordpress.org/openverse/image/d621a8cf-e786-4877-adea- bc5eb0ce7f87)

In the upcoming sections, we will go through the requirements or the business problems at hand and the design thought process of an IoT solution to address those business requirements and deliver the desired business benefits.

Parking space issues

Usually, in large, bustling cities around the world, finding a parking space (aka slot) is challenging and not an easy task at all. People sometimes avoid outings on account of parking issues. Governments and city council administrations usually look for solutions to address parking availability challenges. There are so many factors behind parking space issues, such as ineffective land planning, insufficient parking spaces, traffic congestion, toxic emissions, and others.

Putting ineffective land planning issues aside, as we can't help with that issue, the congested traffic and toxic emissions issues occur essentially because car drivers lack a real-time parking lot availability solution to enable them to efficiently and quickly find available parking lots in the city.

The solution to the parking issue

City administrators started looking into smart parking solution initiatives to gain the following business benefits and solve parking space issues:

  • Reduce toxic emissions and help save the environment.
  • Reduce traffic congestion.
  • Reduce costs with less fuel consumption.
  • Reduce the amount of time drivers spend searching for an available parking lot.
  • Increase revenue – with smart parking, parking lots will be occupied efficiently by different drivers throughout the day, and a new billing model will be introduced, such as paying for exactly how long you stay.
  • With smart parking analytics and ML technologies, city council administrators can undertake proper land planning to increase the number of parking spaces across the city.

Business requirement analysis

As stated earlier, understanding business problems and pain points in full is an important step toward solving such business problems. Sometimes the solution might be a non-technical solution and might not depend on technology at all. The solution might be just enhancing existing business processes or people skills.

In that phase of the project, that is, business requirement clarification, you, as a business or system analyst or solution architect, should put any questions you have to the business stakeholders as answering such questions will have an impact on the solution selected at the end. For example, Is the parking space indoors or outdoors? You might ask, will this matter? And the answer is yes, it will matter, as, based on the answer, you will define which devices, sensors, and connectivity options could be used. For example, if it is indoor parking, then you could have a source of electrical power available in the parking lot or nearby so you might not worry much about low-power, very constrained IoT devices and so on. As regards the sensors, you might not worry much about the external environmental and non-environmental impact on the functionality of the sensors, such as dust and lighting levels. For connectivity, you might choose Wi-Fi and fixed broadband instead of cellular (mobile) connectivity and so on.

The business and system requirement identification process is not an easy task; it would need a book to explain that topic at length. However, here are the general steps used in that process:

  • What are the smart parking solution use cases? What are the business drivers for a smart parking solution or what are the goals of the business related to the smart parking solution?
  • Who are the stakeholders of the smart parking solution?
  • What are the business's smart parking and technical KPIs?
  • What are the smart parking solution's Non-Functional Requirements (NFRs), such as security, availability, scalability, disaster recovery, and resiliency?

Let's dive a bit deeper here and discuss the role of various stakeholders in a smart parking solution.

City council administrators

These individuals will need to manage and monitor the smart parking solution, meaning they might require the following:

  • A dashboard showing the occupancy level of the different parking spaces in the city
  • An alert management solution to alert and notify council administrators of any issues that occur
  • Analytics and reporting for smart parking solutions across the city
  • A wholesale billing solution to help the council manage and bill different smart parking operators
  • Data lakes and ML capabilities to enable council or government data scientists to run different ML predictive models to help with land planning and other logistics

Smart parking operators

Usually, councils do not run smart parking solutions themselves; rather, they make data such as parking space details and other data they can expose through a rich set of APIs available to third-party companies and partners to operate and manage the smart parking solutions.

The operators usually cover the following tasks:

  • Setting up and converting parking spaces into smart parking spaces
  • Offering vehicle owners different digital engagement channels, such as on web, mobile, or virtual assistant devices such as Alexa to do the following:

a. Search for available parking lots nearby

b. Book, reserve, and pay for an available parking lot

c. Parking lot management, that is, updates, changes, cancelations, and so on

  • Monitoring and operating the whole smart parking solution on a production-grade scale
  • Exposing or selling smart parking APIs to other third parties and start-ups for the co-creation of different ecosystems for smart parking
  • Integrating with internal and external capabilities such as payment gateways, OpenCV (OpenCV is an open source library for computer vision or image processing), and many other features as part of the final smart parking solution

Vehicle drivers

These are the end users and the target users at the end of the new business model or smart parking solution. Drivers will be provided with the features mentioned above by smart parking operators.

Smart parking solution details

Now let's think about how the smart parking operator will address the requirements we just discussed and come up with a solution.

IoT solution field planning

In networking, network engineers usually do a survey where network nodes such as switches and routers will be placed on the site, which kind of cables will be used, and which kind of network topology will be used (that is, star, mesh, ring).

Cables going through specific routes and even walls is not a haphazard task; there are engineering practices and explanations behind all of that setup.

IoT solutions are no different from traditional networking solutions. In fact, the IoT is sometimes classified as a networking engineering topic. It is called the internet of things, so connectivity and networking are a core part of it.

With any IoT solution, there is a physical world that it needs to sense and then act upon that sensing, which means in an IoT solution, there is always a field or playground where you need to deploy and install IoT devices and sensors.

Step number 1 in any IoT solution is to study very well the field or the ground where the IoT devices will be deployed. In our case, the parking space that we need to study is a field.

We need to know the following about the city's parking spaces:

  • The number of parking spaces in the city and their location details
  • Parking space type(s):
    1. Is it a one-level parking lot or multi-level?
    2. Is it off-street (a kind of closed/private parking space where there is a gate (in and out) controlling the vehicles coming in and out of the parking lot), or is it on-street (an open parking area) where there is no gate to control the vehicles coming in and out of the parking space?
  • Parking lot type:
    1. Within the parking lot, how many parking spaces are there? What type of parking spaces are they? That is, are they perpendicular, angled, or double parking spaces?
    2. Any other metadata about the parking lot, for example, its latitude and longitude.

IoT solution devices and sensors

Given the above IoT solution field planning, the next steps will involve choosing the IoT devices and sensors needed to detect the occupancy status of the parking lot.

There are so many ways in which to detect the occupancy status of a parking lot. Here, we'll mention a few:

  • You could install one or more – based on the parking space size – surveillance cameras to cover the whole parking space. Then you could use the ready-made, advanced AI Video Analytics technology to detect whether the parking lot has spaces or is occupied.

Previously, you'd have had to build such services from scratch, using technologies such as image processing and similar, but now, with AI and ML's maturity, hyperscale companies such as Amazon, Microsoft, and Google provide so many ready-to-use cognitive services, such as object detection, image classification, or activity detection (video), that you don't need to build an ML model yourself; it is already done for you. You might only need to feed the model with your data, that is, parking lot images or videos to show when they are occupied and when they are empty. I said might, as detecting a parking lot's occupancy status is becoming a standard, popular use case in that domain, so you might even find a third-party company that offers APIs ready to detect that. You would just need to pass to those APIs images of the parking lot you want to check the occupancy status of and you would get the answer: Occupied or Empty.

  • You could install in each parking space an IoT device buried underground. That device would be equipped with an infrared sensor to detect vehicle motion and report accordingly whether a space is occupied. This option might be a bit expensive as you would have to install an endpoint IoT device with sensors underneath each parking space. Yes, we could have low-value ($5 or less), low-power (battery-based, lasting 10 years or more) IoT endpoint devices to serve that purpose, but the cost will be linear with the number of parking lots in all parking spaces across the city as a whole.
  • You could install – in the case of gate-controlled parking spaces – a sensor on the in and out gate controllers to count the number of incoming vehicles minus the number of outgoing vehicles to detect how many parking spaces are available and how many are occupied. Detecting how many parking spaces are available might be enough in some cases, but in other cases, there might be a need to show the vehicle owner or driver where exactly the available parking space they booked or reserved is. For the latter use case, you need to use a different approach, as explained earlier.

Smart parking in offline mode (or IoT Edge)

You might, as part of smart parking business requirements, need to run some smart parking workloads at the edge, like the example we mentioned before about calculating the number of incoming vehicles versus the outgoing vehicles, or some real-time or near-real-time analytic requirements, and so on.

You must think carefully about such requirements. You need to think about the locations of where you will put the edge computing nodes. Edge units could be as simple as a small Raspberry Pi device or as medium complexity as a small-scale data center, or even as complex as a complete data center hosted at the edge.

In the case of off-street or private parking spaces, edge units could be placed in the parking spaces or nearby, while in the case of on-street parking, it would be a bit challenging to know where you should put the edge units. There are many options for that case, such as renting a nearby facility such as a building or even a room(s) in a shared building. You need, though, to think about the connectivity between the IoT endpoint devices on the field and such edge units, or you could buy managed edge services or hosting facilities from telecommunication companies or public cloud providers who have massive infrastructure footprints to host your units.

We have mentioned telecommunications companies as they have really massive footprint infrastructure coverage for a district or even – with 5G technology – street levels. That is because of their need to host telecommunication radio equipment. Public cloud providers also have edge locations but, for historical reasons, they don't have the level of coverage of telecommunication companies.

IoT solution connectivity

Given the preceding IoT solution field planning and based on the parking space type and details, you can, as a solution architect, define what kind of connectivity is required in the solution.

First, you need to ask yourself, I need to connect what to what? You basically need to connect the endpoint IoT devices to the internet or to the backend servers of the smart parking operators or council. Those backend servers are hosted in what is called the IoT Cloud or IoT backend cloud.

There are two options available:

  • Connect endpoint IoT devices directly to the IoT backend cloud.
  • Connect endpoint IoT devices to the edge gateway first and then the edge gateway will connect on behalf of those IoT endpoint devices to the IoT backend cloud.

With the first option, that is, directly connecting endpoint IoT devices to the IoT Cloud, you will need the IoT endpoint device to do the following:

  • Have a communication module to support WAN connectivity, that is, like the LTE (LTE-M) module in the case of using a telecommunication network (a SIM card-based or Ethernet module in the case of using fixed broadband)
  • Have an IP stack running in the device to support IP-based connectivity protocols

Those two conditions make such a connectivity option, that is, connecting to the IoT Cloud directly, not the preferred option due to the following:

  • IoT endpoint devices – in the case of underground buried parking lot sensors – will become a problem as the cost will be high, that is, you have to have in each endpoint IoT device a cellular or LTE-M module with a SIM card. Also, in order to run the IP stack, the devices can't be constrained to low-value IoT devices.
  • Those IoT endpoint devices will usually be on streets or far away from power supplies, hence they are usually battery-based devices. Running an IP stack on such endpoint IoT devices would impact the battery life of the devices, which usually work for 10 years or more to avoid the disturbance of changing such devices or their batteries.

The second connectivity option – that is, through the gateway – looks good as you could deploy low-value, low-power, and constrained endpoint IoT devices in the parking spaces. Those devices could communicate between each other and gateway devices using low-power local wireless networking such as Zigbee or Bluetooth Low Energy. Such connectivity technology will help in saving device batteries and it is cheap. The gateway will connect to the internet, meaning the gateway device could use a fixed broadband (wired option) if available or use cellular connectivity (wireless option). The gateway device or edge could or should be a powerful device in terms of computing resources.

We could use the following communication protocols:

  • IoT endpoint devices could use the Zigbee protocol to communicate with other Zigbee-enabled devices, including the IoT gateway device.
  • IoT edge devices or gateway devices – IP-enabled devices communicate with the IoT backend cloud using MQTT or HTTP(S) over the internet.

Smart parking IoT backend cloud

From the requirements, we will need to build (or buy) the following backend components for the smart parking solution:

  • An IoT message broker/ IoT device gateway supporting MQTT, HTTP(S)
  • A smart parking operational center – a web app for council and smart parking operators' admin users for managing and monitoring the smart parking solution
  • Smart parking web and mobile apps – for the vehicle owners or drivers to book, manage, cancel, and pay for a parking space
  • A smart parking API hub – an API hub for external third parties and partners looking to use the smart parking solution's data and its available set of APIs in their solutions
  • A smart parking data lake – a data lake solution storing all smart parking data from all sources in its original raw format for further analysis by other downstream systems and solutions
  • A smart parking analytic solution – an analytical and machine learning solution based on the smart parking data lake solution

Smart parking IoT solution – non-functional requirements

The features or functionality of the smart parking solution might be negotiated with business stakeholders. Usually, the most common statements you will hear at that stage are we can't boil the whole ocean, which feature is a must and which feature is nice to have?, and so on. There are lots of features requested by business stakeholders, but not all such features could be delivered in one release of the IoT project for a myriad of reasons, such as a lack of resources, the pressure of the delivery timeline, or even some features just being fancy features that the solution can work without.

But, when it comes to non-functional requirements, usually there's no negation with the assumption that the business stakeholders agreed with the increasing costs associated with achieving those non-functional requirements. Every business wants its solutions to be highly available (up and running 24x7), highly performant, highly scalable, highly reliable, and highly secure.

As a solution architect, you should consider non-functional requirements from the design stage and provide all the design principles, technical components, and controls required to fulfill such requirements. Also, those design principles should be applied from the device to the cloud, in each layer of the IoT solution.