Tools to get the job done

This book focuses on tools offered by AWS to deliver ML-based solutions at the edge. Leading with the 2015 launch of the AWS IoT Core service, AWS has built out a suite of IoT services to help developers build cyber-physical solutions that benefit from the power of the cloud. These services range from edge software, such as the FreeRTOS real-time operating system for microcontrollers, to command and control (C2) of device fleets with IoT Core and IoT Device Management, and analytical capabilities for yielding actionable insights from data with services such as IoT SiteWise and IoT Events. The IoT services interplay nicely with Amazon's suite of ML services, enabling developers to ingest massive quantities of data for use in training ML models with services such as Amazon SageMaker. AWS also makes it easy to host trained models as endpoints for making inferences against real-time data or deploying these models to the edge for local inferencing.

There are three kinds of software tools you will need to create and operate a purposeful, intelligent workload at the edge. Next, we will define each tool by its general capabilities and also the specific implementation of the tool we are using, provided by AWS, to build the project in this book. There is always more complexity to any information technology (IT), but for our purposes, these are the three main kinds of tools this book will focus on in order to deliver intelligence to the edge.

Edge runtime

The first tool is a runtime for orchestrating your edge software. The runtime will execute your code and process local events to and from your code. Ideally, this runtime is self-healing, meaning that if any service fails, it should automatically recover by using failovers or restarting the service. These local events can be hardware interrupts that trigger some code to be run, timed events to read inputs from an analog sensor, or translating digital commands to change the state of a connected actuator such as a switch.

The AWS service that is the star of this book is IoT Greengrass. This is the service that we will use for the first kind of tool: the runtime for orchestrating edge software. IoT Greengrass defines both a packaged runtime for orchestrating edge software solutions and a web service for managing fleets of edge deployments running on devices such as gateways. In 2020, AWS released a new major version of IoT Greengrass, version 2, that rearchitected the edge software package as an open source Java project under the Apache 2.0 license. With this version, developers got a new software development model for authoring and deploying linked components that lets them focus on building business applications instead of worrying about the infrastructure of a complex edge-to-cloud solution. We will dive into more details of IoT Greengrass and start building our first application with it in the next chapter.

The following diagram illustrates how IoT Greengrass plays a role both at the edge and in the cloud:

Figure 1.6 – Illustration of how IoT Greengrass plays a role both at the edge and in the cloud

ML

The second tool is an ML library and model. The library dictates how to read and consume the model and how local code can invoke the model, also called making an inference. The model is the output of a training job that packages up the intelligence into a simpler framework for translating new inputs into inferences. We will need a tool to train a new model from a set of data called the training set. Then, that trained model will be packaged up and deployed to our edge runtime tool. The edge solution will need the corresponding library and code that knows how to process new data against the model to yield an inference.

The implementation of our second tool, the ML library and model, is delivered by Amazon SageMaker. SageMaker is Amazon's suite of services for the ML developer. Included are services for preparing data for use in training, building models with built-in or custom algorithms, tuning models, and managing models as application programming interface (API) endpoints or deploying them wherever they need to run. You can even train a model without any prior experience, and SageMaker will analyze your dataset, select an algorithm, build a set of tuned models, and tell you which one is the best fit against your data.

For some AI use cases, such as forecasting numerical series and interpreting human handwriting as text, Amazon offers purpose-built services that have already solved the heavy lifting of training ML models. Please note that the teaching of data science is beyond the scope of this book. We will use popular ML frameworks and algorithms common for delivering outcomes in IoT solutions. We will also provide justification for frameworks and algorithms when we use them to give you some insight into how we arrived at choosing them. We will introduce the concept of deploying ML resources to the edge in Chapter 4, Extending the Cloud to the Edge, and dive deeper into ML workloads in Chapter 7, Machine Learning Workloads at the Edge, and Chapter 9, Fleet Management at Scale.

Communicating with the edge

The third tool is the methodology for communicating with the edge solution. This includes deploying the software and updates to the edge hardware and the mechanism for exchanging bi-directional data. Since the kinds of edge solutions that this book covers are those that interoperate with the cloud, a means of transmitting data and commands between the edge and the cloud is needed. This could be any number of Open Systems Interconnection (OSI) model Layer 7 protocols, with common examples in IoT being HyperText Transfer Protocol (HTTP), Message Queuing Telemetry Transport (MQTT), and Constrained Application Protocol (CoAP).

The AWS IoT suite of services fits the needs here and acts as a bridge between the IoT Greengrass solution running at the edge and the ML capabilities we will use in Amazon SageMaker. IoT Core is a service that provides a scalable device gateway and message broker for both HTTP and MQTT protocols. It will handle the cloud connectivity, authentication and authorization, and routing of messages between the edge and the cloud. IoT Device Management is a service for operating fleets of devices at scale. It will help us define logical groupings of edge devices that will run the same software solutions and deploy updates to our fleet. Most chapters in this book will rely on tools such as these, and there is a focus on the scale of fleet management in Chapter 8, DevOps and MLOps for the Edge, and Chapter 9, Fleet Management at Scale.

With these tools in mind, we will next explore the markets, personas, and use cases where edge-to-cloud solutions with ML capabilities are driving the most demand.