For those unfamiliar with the term edge computing, it simply refers to computation performed at the end, or edge, of a network as opposed to sending it to a central server for computation. Some examples of edge devices include cars, robots, Internet of Things (IoT), and, of course, smartphones. 

The motivation for performing computation at the edge, where the data resides, is that sending data across the network is expensive and time-consuming; this incurred latency and cost restrict us with what experiences we can deliver to the user. Removing these barriers opens up new applications that would otherwise not be possible. Another benefit of performing inference at the edge is data privacy; removing the need of having to transmit personal data across the network reduces the opportunities that a malicious user has for obtaining it.

Luckily, technology advances at an astonishing rate and improvements in hardware and software have now made it feasible to perform inference...