-
Book Overview & Buying
-
Table Of Contents
Hands-On Machine Learning with C++ - Second Edition
By :
There are many approaches regarding how to deploy an ML model to a mobile device with Android. We can use PyTorch, ExecuTorch, TensorFlow Lite, NCNN, ONNX Runtime, or others. We’ll use the PyTorch framework in this chapter since we have discussed it in the previous chapters, and because it allows us to use almost any PyTorch model with minimal functional restrictions. Unfortunately, we will be able to use only the target device CPU for inference. Other frameworks, such as ExecuTorch, TensorFlow Lite, NCNN, and ONNX Runtime, allow you to use other inference backends, such as onboard GPU or Neural Processing Unit (NPU). However, this option also comes with a notable restriction, which is the lack of certain operators or functions, which can limit the types of models that can be deployed on mobile devices. Dynamic shape support is usually limited, making it difficult to handle data with varying dimensions.
Another challenge is restricted...