Logging inference calls
Logging is an essential part of any software architecture. We use logs to recall and investigate what happened to the system in the past. Unlike monitoring, logs are more focused on the events that occurred in the system in the past with the objective of providing the capability to look back on or perform an audit of past events.
Logging in MLOps is no different. However, there are a few aspects of logging that are more common in ML model inference than in traditional software. Here are some of the properties that we need to look out for in ML model inference logging:
- Unstructured data: In some cases, the data you input into the inference call may not always be simple JSON-formatted text; it could be an image, video, or audio as well. This kind of unstructured data may require a different kind of storage system for logs.
- Non-deterministic behavior: Some models, depending on the algorithm used, may not always return the same output for the same...