Monitoring your models
You should also make sure to monitor any potential shift in your data. To do this, you can follow this process:
- Build and store a dataset with the training data of all the time series that you want to use to build an anomaly detection model.
- Compute the statistical characteristics of each time series (for example, average, standard deviation, and histograms of the distribution of values).
- Train your models with these initial datasets and save the performance metrics (how well they capture the anomalies you are interested in).
- When new data comes in, compute the same statistical characteristics and compare them with the original values used at training time.
- You can display these statistics next to the predictions for your analysts to take the appropriate decisions. This will help them better trust the results generated by Amazon Lookout for Equipment. In particular, visualizing a potential distribution shift from training to inference...