Choosing a good data split between training and evaluation
When you're choosing a data split between your training and evaluation periods, you need to consider the following constraints or recommendations:
- The first constraint to consider is the requirement to have at least 90 days in the training range. At the time of writing, Amazon Lookout for Equipment considers that it needs at least this period to model the normal operating behavior of a piece of industrial equipment. This physical behavior is independent of the granularity at which the sensor data is collected.
- Ideally, the training range should include all the normal operating behaviors of your process or equipment. If a new behavior is only seen during the evaluation range, then there will be a high chance that Amazon Lookout for Equipment will flag it as an anomaly.
Important Note
Make sure that you don't have severe level shifts in some of your sensors (for instance, sensors that stopped working over...