Although analytics should be agnostic with regard to how the data is fed to the platform, we have to consider several potential pitfalls that can affect the efficiency of the analytics. There are several strategies that we can use to feed I-IoT data to the platform:
- Bulk ingestion, for example, one file daily
- Small portion, for example, one file every five minutes
- Data streams, where files are fed continuously with a small latency
Data is also affected by several issues:
- It might be in the wrong order. For example, a data point at 18:00 might be sent at 18:10 and a data point at 17:59 might be sent at 18:11.
- It might be of a bad quality.
- It might have holes in it.
- It might have anomalous spikes in it.
- It might be frozen. This refers to a situation where you have a suspiciously flat number for a long time.
These issues are illustrated...