So far, we have mostly discussed, at various levels of granularity, the architectural patterns that are important to keep in mind when designing a data-intensive system. As you will realize, knowledge of these Architectural Patterns will come in handy when we start to define the architecture of various functional components of a data-intensive system. This chapter starts diving deep in the first functional component that sits at the edge of any Data-Intensive System.
In this chapter, we will cover the following:
- Understanding, with examples of use cases, how to approach pinning down the requirements for your data-collection system
- Functional characteristics of a data collection System, such as keeping track of data lineage and preserving the quality of the data
- Various types of data sources (this was also discussed in Chapter 4, Discussing Data-Centric Architectures)
- Various requirements (both functional and...