Summary
In this chapter, we learned how to create a Scalable and Highly-Available Architecture for designing and implementing a Data Pipeline in our overall architecture. We delved deeper into the different considerations of designing the Data Pipeline, looking at various Design patterns that would help us in creating a Resilient-Data Pipeline.
The next chapter will build on the knowledge we have gained in this chapter to try to architect a Robust and Fault-Tolerant System capable of Collecting data from various sources efficiently and processing it in real-time so it can be ingested into the Central System.
Stay tuned.