Before we delve deeper into modeling the data layer for the link graph and text indexer components, we need to spend some time discussing the reasoning behind the introduction of a data layer abstraction.
First and foremost, the primary purpose of the data layer is to decouple our code from the underlying data store implementation. By programming against a well-defined and data store-agnostic interface, we ensure that our code remains clean, modular, and totally oblivious to the nuances of accessing each data store.
An extra benefit of this approach is that it offers us the flexibility to A/B test different data store technologies before we decide which one to use for our production systems. What's more, even if our original decision proves to be less than stellar in the long term (for example, service traffic exceeds the...