Why is data architecture important?

A common mistake that rookie software developers make is to jump in and start writing code, thinking that the design can be modified later to fit requirements. Sometimes, they may refer to this as Agile thinking and approach. However, this is a fallacy as the data model, once defined, will be used as the foundation upon which other required business functionality is built; it is usually not straightforward to make changes to it quickly.

Let's take an example of an integration that pushes data from an external webinar system into a Salesforce Campaign Member object. Here, the webinar system takes the amount of time a person watched the live webinar and records it as a percentage. The developer has created a custom field of the number type called Webinar % Attended on the Campaign Member object. This field will only accept a number and therefore rejects any letters or special characters.

When the integration is run for the first time, it errors out because the webinar system is sending the percentage attended with the special character, %, and Salesforce therefore rejects it. This is a simple example to demonstrate how the data model needs to be well thought out and designed, not only considering the business requirements, but also understanding system functionality and limitations. In this case, the business doesn't need the values in the Webinar Completed field to make any calculations, so changing the field type to Text will work. In most cases, though, a change like this would require a thorough analysis of impacts on existing functionality and integrations and this can be a time-consuming effort.

Moreover, owing to the large number of systems that are now used in an organization, the sharing of data between these systems (that is, integration) has become crucial. Proper data architecture can avoid costly mistakes such as integrating two systems only to find out at a later stage in the project that there are technical limitations on how often the data can be moved between the two systems.

Important note

In architecture, decisions made earlier affect decisions made later. For example, should a Team Member-related list, intended to record team members working on the lead, be a lookup or a master-detail relationship? These are important design decisions to consider upfront as a change during build, or worse, when the application is in production, can be costly. For example, a decision to change from a multi-select picklist to a single picklist when the app is in production can lead to data loss. Consequently, our earlier decision affects decisions we will have to make surrounding the subsequent data loss.

Another consideration for data architecture is data security. Nefarious individuals or state-sponsored actors understand the value of data, and they look to exploit vulnerabilities for their own advantage. Data architecture ensures that the proper security processes and techniques are applied to the data when it's at rest or in transit. Modern data architecture also needs to conform to data protection regulations such as General Data Protection Regulation (GDPR) and California Consumer Protection Act (CCPA).

Organizations recognize the value of data and derive insights from it. Moving data from one system to another is not done in vain, rather, there are clear business objectives that are to be achieved. The most common of these objectives is decision making based on hard data. Proper data architecture will take into account how data from a multitude of systems can be brought together to meet the operational and analytical requirements of the business. An intense topic of interest these days, which we will discuss in the upcoming chapters, is how to achieve that single, unified view of the customer commonly referred to as Customer 360, and data architecture plays a key role in that.

The benefits of data architecture

Having discussed the importance of data architecture, let's look at some of the benefits it offers:

• It sets the stage for discussion and easy communication with different stakeholders: Creating a data architect diagram can facilitate discussions with solution architects and Business Analysts (BAs). It also facilitates getting approval from security teams that are concerned with how data is utilized and the boundaries it's going to cross; for example, a multi-national organization that has multiple legacy Customer Relationship Management (CRM) systems will want to address how data is going to move and be stored across continents in consideration of compliance with the General Data Protection Regulation (GDPR).
• It plays a vital role in driving non-functional requirements (NFRs): NFRs are related to the quality of the system being designed, such as performance, usability, and maintainability. Considering maintainability, for example, if the architecture is well defined, we will be able to enhance our application with relative ease compared to when the architecture has major flaws and in certain situations may require a complete redesign of the application, which is a costly and time-consuming endeavor.
• It enables change management: Change is inevitable. Now more than ever, internal and external forces are driving change at a rapid pace. Internally, enhancement requests, new business processes, and technological changes drive this change, whereas externally, raw market forces such as competition, geopolitical changes, and regulatory policies can drive this change. Having a well-defined data architecture that is properly documented and uses commonly understood language and standards enables stakeholders to respond to change quickly.
• It enables fast ramp-ups: Part of the change that we discussed in the last point also applies to changes in resourcing, meaning people in the company come and go. Having a good architecture established can be very helpful in training new personnel. Take the example of a new hire who joins the support team of an AppExchange partner that sells a popular recruitment app. Having the data model and reviewing it with the new hire can cut short the time needed to ramp up the resource to provide quality support to their customers.
• It is reusable: If an architecture was used once for an application and it was successful, it can be easily adapted by other areas in the enterprise or used for similar-natured projects in the future. Architecture work is intense and time-consuming and for large projects, it requires a lot of collaboration with multiple stakeholders. Reusing architecture eliminates or reduces the need to put in the same level of effort as the first time and this alone can translate into hard dollar savings for an enterprise.

There is also the quality aspect: when you are reusing your architecture the second or third time, you will be thinking of further improving it rather than rethinking the architecture from scratch. Over time, this cycle of continuous improvement can lead to an architecture that will start to be accepted in the company as a reference architecture. More and more, projects will start to adopt it, or the Enterprise Architect (EA) may mandate it as the starting point for designing other projects.

Having learned about the importance of data architecture and its benefits, we will next discuss in detail a data architect's responsibilities in the next section.