In this section, we will try to explain some of the most important BI concepts that you need to be familiar with. Before we start creating Universes, we need to make sure that we are talking the same language. The concepts, terms, and language used here are generic BI terminologies, and they are not related to specific BI reporting tools.
We will start with a knowledge pyramid that describes how data evolves from information to knowledge and finally, to wisdom. This is important because BI focuses on achieving knowledge and wisdom.
We will then talk about the difference between hindsight, insight, and foresight. After that, we will go through a fast overview of a data warehouse (DWH) and how it is related to BI.
The knowledge pyramid is also known as the data, information, knowledge, and wisdom model. Here is an example that describes each stage of the knowledge pyramid:
Data: This consists of scattered discrete facts that you can't understand alone, because they are not in a context. The following facts are an example of data:
[is 200 Temperature C].These facts cannot be understood in the current order and format. This is because they are discrete, scattered, and without a context. This means that data alone is not useful and somehow needs to evolve to the other levels in the knowledge pyramid in order to gain some extra value.
Information: This consists of some discrete facts (data) evolved by putting them in a context. Context is a specific order of facts that will help us understand them and gain information. Let's check out the following example:
[Temperature is 200 C].Now, we start having a context after reordering the discrete facts presented in the previous data example. We know that
200is a number representing the temperature of something and that it is measured in Celsius.Knowledge: This can be achieved by adding more context to the information. Let's check out the following example:
[Car engine temperature is 200 C].[Car engine normal temperature is between 100 and 150].Now, you have more information grouped together in a context, and you know that your car engine's temperature is above the normal temperature. You might take an action, but you still need some more information to be able to take the right decision at the right time.
Wisdom: We will reach wisdom when we increase the context level by adding more knowledge and information together in the right order that can help us gain information and take actions. Let's check out the following example:
[Car engine temperature is 200 C].[Car engine normal temperature is between 100 and 150].[Car engine temperature red zone starts from 200 C].[You need to stop your car if engine temperature reaches the red zone].Now, you can take a precise action based on the data, information, knowledge, and wisdom you have, and you will stop your car and go to check your engine. This is because you realized that your car temperature is higher than normal, and it is in the red (dangerous) zone.
The different stages of the knowledge pyramid are shown in the following diagram:
Now, after you have learned the knowledge pyramid, we need to find out what the relation between the pyramid and BI is. BI will evolve as data evolves. BI starts with raw data that will evolve into information after presenting it in a format that is suitable for analysis. Information will evolve to knowledge after doing the proper analysis on the information. Historical information and current knowledge will evolve and lead to future wisdom. It will help us take the right action in the current situation and make the right decisions in the future.
Note
More information on the knowledge pyramid is available at http://en.wikipedia.org/wiki/DIKW_Pyramid.
You will hear these three words, hindsight, insight, and foresight, many times if you work in BI field. They are strongly connected with BI because they simply describe what BI is. We've already explored these concepts in the What is Business Intelligence? section; now, we're going to discuss them in more detail:
Hindsight: This refers to focusing on the past and history. We learn from our past to avoid making the same mistakes and to explore new opportunities that we didn't catch.
Insight: This refers to the balance and start point for both hindsight and foresight. The action that we will take now will be history in a few moments and will shape our future. We can have a better present by learning from our history, and this will lead to a better future.
Foresight: This refers to what we expect in future, that is, how we will predict what will happen based on what has already happened.
BI is a mix of hindsight, insight, and foresight. As they are somehow related and connected, the main target of BI is to learn from our hindsight to take the right decision in our insight to have a better foresight.
Note
For more information, you can refer to http://www.learnthelessons.com/Ponderables/sights.htm.
The data warehouse (DWH) is a central big repository to hold extracted data from multiple source systems across the organization. This is an important thing to think about before starting any BI initiative in your organization. The DWH will act as a single source for your BI reporting, and you will be able to integrate your isolated source systems and make your information available to top management and decision makers.
In the following diagram, you can see the data flow, which starts from source systems and ends at knowledge and wisdom, delivered to BI users in many formats:
DWH comes with many other concepts, which are given as follows:
Data Quality (DQ): This focuses on enhancing the quality of the data extracted from source systems to get more accurate information and build more valuable knowledge. Also, it takes care of enhancing source systems' user interfaces by doing the required data validation to make sure that the proper data is being entered and stored.
Master Data Management (MDM): This will focus on unifying the data to get the most accurate records. For example, let's take a customer's information. You might have a customer's mobile number and address stored in more than one system, but you know that a specific system contains the most accurate phone number of the customer, because it is used to perform transactions through calls. So, you will consider this system to get the most accurate phone number for your clients and other customer information such as address and name. This will help us get the most accurate and unified record for the customer from across our organization's source systems and also get what we call the customer golden record.
Metadata: Imagine that you have many source systems in your organization that you need to consider for data extraction. In some organizations, DWH contains thousands of tables and hundreds of thousands of columns and billions of records. For example, banking and telecommunication industries. For such huge DWH, you need to track what kind of information you have, where this information is stored, and how to access it. Metadata is data about data, and it will help you answer all questions raised earlier.
Data Governance: This is your DWH police. It will govern DWH by controlling the data flow between DWH and source systems. It will help unify business rules and criteria across the organization. Finally, it will control the process as well. Data Governance is the big umbrella that holds everything that we talked about in this section.
Besides data governance, there are many other types of governance that can run in your organization, such as IT governance, enterprise governance, and BI governance. In the following diagram, you can see just an example of multiple levels of governance:
DWH will act like the single point of truth, as everyone is accessing the same information that is stored in the same location with the same business logic and rules applied.
As you can see, there is a strong relation between BI and DWH as both of them complement each other. BI needs DWH to achieve its goals, and DWH needs BI to avail its data and make it utilized.
Note
For more information on data warehouse and BI, you can visit http://en.wikipedia.org/wiki/Data_warehouse and http://en.wikipedia.org/wiki/Business_intelligence.