Depth of data includes the granularity of data. This factor mainly depends on how deep we are diving into analyzing the data. For example, when a global head queries the data, it will hit on the entire set of data. Let us say a regional manager of a particular country is querying the data; in this case, only a set of data will be hit. The analysis made is the same, but the amount of data being queried differs. Once the query is obtained, there will be situations where data has to be sliced and diced. Let us consider that the query run by the global/regional head is on a single year's data. Now, we can also start drilling down the report up to a single day. This involves huge amounts of data. The queries run may be simple or more interactive with the user.

The data we are dealing with need not necessarily be of the same data types. So, in a very simple way, we can categorize this as big data. The data may comprise text, pictures, video, and so on. When a query hits on such a huge amount and variety of data, it becomes difficult to combine the bits set to give only a small set of data.

The analysis should be running in real time rather than post-mortem. Decisions taken based on real-time data will be more productive compared to the post-mortem analysis. To achieve this, data has to be continuously replicated into SAP HANA in real time so that queries are run over it.

The data has to be prepared before presenting it to the user. There will be scenarios where we may need to build pre-aggregates and also tune the data. The simpler the data is, the faster it can be accessed. When we go on increasing the complexity, it takes more time to retrieve the results.

The data should be retrieved very quickly. When we are interacting with the data, results should be displayed on-the-fly without any latency. When we aim at a very low retrieval time for the data, all the preceding factors should be at a minimum.

With present-day technologies available, we can have optimal results in terms of a single factor only; that is, we can only go in-depth on large varieties of data. In doing this, we cannot expect results to be fast, and the same results cannot be run on real-time data or a complex set of data. To get the results very quickly, the data set has to be very simple.

To overcome all these problems and obtain the best results, we need an innovative, groundbreaking technology.

In the initial development stages of technology, the core area in processors started decreasing, increasing the cache. This was the trend during the period between Pentium-I and Pentium-IV, but not after that. Instead, the number of cores in a single processor was increased and multicore processors evolved. We can compare a core of a processor to the brain of a human being. We use our brain to do all the activities in our day-to-day lives. The same brain takes care of all the activities. When we are doing one activity, we cannot do another with the same level of concentration. Similarly, when a processor is busy performing one operation, it cannot take up another. Imagine how it would be if we were to have multiple brains. We could allocate each brain for separate activities. This is exactly what happens in a multicore processor.

Each core can take up a separate operation, thereby increasing the capabilities of a computer. This is a revolutionary change in the field of processors. Moore's law states that the number of transistors on integrated circuits doubles approximately every two years. Based on this law, it has been predicted that if the current trend continues to 2020, the number of transistors will reach 32 billion.

Source: http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2011.svg

Since the beginning of the year 2000, there has been a drastic change in the processing power and speed of processors. Modern systems that are in use today have an architecture that can support up to eight separate CPUs, each being able to contain 12 separate cores. This change in processing power should be used to get the best throughput for both analytical and transactional applications. It is very important for enterprise applications that their work be reduced and they be developed in such a way that parallelization is always achieved. Parallelization has to be achieved at different levels, including the application being run on an application server until the query execution in the database system. Multithreading helps process multiple queries at a time so that the applications don't halt when there are more queries to be processed. Each CPU core will take care of a single process. Hence, when we have more core CPUs, the system response time is optimal as multicore CPUs process any number of queries effectively.

Multicore CPUs also help achieve higher CPU execution speeds. With multicore CPUs, calling parallel algorithms in databases is possible; therefore, all the available computing resources can be utilized. As SAP HANA supports column-based storage, it is easy to execute operations in parallel using multiple processor cores. This is possible as data will be partitioned column-wise. Hence, queries can be run in parallel on different columns. When queries hit multiple columns, different processor cores work on different columns so that final results are aggregated. In the case where a query hits only a single column, it is split into several sections, each of which will be handled by a different processor core.

We store all data in the main memory. Hence, it is of high priority that data be compressed and stored for the efficient utilization of memory; if not, it leads to very high costs. With columnar storage, high compression rates are possible; typically, a compression rate of 10 times the original can be achieved compared to that in row storage systems, because when we sort the values in a column, several contiguous values will be placed adjacent to each other. In this scenario, compression techniques such as cluster coding, run-length encoding, and dictionary coding are used.