Modern enterprise architectures consist of a huge number of solutions, each comprising many nodes. Some MapReduce clusters contain hundreds of nodes. Each node contains an array of applications and services, both at the operating system and Application layers. These services and applications generate varying volumes of log data. There is an increasing recognition of the importance of log data within the enterprise community for the following reasons:
It is a key source of information for any IT operations team to maintain available systems
It is key to finding and resolving problems, both in the production and systems testing phases
It is increasingly becoming a source of business value, where valuable business data is trapped in this semi-structured data, including:
Risk and compliance data
Business operational data
Web analytics
Security
Financial forecasts
In order to leverage valuable log data, it must be sourced from these nodes and delivered securely and easily to a centralized...