Book Image

Hands-on DevOps

By : Sricharan Vadapalli
Book Image

Hands-on DevOps

By: Sricharan Vadapalli

Overview of this book

<p>DevOps strategies have really become an important factor for big data environments.</p> <p>This book initially provides an introduction to big data, DevOps, and Cloud computing along with the need for DevOps strategies in big data environments. We move on to explore the adoption of DevOps frameworks and business scenarios. We then build a big data cluster, deploy it on the cloud, and explore DevOps activities such as CI/CD and containerization. Next, we cover big data concepts such as ETL for data sources, Hadoop clusters, and their applications. Towards the end of the book, we explore ERP applications useful for migrating to DevOps frameworks and examine a few case studies for migrating big data and prediction models.</p> <p>By the end of this book, you will have mastered implementing DevOps tools and strategies for your big data clusters.</p>
Table of Contents (22 chapters)
Title Page
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
11
DevOps Adoption by ERP Systems
12
DevOps Periodic Table
13
Business Intelligence Trends
14
Testing Types and Levels
15
Java Platform SE 8

Traditional enterprise architecture


Traditionally, an enterprise data warehouse (EDW) system is considered as a core component of the business intelligence environment. Data warehouse systems are central repositories built by integrating data from multiple disparate source systems, used for data analysis and reporting the needs of the enterprise.

Let's review the end-to end data life cycle components of the traditional system:

  • The data discovery phase is where the source systems are explored and analyzed for relevant data and data structures. If the analyzed data is valid, correct, and usable, it is ingested into the data warehouse system. For example, if we need customer ID information, we should be connecting and extracting data from the correct columns and tables.
  • Data quality ensures that the ingested data is acceptable and usable. A simple example is name formats of the first name and last name convention, which should be adhered to and, as appropriate, corrected for a few records.
  • Data...