Book Image

Hands-on DevOps

By : Sricharan Vadapalli
Book Image

Hands-on DevOps

By: Sricharan Vadapalli

Overview of this book

<p>DevOps strategies have really become an important factor for big data environments.</p> <p>This book initially provides an introduction to big data, DevOps, and Cloud computing along with the need for DevOps strategies in big data environments. We move on to explore the adoption of DevOps frameworks and business scenarios. We then build a big data cluster, deploy it on the cloud, and explore DevOps activities such as CI/CD and containerization. Next, we cover big data concepts such as ETL for data sources, Hadoop clusters, and their applications. Towards the end of the book, we explore ERP applications useful for migrating to DevOps frameworks and examine a few case studies for migrating big data and prediction models.</p> <p>By the end of this book, you will have mastered implementing DevOps tools and strategies for your big data clusters.</p>
Table of Contents (22 chapters)
Title Page
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
11
DevOps Adoption by ERP Systems
12
DevOps Periodic Table
13
Business Intelligence Trends
14
Testing Types and Levels
15
Java Platform SE 8

In-memory technology


In traditional application development, the disk was the main persistence for data storage. The challenge in this method was that, for business logic and application computation, data was transferred from storage disk to main memory, causing huge I/O overhead. Again, after the computations based on the business logic, the data from aggregation, computational, or analytic results was transferred from CPU and main memory to store, or the data was persisted back to storage disk, causing I/O overhead multiple times.

As the following simple illustration shows, disk speed is growing slower compared to other hardware components, while the need for higher performance and speed is increasing day by day:

In-memory database (IMDB)

With In-memory technology, in contrast to traditional disk-based data persistence methods, the complete application requires the data is loaded into the main memory of the system. That makes the applications perform 10 to 20 times faster.

  • Data resides permanently...