Book Image

Hands-on DevOps

By : Sricharan Vadapalli
Book Image

Hands-on DevOps

By: Sricharan Vadapalli

Overview of this book

<p>DevOps strategies have really become an important factor for big data environments.</p> <p>This book initially provides an introduction to big data, DevOps, and Cloud computing along with the need for DevOps strategies in big data environments. We move on to explore the adoption of DevOps frameworks and business scenarios. We then build a big data cluster, deploy it on the cloud, and explore DevOps activities such as CI/CD and containerization. Next, we cover big data concepts such as ETL for data sources, Hadoop clusters, and their applications. Towards the end of the book, we explore ERP applications useful for migrating to DevOps frameworks and examine a few case studies for migrating big data and prediction models.</p> <p>By the end of this book, you will have mastered implementing DevOps tools and strategies for your big data clusters.</p>
Table of Contents (22 chapters)
Title Page
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
11
DevOps Adoption by ERP Systems
12
DevOps Periodic Table
13
Business Intelligence Trends
14
Testing Types and Levels
15
Java Platform SE 8

Building enterprise applications with Spark


For enterprise applications to be successful, it is very important that you carefully define the data access, processing, and governance framework.

Client-services presentation tier

This graphical user interface will be backed by a set of APIs to help on-board new users. Some features that can be supported are as follows:

  • Manage client data sources, file formats, delivery frequency, validation rules, join conditions (if multiple datasets are present), and so on.
  • Validate and transform datasets
  • Manage access to datasets
  • Additional data delivery requirements from Eureka

Data catalog services

This graphical user interface will be backed by a set of APIs to provide data-related services. Some of the features that can be supported are:

  • Search for any dataset/data in the data lake in a fashion similar to Google Search
  • Browse (preview with pagination) search results
  • Display the lineage and data profile of the selected data set

Workflow catalog

This graphical user...