Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Pig Design Patterns
  • Table Of Contents Toc
Pig Design Patterns

Pig Design Patterns

By : Pradeep Pasupuleti
4.3 (6)
Buy this Book
close
close
Pig Design Patterns

Pig Design Patterns

4.3 (6)
By: Pradeep Pasupuleti
Buy this Book

Overview of this book

Table of Contents (16 chapters)
close
close
close
Pig Design Patterns
Icon
Pig Design Patterns
Credits
Icon
Credits
Foreword
Icon
Foreword
About the Author
Icon
About the Author
Acknowledgments
Icon
Acknowledgments
About the Reviewers
Icon
About the Reviewers
www.PacktPub.com
Icon
www.PacktPub.com
Preface
Icon
Preface
Lock Free Chapter
1
Setting the Context for Design Patterns in Pig
Icon
Setting the Context for Design Patterns in Pig
Icon
Understanding design patterns
Icon
The scope of design patterns in Pig
Icon
Hadoop demystified – a quick reckoner
Icon
Pig – a quick intro
Icon
Understanding Pig through the code
Icon
Summary
2
Data Ingest and Egress Patterns
Icon
Data Ingest and Egress Patterns
Icon
The context of data ingest and egress
Icon
Types of data in the enterprise
Icon
Ingest and egress patterns for multistructured data
Icon
The ingress and egress patterns for the NoSQL data
Icon
The ingress and egress patterns for structured data
Icon
The ingress and egress patterns for semi-structured data
Icon
JSON ingress and egress patterns
Icon
Summary
3
Data Profiling Patterns
Icon
Data Profiling Patterns
Icon
Data profiling for Big Data
Icon
Rationale for using Pig in data profiling
Icon
The data type inference pattern
Icon
The basic statistical profiling pattern
Icon
The pattern-matching pattern
Icon
The string profiling pattern
Icon
The unstructured text profiling pattern
Icon
Summary
4
Data Validation and Cleansing Patterns
Icon
Data Validation and Cleansing Patterns
Icon
Data validation and cleansing for Big Data
Icon
Choosing Pig for validation and cleansing
Icon
The constraint validation and cleansing design pattern
Icon
The regex validation and cleansing design pattern
Icon
The corrupt data validation and cleansing design pattern
Icon
The unstructured text data validation and cleansing design pattern
Icon
Summary
5
Data Transformation Patterns
Icon
Data Transformation Patterns
Icon
Data transformation processes
Icon
The structured-to-hierarchical transformation pattern
Icon
The data normalization pattern
Icon
The data integration pattern
Icon
The aggregation pattern
Icon
The data generalization pattern
Icon
Summary
6
Understanding Data Reduction Patterns
Icon
Understanding Data Reduction Patterns
Icon
Data reduction – a quick introduction
Icon
Data reduction considerations for Big Data
Icon
Dimensionality reduction – the Principal Component Analysis design pattern
Icon
Numerosity reduction – the histogram design pattern
Icon
Numerosity reduction – sampling design pattern
Icon
Numerosity reduction – clustering design pattern
Icon
Summary
7
Advanced Patterns and Future Work
chevron up
Icon
Advanced Patterns and Future Work
Icon
The clustering pattern
Icon
The topic discovery pattern
Icon
The natural language processing pattern
Icon
The classification pattern
Icon
Future trends
Icon
Summary
1
Index
Icon
Index

Chapter 7. Advanced Patterns and Future Work

In the previous chapter, you have studied various Big Data reduction techniques that aim to reduce the amount of data being analyzed or processed. We have explored design patterns that perform dimensionality reduction using the Principal Component Analysis technique and numerosity reduction using clustering, sampling, and histogram techniques.

In this chapter, we will start by discussing design patterns that primarily deal with text data and will explore a wide array of analytics pipelines that can be built using Pig as the key ingestion and processing engine.

We will be delving into the following patterns:

  • Clustering textual data

  • Topic discovery

  • Natural language processing

  • Classification

We will also speculate about what the future holds for Pig design patterns. These future trends analyze the kind of trends that are being followed now in the mainstream to modify Pig for specific use cases. These include where these trends will originate, what trends...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Pig Design Patterns
End of Section 7
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon