Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Pig Design Patterns
  • Table Of Contents Toc
Pig Design Patterns

Pig Design Patterns

By : Pradeep Pasupuleti
4.3 (6)
Buy this Book
close
close
Pig Design Patterns

Pig Design Patterns

4.3 (6)
By: Pradeep Pasupuleti
Buy this Book

Overview of this book

Table of Contents (16 chapters)
close
close
close
Pig Design Patterns
Icon
Pig Design Patterns
Credits
Icon
Credits
Foreword
Icon
Foreword
About the Author
Icon
About the Author
Acknowledgments
Icon
Acknowledgments
About the Reviewers
Icon
About the Reviewers
www.PacktPub.com
Icon
www.PacktPub.com
Preface
Icon
Preface
Lock Free Chapter
1
Setting the Context for Design Patterns in Pig
Icon
Setting the Context for Design Patterns in Pig
Icon
Understanding design patterns
Icon
The scope of design patterns in Pig
Icon
Hadoop demystified – a quick reckoner
Icon
Pig – a quick intro
Icon
Understanding Pig through the code
Icon
Summary
2
Data Ingest and Egress Patterns
Icon
Data Ingest and Egress Patterns
Icon
The context of data ingest and egress
Icon
Types of data in the enterprise
Icon
Ingest and egress patterns for multistructured data
Icon
The ingress and egress patterns for the NoSQL data
Icon
The ingress and egress patterns for structured data
Icon
The ingress and egress patterns for semi-structured data
Icon
JSON ingress and egress patterns
Icon
Summary
3
Data Profiling Patterns
Icon
Data Profiling Patterns
Icon
Data profiling for Big Data
Icon
Rationale for using Pig in data profiling
Icon
The data type inference pattern
Icon
The basic statistical profiling pattern
Icon
The pattern-matching pattern
Icon
The string profiling pattern
Icon
The unstructured text profiling pattern
Icon
Summary
4
Data Validation and Cleansing Patterns
Icon
Data Validation and Cleansing Patterns
Icon
Data validation and cleansing for Big Data
Icon
Choosing Pig for validation and cleansing
Icon
The constraint validation and cleansing design pattern
Icon
The regex validation and cleansing design pattern
Icon
The corrupt data validation and cleansing design pattern
Icon
The unstructured text data validation and cleansing design pattern
Icon
Summary
5
Data Transformation Patterns
chevron up
Icon
Data Transformation Patterns
Icon
Data transformation processes
Icon
The structured-to-hierarchical transformation pattern
Icon
The data normalization pattern
Icon
The data integration pattern
Icon
The aggregation pattern
Icon
The data generalization pattern
Icon
Summary
6
Understanding Data Reduction Patterns
Icon
Understanding Data Reduction Patterns
Icon
Data reduction – a quick introduction
Icon
Data reduction considerations for Big Data
Icon
Dimensionality reduction – the Principal Component Analysis design pattern
Icon
Numerosity reduction – the histogram design pattern
Icon
Numerosity reduction – sampling design pattern
Icon
Numerosity reduction – clustering design pattern
Icon
Summary
7
Advanced Patterns and Future Work
Icon
Advanced Patterns and Future Work
Icon
The clustering pattern
Icon
The topic discovery pattern
Icon
The natural language processing pattern
Icon
The classification pattern
Icon
Future trends
Icon
Summary
1
Index
Icon
Index

The data normalization pattern

The data normalization design pattern discusses ways to perform normalization or standardization of data values.

Background

Normalization of data implies fitting, adjusting, or scaling the data values measured on different scales to a notionally common range. As a simplistic example, joining datasets that consist of different units for distance measurement, such as kilometers and miles, can provide varying results when they are not normalized. Hence, they are normalized to bring them back to one common unit, such as kilometers or miles, so that the effect of different units of measurement is not felt by the analytics.

Motivation

In Big Data, it is common to encounter varying values in the same data attribute while integrating multiple data sources.

Normalization performs data pre-processing and transformation by changing the original data and scaling it to bring it within a specified range (for example, in the range 0 to 1), and assigning equal weight for all attributes...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Pig Design Patterns
End of Section 5
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon