Book Image

Practical Data Analysis - Second Edition

By : Hector Cuesta, Dr. Sampath Kumar
Book Image

Practical Data Analysis - Second Edition

By: Hector Cuesta, Dr. Sampath Kumar

Overview of this book

Beyond buzzwords like Big Data or Data Science, there are a great opportunities to innovate in many businesses using data analysis to get data-driven products. Data analysis involves asking many questions about data in order to discover insights and generate value for a product or a service. This book explains the basic data algorithms without the theoretical jargon, and you’ll get hands-on turning data into insights using machine learning techniques. We will perform data-driven innovation processing for several types of data such as text, Images, social network graphs, documents, and time series, showing you how to implement large data processing with MongoDB and Apache Spark.
Table of Contents (21 chapters)
Practical Data Analysis - Second Edition
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface

Filtering the input collection


Sometimes, we don't need the entire collection for our MapReduce process. Because of this, the mapReduce command provides us with optional parameters to filter the input collection.

The query parameter allows us to apply criteria using the query operators to filter the documents input into the map function. In the following code, we will filter the documents in the collection and only include the documents where the number attribute is greater than 10 ("$gt":10):

collection.map_reduce(map_function,                         
                         reduce_function,                      
                      "output_collection",                            
                            query={"number":{"$gt":10}})

The Query Operators used in the MapReduce query parameter are the same query selectors seen in Chapter 12, Data Processing and Aggregation with MongoDB, which perform simple queries. In the following table, we present the most common operators and their...