Often when data mining, an analyst is happy to get their hands on a new feature column because the hope is that this added feature will bring additional new information. While this expectation fits with human intuition, there is an enormous caveat that must be understood and respected. This caveat is a result of what's known as the curse of dimensionality, which was coined in the 1950s by the mathematician, Richard E. Bellman. In short, a statistically-significant representation of chunks of feature space requires exponentially more and more examples (that is, rows) as the number of dimensions (that is, features) grows. Failure to grow the number of examples with the number of dimensions causes the dataset to become sparse and no longer representative of ground truth. The common rule of thumb is that you should have five examples for every one dimension...
Python Data Mining Quick Start Guide
By :
Python Data Mining Quick Start Guide
By:
Overview of this book
Data mining is a necessary and predictable response to the dawn of the information age. It is typically defined as the pattern and/ or trend discovery phase in the data mining pipeline, and Python is a popular tool for performing these tasks as it offers a wide variety of tools for data mining.
This book will serve as a quick introduction to the concept of data mining and putting it to practical use with the help of popular Python packages and libraries. You will get a hands-on demonstration of working with different real-world datasets and extracting useful insights from them using popular Python libraries such as NumPy, pandas, scikit-learn, and matplotlib. You will then learn the different stages of data mining such as data loading, cleaning, analysis, and visualization. You will also get a full conceptual description of popular data transformation, clustering, and classification techniques.
By the end of this book, you will be able to build an efficient data mining pipeline using Python without any hassle.
Related Content you might be interested in
Current Title:
Python Data Mining Quick Start Guide
Table of Contents (9 chapters)
Preface
Free Chapter
Data Mining and Getting Started with Python Tools
Basic Terminology and Our End-to-End Example
Collecting, Exploring, and Visualizing Data
Cleaning and Readying Data for Analysis
Grouping and Clustering Data
Prediction with Regression and Classification
Advanced Topics - Building a Data Processing Pipeline and Deploying It
Other Books You May Enjoy
Customer Reviews