Data science is an exciting new field that is used by various organizations to perform data-driven decisions. It is a combination of technical knowledge, mathematics, and business. Data scientists have to wear various hats to work with data and derive some value out of it. Python is one of the most popular languages among all the languages used by data scientists. It is a simple language to learn and is used for purposes, such as web development, scripting, and application development to name a few.
The ability to perform data science using Python is very powerful as it helps clean data at a raw level to create advanced machine learning algorithms that predict customer churns for a retail company. This book explains various concepts of data science in a structured manner with the application of these concepts on data to see how to interpret results. The book provides a good base for understanding the advanced topics of data science and how to apply them in a real-world scenario.
Chapter 1, Getting Started with Raw Data, teaches you the techniques of handling unorganized data. You'll also learn how to extract data from different sources, as well as how to clean and manipulate it.
Chapter 2, Inferential Statistics, goes beyond descriptive statistics, where you'll learn about inferential statistics concepts, such as distributions, different statistical tests, the errors in statistical tests, and confidence intervals.
Chapter 3, Finding a Needle in a Haystack, explains what data mining is and how it can be utilized. There is a lot of information in data but finding meaningful information is an art.
Chapter 4, Making Sense of Data through Advanced Visualization, teaches you how to create different visualizations of data. Visualization is an integral part of data science; it helps communicate a pattern or relationship that cannot be seen by looking at raw data.
Chapter 5, Uncovering Machine Learning, introduces you to the different techniques of machine learning and how to apply them. Machine learning is the new buzzword in the industry. It's used in activities, such as Google's driverless cars and predicting the effectiveness of marketing campaigns.
Chapter 6, Performing Predictions with a Linear Regression, helps you build a simple regression model followed by multiple regression models along with methods to test the effectiveness of the models. Linear regression is one of the most popular techniques used in model building in the industry today.
Chapter 7, Estimating the Likelihood of Events, teaches you how to build a logistic regression model and the different techniques of evaluating it. With logistic regression, you'll be able learn how to estimate the likelihood of an event taking place.
Chapter 8, Generating Recommendations with Collaborative Filtering, teaches you to create a recommendation model and apply it. It is similar to websites, such as Amazon, which are able to suggest items that you would probably buy on their page.
Chapter 9, Pushing Boundaries with Ensemble Models, familiarizes you with ensemble techniques, which are used to combine the power of multiple models to enhance the accuracy of predictions. This is done because sometimes a single model is not enough to estimate the outcome.
Chapter 10, Applying Segmentation with k-means Clustering, teaches you about k-means clustering and how to use it. Segmentation is widely used in the industry to group similar customers together.
Chapter 11, Analyzing Unstructured Data with Text Mining, teaches you to process unstructured data and make sense of it. There is more unstructured data in the world than structured data.
Chapter 12, Leveraging Python in the World of Big Data, teaches you to use Hadoop and Spark with Python to handle data in this chapter. With the ever increasing size of data, big data technologies have been brought into existence to handle such data.
The following softwares are required for this book:
Ubuntu OS, preferably 14.04
Python 2.7
The pandas 0.16.2 library
The NumPy 1.9.2 library
The SciPy 0.16 library
IPython 4.0
The SciKit 0.16.1 module
The statsmodels 0.6.1 module
The matplotlib 1.4.3 library
Apache Hadoop CDH4 (Cloudera Hadoop 4) with MRv1 (MapReduce version 1)
Apache Spark 1.4.0
If you are a Python developer who wants to master the world of data science, then this book is for you. It is assumed that you already have some knowledge of data science.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The json.load() function loads the data into Python."
Any command-line input or output is written as follows:
$ pig ./BigData/pig_sentiment.pig
New terms and important words are shown in bold.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
The codes provided in the code bundle are for both IPython notebook and Python 2.7. In the chapters, Python conventions have been followed.
We also provide you a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from: http://www.packtpub.com/sites/default/files/downloads/0150OS_ColorImage.pdf.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.