Chapter 1, Thinking Probabilistically – A Bayesian Inference Primer, tells us about Bayes' theorem and its implications for data analysis. We then proceed to describe the Bayesian-way of thinking and how and why probabilities are used to deal with uncertainty. This chapter contains the foundational concepts used in the rest of the book.

Chapter 2, Programming Probabilistically – A PyMC3 Primer, revisits the concepts from the previous chapter, this time from a more computational perspective. The PyMC3 library is introduced and we learn how to use it to build probabilistic models, get results by sampling from the posterior, diagnose whether the sampling was done right, and analyze and interpret Bayesian results.

Chapter 3, Juggling with Multi-Parametric and Hierarchical Models, tells us about the very basis of Bayesian modeling and we start adding complexity to the mix. We learn how to build and analyze models with more than one parameter and how to put structure into models, taking advantages of hierarchical models.

Chapter 4, Understanding and Predicting Data with Linear Regression Models, tells us about how linear regression is a very widely used model per se and a building block of more complex models. In this chapter, we apply linear models to solve regression problems and how to adapt them to deal with outliers and multiple variables.

Chapter 5, Classifying Outcomes with Logistic Regression, generalizes the the linear model from previous chapter to solve classification problems including problems with multiple input and output variables.

Chapter 6, Model Comparison, discusses the difficulties associated with comparing models that are common in statistics and machine learning. We will also learn a bit of theory behind the information criteria and Bayes factors and how to use them to compare models, including some caveats of these methods.

Chapter 7, Mixture Models, discusses how to mix simpler models to build more complex ones. This leads us to new models and also to reinterpret models learned in previous chapters. Problems, such as data clustering and dealing with count data, are discussed.

Chapter 8, Gaussian Processes, closes the book by briefly discussing some more advanced concepts related to non-parametric statistics. What kernels are, how to use kernelized linear regression, and how to use Gaussian processes for regression are the central themes of this chapter.

This book is written for Python version >= 3.5, and it is recommended that you use the most recent version of Python 3 that is currently available, although most of the code examples may also run for older versions of Python, including Python 2.7 with minor adjustments.

Maybe the easiest way to install Python and Python libraries is using Anaconda, a scientific computing distribution. You can read more about Anaconda and download it from https://www.continuum.io/downloads. Once Anaconda is in our system, we can install new Python packages with this command: conda install NamePackage.

We will use the following python packages:

Undergraduate or graduate students, scientists, and data scientists who are not familiar with the Bayesian statistical paradigm and wish to learn how to do Bayesian data analysis. No previous knowledge of statistics is assumed, for either Bayesian or other paradigms. The required mathematical knowledge is kept to a minimum and all concepts are described and explained with code, figures, and text. Mathematical formulas are used only when we think it can help the reader to better understand the concepts. The book assumes you know how to program in Python. Familiarity with scientific libraries such as NumPy, matplotlib, or Pandas is helpful but not essential.

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "To compute the HPD in the correct way we will use the function plot_post."

A block of code is set as follows:

n_params = [1, 2, 4]
p_params = [0.25, 0.5, 0.75]
x = np.arange(0, max(n_params)+1)
f, ax = plt.subplots(len(n_params), len(p_params), sharex=True, 
  sharey=True)
for i in range(3):
    for j in range(3):
        n = n_params[i]
        p = p_params[j]
        y = stats.binom(n=n, p=p).pmf(x)
        ax[i,j].vlines(x, 0, y, colors='b', lw=5)
        ax[i,j].set_ylim(0, 1)
        ax[i,j].plot(0, 0, label="n = {:3.2f}\np = {:3.2f}".format(n, p), alpha=0)
        ax[i,j].legend(fontsize=12)
ax[2,1].set_xlabel('$\\theta$', fontsize=14)
ax[1,0].set_ylabel('$p(y|\\theta)$', fontsize=14)
ax[0,0].set_xticks(x)

Any command-line input or output is written as follows:

conda install NamePackage

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.