Import all the required libraries and packages into the Jupyter notebook. Make sure to read the data from bank.csv into the Spark DataFrame.
Import the libraries for Plotly, as illustrated here:
import plotly.graph_objs as go from plotly.plotly import iplot import plotly as py
Now, for visualization in Plotly, we need to initiate an offline session. Use the following command (requires version >= 1.9.0):
from plotly import __version__ from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot print(__version__)
Now Plotly is initiated offline. Use the following command to start a Plotly notebook:
import plotly.plotly as py import plotly.graph_objs as go init_notebook_mode(connected=True)
After starting the Plotly notebook, we can use Plotly to generate many types of graphs, such as a bar graph, a boxplot, or a scatter plot, and convert the entire output into a user interface or an app that is supported by Python's Flask framework.
Now, plot each graph using Plotly:
Bar graph:
df = pd.read_csv('bank.csv', sep=';') data = [go.Bar(x=df.y, y=df.balance)] py.iplot(data)The bar graph is as follows:
Figure 8.18: Bar graph
Scatter plot:
py.iplot([go.Histogram2dContour(x=df.balance, y=df.age, contours=dict(coloring='heatmap')), go.Scatter(x=df.balance, y=df.age, mode='markers', marker=dict(color='red', size=8, opacity=0.3))], show_link=False)The scatter plot is as follows:
Figure 8.19: Scatter plot
Boxplot:
plot1 = go.Box( y=df.age, name = 'age of the customers', marker = dict( color = 'rgb(12, 12, 140)', ) ) py.iplot([plot1])The boxplot is as follows:
Figure 8.20: Boxplot
Big Data Analysis with Python
By :
Big Data Analysis with Python
By:
Overview of this book
Processing big data in real time is challenging due to scalability, information inconsistency, and fault tolerance. Big Data Analysis with Python teaches you how to use tools that can control this data avalanche for you. With this book, you'll learn practical techniques to aggregate data into useful dimensions for posterior analysis, extract statistical measurements, and transform datasets into features for other systems.
The book begins with an introduction to data manipulation in Python using pandas. You'll then get familiar with statistical analysis and plotting techniques. With multiple hands-on activities in store, you'll be able to analyze data that is distributed on several computers by using Dask. As you progress, you'll study how to aggregate data for plots when the entire data cannot be accommodated in memory. You'll also explore Hadoop (HDFS and YARN), which will help you tackle larger datasets. The book also covers Spark and explains how it interacts with other tools.
By the end of this book, you'll be able to bootstrap your own Python environment, process large files, and manipulate data to generate statistics, metrics, and graphs.
Related Content you might be interested in
Current Title:
Big Data Analysis with Python
Table of Contents (11 chapters)
Big Data Analysis with Python
Preface
Free Chapter
The Python Data Science Stack
Statistical Visualizations
Working with Big Data Frameworks
Diving Deeper with Spark
Handling Missing Values and Correlation Analysis
Exploratory Data Analysis
Reproducibility in Big Data Analysis
Creating a Full Analysis Report
Appendix
Customer Reviews