Book Image

Hands-On Big Data Modeling

By : James Lee, Tao Wei, Suresh Kumar Mukhiya
Book Image

Hands-On Big Data Modeling

By: James Lee, Tao Wei, Suresh Kumar Mukhiya

Overview of this book

Modeling and managing data is a central focus of all big data projects. In fact, a database is considered to be effective only if you have a logical and sophisticated data model. This book will help you develop practical skills in modeling your own big data projects and improve the performance of analytical queries for your specific business requirements. To start with, you’ll get a quick introduction to big data and understand the different data modeling and data management platforms for big data. Then you’ll work with structured and semi-structured data with the help of real-life examples. Once you’ve got to grips with the basics, you’ll use the SQL Developer Data Modeler to create your own data models containing different file types such as CSV, XML, and JSON. You’ll also learn to create graph data models and explore data modeling with streaming data using real-world datasets. By the end of this book, you’ll be able to design and develop efficient data models for varying data sizes easily and efficiently.
Table of Contents (17 chapters)

Importing Twitter feed data

In this chapter, we are going to use the Twitter dataset. The dataset can be downloaded from Kaggle (https://www.kaggle.com/kingburrito666/better-donald-trump-tweets). In addition to that, you can find the dataset inside this book's GitHub repository. The dataset is found inside the CH13 folder. The file is called Donald-Tweets!.csv.

Let's dive into the exercise by running the Jupyter notebook we've been using throughout this book. Once you have the notebook up and running, import the required essential packages. Most of the Python packages should be very familiar by now:

from textblob import TextBlob
import math
import pandas as pd
import numpy as np
import os
from pandas import DataFrame
from sklearn.cluster import KMeans
from sklearn import preprocessing
import matplotlib.pyplot as plt
from matplotlib import style
import matplotlib.pyplot as...