In the 2008 historic presidential election in the United States, President Barack Obama captured the imagination of the nation with his "Yes we can" slogan and his different but definitive ideas. He employed social media very well to get his message out to the millions of new voters. Fast forward to 2012; he was facing a very difficult re-election campaign. This time he employed data and analytics to win this election. In his November 16, 2012 article in The Atlantic, Alexis Madrigal captures this transformative experimentation in details. For the first time in history, a political campaign had a Chief Technology Officer in Harper Reed. Mr. Reed assembled an eclectic team from Google, Facebook, Twitter, and many other new age technology companies and initiated the project Narwahl. They mined through huge volumes of data—demographic, past voting patterns, economic data, social media interactions, and others to predict how the campaign is going to perform in each seat and how they can persuade individual voters. Big Data is here and now.
In the past 50 years, the world has seen itself transforming to the era of the information age; in the last 5 years, we have seen ourselves gravitating towards the Big Data age. Big Data touches every aspect of our lives. Talking about Big Data, we often refer to huge volumes, variety, and velocity of data—lots of data, different types of data, and data getting created, captured, and processed at breakneck speeds. We often use examples of the 1 billion plus Facebook users, the 3 billion plus likes they click on every day, the 100 billion credit card transactions that happen across the world, the millions of transactions that occur in individual retail chains every hour, the 200 million + e-mails sent every minute, and so on. Sometimes it is difficult for us to fathom the quantum of data.
Let me share an excellent representation that I saw in one of the infographics published by EMC (http://www.emc.com/campaign/global/big-data/hfbd-infographic-4web-1500.jpg). Data is measured in bytes or multiples of that. In the following table, we compare these multiples with more physical equivalents of words/pictures and sand:
|
Data |
Numerical representation |
Words/pictures |
Sand |
|---|---|---|---|
|
Byte |
1 |
Single character |
One grain of sand |
|
Kilobyte |
1,000 |
A sentence |
Couple of pinches of sand |
|
Megabyte |
1,000,000 |
A 20 slide PowerPoint |
Tablespoon of sand |
|
Gigabyte |
1,000,000,000 |
Ten yards of books on a shelf |
Shoebox full of sand |
|
Terabyte |
1,000,000,000,000 |
300 hours of good quality video |
Playground sized sandbox |
|
Petabyte |
1,000,000,000,000,000 |
350,000 digital pictures |
Sand on a mile long beach |
|
Exabyte |
1,000,000,000,000,000,000 |
100,000 times all the printed material in the Library of Congress |
Sand on a beach from Miami to North Carolina |
|
Zettabyte |
1,000,000,000,000,000,000, 000 |
Difficult to get an example |
Sand on all the coastlines across the world |
Traditional technologies have demonstrated limitations when the volume goes beyond a few terabytes. The new reality is terabytes of data are not considered enough to capture every transaction that happens in organizations or businesses in a year. Research group SINTEF published in their May 2013 report that 90 percent of the world's data was created in the past 2 years. In their 2011 Digital Universe Study, IDC has projected growth of information generated to be 50 times of current rate by 2020. Last year the global mobile data traffic is expected to be close to 1 Exabyte per month. So we see massive proliferation of Big Data everywhere every day.
Big Data not only relates to the new age technology companies, large financial institutions, or the mega retail chains; it also relates to traditional manufacturing companies and brick and mortar industries like construction. Today, a medium size building of 10,000 square meters generates over 10 gigabytes of data every year! Everybody is in awe of the size of data and the possibilities it brings. In the last few years, the companies have been scrambling to make sense of this type of Big Data and effectively use this to create new products and services to differentiate themselves and transform their businesses. Billions of dollars are getting invested worldwide in Big Data and trillions of dollars' worth of benefits are expected to be generated from them.
New disciplines of technology like Near Field Communications, Augmented Reality, and many others are getting enabled and impacted by Big Data. New economic models like Bitcoins are getting introduced because of the processing power unleashed by Big Data. Big Data is changing everything—the way we work, the way we live, and the way we interact with each other.
Not everybody has tasted success in pursuit of Big Data. While some companies experienced phenomenal success in adapting Big Data for their business, many others are still tentative and not sure how to pursue this space. There are many reasons for limited success or apprehension:
Big Data is not only about technology, there needs to be a larger enabling ecosystem and the program needs to be integrated into the DNA of the business. There is limited assistance available on how to do that.
There is a dearth of qualified and experienced people in different aspects of Big Data.
There are very few successful established business models to create new value or unlock untapped potential across various industry verticals.
When analytics supported by Big Data challenges many of the conventional practices and beliefs, a number of managers find it difficult to accept these new insights and act upon them. After all, they are influenced by over 50 years of evolution of management practices and models.
Sometimes people equate Big Data to a specific technology application without understanding the complete stack, thereby limiting the possibilities of what can be leveraged or how to apply the different capabilities.
There is clear acknowledgement today that we need a more holistic approach to and understanding of Big Data to use it effectively.
The technology for Big Data is evolving at a very rapid pace. You can access the new advancements easily and inexpensively. On the other hand, Big Data is fundamentally changing many paradigms of how we manage and grow businesses. Unfortunately, unlike many other disciplines, there is not enough guidance available on how to best leverage all of these new capabilities. This book is an attempt to address that space. This book also makes an attempt to tackle some of the limiting factors in the adoption of Big Data from amongst the points we discussed above.
In this book we will cover 8 different aspects of Big Data and how to be most effective in them.
Chapter 1, Building Your Strategy Framework, focuses on helping you map a path for yourself to make a big impact in your business using Big Data. This chapter helps you put a strategic context to Big Data and introduces a few tools to develop strategies, align existing ones, and cascade your new strategies across the organization.
Chapter 2, Creating an Opportunity Landscape and Collecting Your Gold Coins, deals with identifying and structuring specific initiatives around Big Data. This chapter introduces you to an approach of crafting Big Data projects as gold coins—identifying the gold coins, assessing their value, prioritizing them, and finally building a gold mine for greater organizational value by putting together all the gold coins.
Chapter 3, Managing Your Big Data Projects Effectively, starts with understanding how Big Data projects are different from other types of technology-intensive business transformation initiatives. It then helps you create a new project management framework Explore Validate Amplify better suited for Big Data projects, defining success criteria and management methodologies to deploy this framework. Finally, it provides guidance on how to handle different types of Big Data initiatives.
Chapter 4, Building the Right Technology Landscape, helps you understand the various technology layers and choices around Big Data and how they have evolved. This chapter helps you understand the differences between Big Data initiatives and massive data warehousing projects that people often get confused about. In this chapter, you are provided with some guidance on designing storage for Big Data and selecting the right technology for the various layers described.
Chapter 5, Building a Winning Team, introduces you to the various skills and the 6 different profiles required in your project's team to effectively implement various Big Data initiatives. This chapter also helps you identify where to source people for your Big Data projects, how to organize them, how to manage and motivate them, how to leverage external resources, and develop organizational capability for future needs.
Chapter 6, Managing Investments and Monetization of Data, deals with financial aspects around Big Data. It focuses on two important topics—valuation and monetization of data, and investment management of Big Data initiatives. The tools and techniques in this chapter will help you maximize your investments and unlock untapped value from your business.
Chapter 7, Driving Change Effectively, helps you increase the sustainability of business benefits brought about by Big Data through effective change management practices. This chapter helps you understand the major changes that are caused by Big Data and their significance for your business. This chapter also introduces you to a new change management framework called IMMERSE and how to deploy it. Finally, this chapter helps you develop a guiding coalition to drive change in your organization.
Chapter 8, Driving Communication Effectively, starts with identifying the need for increased communication around Big Data. It helps you identify your communication needs for various audiences, understand the various communication channels available and determine which ones are the most appropriate ones for those audience groups, build a communication plan, and manage the communication deployment. This chapter introduces you the TALK model to make communication effective.
This is a how-to book, designed and developed to help you implement some of the concepts in a very practical manner. It is therefore filled with examples from many walks of business. For many of these concepts and practices, there is no well-established precedence, so you will be introduced to many new frameworks and models.
This book is intended for all the current and prospective practitioners of Big Data. This book is ideal if you are:
Leading a Big Data initiative for your business or are thinking about starting one
Part of the executive management team of a business and want to explore what this new craze around Big Data can do for your business
A technology expert in the space of data and analytics who wants to become more effective in adding value to your business through better adoption of Big Data
Part of a team that is struggling with using Big Data in your business
A student who sees this as a potential career option
A consultant who wants to help clients make more effective use of Big Data
An industry analyst who wants to understand what makes one successful in pursuing Big Data
An investor with companies that you think can make a big difference to its performance using Big Data
Every business of all sizes has equal opportunities with Big Data. This book is as much for people who work in or work for large companies, as much it is for people engaged with smaller businesses.
Peter Drucker made these now famous remarks, culture eats strategy for breakfast. To be effective in future, we need a culture of Big Data in our businesses. We hope this book will help you create that culture for your business.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
New terms and important words are shown in bold.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.