Book Image

Using OpenRefine

Book Image

Using OpenRefine

Overview of this book

Data today is like gold - but how can you manage your most valuable assets? Managing large datasets used to be a task for specialists, but the game has changed - data analysis is an open playing field. Messy data is now in your hands! With OpenRefine the task is a little easier, as it provides you with the necessary tools for cleaning and presenting even the most complex data. Once it's clean, that's when you can start finding value. Using OpenRefine takes you on a practical and actionable through this popular data transformation tool. Packed with cookbook style recipes that will help you properly get to grips with data, this book is an accessible tutorial for anyone that wants to maximize the value of their data. This book will teach you all the necessary skills to handle any large dataset and to turn it into high-quality data for the Web. After you learn how to analyze data and spot issues, we'll see how we can solve them to obtain a clean dataset. Messy and inconsistent data is recovered through advanced techniques such as automated clustering. We'll then show extract links from keyword and full-text fields using reconciliation and named-entity extraction. Using OpenRefine is more than a manual: it's a guide stuffed with tips and tricks to get the best out of your data.
Table of Contents (13 chapters)
Using OpenRefine
Credits
Foreword
About the Authors
About the Reviewers
www.PacktPub.com
Preface
Index

Recipe 1 – installing OpenRefine


In this recipe, you will learn where to look in order to download the latest release of OpenRefine and how to get it running on your favorite operating system.

First things first: start by downloading OpenRefine from http://openrefine.org/. OpenRefine was previously known as Freebase Gridworks, then as Google Refine for a few years. Since October 2012, the project has been taken over by the community, which makes OpenRefine really open. OpenRefine 2.6 is the first version carrying the new branding. If you are interested in the development version, you can also check https://github.com/OpenRefine.

OpenRefine is based on the Java environment, which makes it platform-independent. Just make sure that you have an up-to-date version of Java running on your machine (available from http://java.com/download) and follow the following instructions, depending on your operating system:

Windows

  1. Download the ZIP archive.

  2. Unzip and extract the contents of the archive to a folder of your choice.

  3. To launch OpenRefine, double-click on openrefine.exe.

Mac

  1. Download the DMG file.

  2. Open the disk image and drag the OpenRefine icon into the Applications folder.

  3. Double-click on the icon to start OpenRefine.

Linux

  1. Download the gzipped tarball.

  2. Extract the folder to your home directory.

  3. In a terminal, enter ./refine to start.

It should be noted that, by default, OpenRefine will allocate only 1 GB of RAM to Java. While this is sufficient to handle small datasets, it soon becomes restrictive when dealing with larger collections of data. In Recipe 7 – going for more memory, we will detail how to allow OpenRefine to allocate more memory, an operation that also differs from one OS to the other.