Book Image

R Programming By Example

By : Omar Trejo Navarro
Book Image

R Programming By Example

By: Omar Trejo Navarro

Overview of this book

R is a high-level statistical language and is widely used among statisticians and data miners to develop analytical applications. Often, data analysis people with great analytical skills lack solid programming knowledge and are unfamiliar with the correct ways to use R. Based on the version 3.4, this book will help you develop strong fundamentals when working with R by taking you through a series of full representative examples, giving you a holistic view of R. We begin with the basic installation and configuration of the R environment. As you progress through the exercises, you'll become thoroughly acquainted with R's features and its packages. With this book, you will learn about the basic concepts of R programming, work efficiently with graphs, create publication-ready and interactive 3D graphs, and gain a better understanding of the data at hand. The detailed step-by-step instructions will enable you to get a clean set of data, produce good visualizations, and create reports for the results. It also teaches you various methods to perform code profiling and performance enhancement with good programming practices, delegation, and parallelization. By the end of this book, you will know how to efficiently work with data, create quality visualizations and reports, and develop code that is modular, expressive, and maintainable.
Table of Contents (12 chapters)

Improving our results with TF-IDF

In general in text analysis, a high raw count for a term inside a text does not necessarily mean that the term is more important for the text. One of the most important ways to normalize the term frequencies is to weigh a term by how often it appears not only in a text, but also in the entire corpus.

The more a word appears inside a given text and doesn't appear too much across the whole corpus, it means that it's probably important for that specific text. However, if the term appears a lot inside a text, but also appears a lot in other texts in the corpus, it's probably not important for the specific text, but for the entire corpus, and this dilutes it's predictive power.

In IR, TF-IDF is one of the most popular term-weighting schemes and it's the mathematical implementation of the idea expressed in the preceding paragraph...