Book Image

Scaling Big Data with Hadoop and Solr, Second Edition

By : Hrishikesh Vijay Karambelkar
Book Image

Scaling Big Data with Hadoop and Solr, Second Edition

By: Hrishikesh Vijay Karambelkar

Overview of this book

Table of Contents (13 chapters)
Scaling Big Data with Hadoop and Solr Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Index optimization


The indexes used in Apache Solr are inverted indexes. In case of the inverted indexing technique, all your text will be parsed and words will be extracted out of it. These words are then stored as index items, with the location of their appearance. For example, consider the following statements:

  1. "Mike enjoys playing on a beach"

  2. "Playing on the ground is a good exercise"

  3. "Mike loves to exercise daily"

The index with location information for all these sentences will look like following (Numbers in brackets denote (sentence number, word number)):

Mike     (1,1), (3,1)
enjoys   (1,2)
playing  (1,3), (2,1)
on       (1,4), (2,2)
a        (1,5), (2,5)
beach    (1,6)
ground   (2,3)
is       (2,4)
good     (2,6)
loves    (3,2)
to       (3,3)
exercise (2,7), (3,4)
daily    (3,5)

When you perform a delete on your inverted index, it does not delete the document; it only marks the document as deleted. It will get cleaned only when the segment that the index is a part of is merged. When you...