6. Querying and Filtering Data | Lucene 4 Cookbook

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

Lucene 4 Cookbook

By : Edwood Ng, Mohan

3.2 (5)

Lucene 4 Cookbook

3.2 (5)

By: Edwood Ng, Mohan

Overview of this book

This book is for software developers who are new to Lucene and who want to explore the more advanced topics to build a search engine. Knowledge of Java is necessary to follow the code samples. You will learn core concepts, best practices, and also advanced features, in order to build an effective search application.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

1. Introducing Lucene

1. Introducing Lucene

Introduction

Installing Lucene

Setting up a simple Java Lucene project

Obtaining an IndexWriter

Creating an analyzer

Creating fields

Creating and writing documents to an index

Deleting documents

Obtaining an IndexSearcher

Creating queries with the Lucene QueryParser

Performing a search

Enumerating results

2. Analyzing Your Text

2. Analyzing Your Text

Introduction

Obtaining a common analyzer

Obtaining a TokenStream

Obtaining TokenAttribute values

Using PositionIncrementAttribute

Using PerFieldAnalyzerWrapper

Defining custom TokenFilters

Defining custom analyzers

Defining custom tokenizers

Defining custom attributes

3. Indexing Your Data

3. Indexing Your Data

Introduction

Obtaining an IndexWriter

Creating a StringField

Creating a TextField

Creating a numeric field

Creating a DocValue Field

Transactional commits and index versioning

Reusing field and document objects per thread

Delving into field norms

Changing similarity implementation used during indexing

4. Searching Your Indexes

4. Searching Your Indexes

Introduction

Obtaining IndexReaders

Un-inverting single-valued fields in memory with FieldCache

TermVectors

IndexSearcher

Constructing queries

Specifying sort logic

Forming a search result

Pagination

Using Collectors

Sorting with custom FieldComparator

5. Near Real-time Searching

5. Near Real-time Searching

Introduction

Using the DirectoryReader to open index in Near Real-Time

Using the SearcherManager to refresh IndexSearcher

Generational indexing with TrackingIndexWriter

Maintaining search sessions with SearcherLifetimeManager

Performance tuning: latency and throughput

6. Querying and Filtering Data

6. Querying and Filtering Data

Introduction

Performing advanced filtering

Creating a custom filter

Searching with QueryParser

TermQuery and TermRangeQuery

BooleanQuery

PrefixQuery and WildcardQuery

PhraseQuery and MultiPhraseQuery

FuzzyQuery

NumericRangeQuery

DisjunctionMaxQuery

RegexpQuery

SpanQuery

CustomScoreQuery

7. Flexible Scoring

7. Flexible Scoring

Introduction

Overriding similarity

Implementing the BM25 model

Implementing the language model

Implementing the divergence from randomness model

Implementing the information-based model

8. Introducing Elasticsearch

8. Introducing Elasticsearch

Introduction

Getting Elasticsearch

Creating a new index

Predefine field mappings

Adding a document

Deleting a document

Updating a document

Performing bulk indexing

Searching the index

Scaling Elasticsearch

9. Extending Lucene with Modules

9. Extending Lucene with Modules

Introduction

Exploring spatial search

Implementing joins

Performing faceting

Implementing grouping

Employing autosuggest

Implementing highlighting

Index

Index

Customer Reviews

3.2 (5)

5 star

20%

4 star

20%

3 star

40%

2 star

0%

1 star

20%

FuzzyQuery

A FuzzyQuery matches terms based on similarity, using the Damerau-Levenshtein algorithm. We are not going into the details of the algorithm as it is outside of our topic. What we need to know is a fuzzy match is measured in the number of edits between terms. FuzzyQuery allows a maximum of 2 edits. For example, between "humptX" and humpty is first edit and between humpXX and humpty are two edits. There is also a requirement that the number of edits must be less than the minimum term length (of either the input term or candidate term). As another example, ab and abcd would not match because the number of edits between the two terms is 2 and it's not greater than the length of ab, which is 2.

How to do it...

Here is a code snippet to demonstrate FuzzyQuery:

FuzzyQuery query = new FuzzyQuery(new Term("content", "humpXX"));

How it works…

This Query will return sentences one, two, and four from our setup, as humpXX matches humpty within the two edits. In QueryParser, FuzzyQuery can be triggered...

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Lucene 4 Cookbook

Search

Your notes and bookmarks