Apache Solr 4 Cookbook

Apache Solr 4 Cookbook

By : Rafał Kuć

Buy this Book

Apache Solr 4 Cookbook

By: Rafał Kuć

Buy this Book

Overview of this book

Apache Solr is a blazing fast, scalable, open source Enterprise search server built upon Apache Lucene. Solr is wildly popular because it supports complex search criteria, faceting, result highlighting, query-completion, query spell-checking, and relevancy tuning, amongst other numerous features. "Apache Solr 4 Cookbook" will show you how to get the most out of your search engine. Full of practical recipes and examples, this book will show you how to set up Apache Solr, tune and benchmark performance as well as index and analyze your data to provide better, more precise, and useful search data. "Apache Solr 4 Cookbook" will make your search better, more accurate and faster with practical recipes on essential topics such as SolrCloud, querying data, search faceting, text and data analysis, and cache configuration. With numerous practical chapters centered on important Solr techniques and methods, Apache Solr 4 Cookbook is an essential resource for developers who wish to take their knowledge and skills further. Thoroughly updated and improved, this Cookbook also covers the changes in Apache Solr 4 including the awesome capabilities of SolrCloud.

Apache Solr 4 Cookbook

Credits

About the Author

Acknowledgement

About the Reviewers

www.PacktPub.com

Preface

Free Chapter

Apache Solr Configuration

Introduction

Running Solr on Jetty

Running Solr on Apache Tomcat

Installing a standalone ZooKeeper

Clustering your data

Choosing the right directory implementation

Configuring spellchecker to not use its own index

Solr cache configuration

How to fetch and index web pages

How to set up the extracting request handler

Changing the default similarity implementation

Indexing Your Data

Introduction

Indexing PDF files

Generating unique fields automatically

Extracting metadata from binary files

How to properly configure Data Import Handler with JDBC

Indexing data from a database using Data Import Handler

How to import data using Data Import Handler and delta query

How to use Data Import Handler with the URL data source

How to modify data while importing with Data Import Handler

Updating a single field of your document

Handling multiple currencies

Detecting the document's language

Optimizing your primary key field indexing

Analyzing Your Text Data

Introduction

Storing additional information using payloads

Eliminating XML and HTML tags from text

Copying the contents of one field to another

Changing words to other words

Splitting text by CamelCase

Splitting text by whitespace only

Making plural words singular without stemming

Lowercasing the whole string

Storing geographical points in the index

Stemming your data

Preparing text to perform an efficient trailing wildcard search

Splitting text by numbers and non-whitespace characters

Using Hunspell as a stemmer

Using your own stemming dictionary

Protecting words from being stemmed

Querying Solr

Introduction

Asking for a particular field value

Sorting results by a field value

How to search for a phrase, not a single word

Boosting phrases over words

Positioning some documents over others on a query

Positioning documents with words closer to each other first

Sorting results by a distance from a point

Getting documents with only a partial match

Affecting scoring with functions

Nesting queries

Modifying returned documents

Using parent-child relationships

Ignoring typos in terms of performance

Detecting and omitting duplicate documents

Using field aliases

Returning a value of a function in the results

Using the Faceting Mechanism

Introduction

Getting the number of documents with the same field value

Getting the number of documents with the same value range

Getting the number of documents matching the query and subquery

Removing filters from faceting results

Sorting faceting results in alphabetical order

Implementing the autosuggest feature using faceting

Getting the number of documents that don't have a value in the field

Having two different facet limits for two different fields in the same query

Using decision tree faceting

Calculating faceting for relevant documents in groups

Improving Solr Performance

Introduction

Paging your results quickly

Configuring the document cache

Configuring the query result cache

Configuring the filter cache

Improving Solr performance right after the startup or commit operation

Caching whole result pages

Improving faceting performance for low cardinality fields

What to do when Solr slows down during indexing

Analyzing query performance

Avoiding filter caching

Controlling the order of execution of filter queries

Improving the performance of numerical range queries

In the Cloud

Introduction

Creating a new SolrCloud cluster

Setting up two collections inside a single cluster

Managing your SolrCloud cluster

Understanding the SolrCloud cluster administration GUI

Distributed indexing and searching

Increasing the number of replicas on an already live cluster

Stopping automatic document distribution among shards

Using Additional Solr Functionalities

Introduction

Getting more documents similar to those returned in the results list

Highlighting matched words

How to highlight long text fields and get good performance

Sorting results by a function value

Searching words by how they sound

Ignoring defined words

Computing statistics for the search results

Checking the user's spelling mistakes

Using field values to group results

Using queries to group results

Using function queries to group results

Dealing with Problems

Introduction

How to deal with too many opened files

How to deal with out-of-memory problems

How to sort non-English languages properly

How to make your index smaller

Diagnosing Solr problems

How to avoid swapping

Real-life Situations

Introduction

How to implement a product's autocomplete functionality

How to implement a category's autocomplete functionality

How to use different query parsers in a single query

How to get documents right after they were sent for indexation

How to search your data in a near real-time manner

How to get the documents with all the query words to the top of the results set

How to boost documents based on their publishing date

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Detecting and omitting duplicate documents

Imagine your data consists of duplicates because they come from different sources. For example, you have books that come from different suppliers, but you are only interested in a single book with the same name. Of course you could use the field collapsing feature during the query, but that affects query performance and we would like to avoid that. This recipe will show you how to use the Solr deduplication functionality.

How to do it...

We start with the simple index structure. This should be placed in the fields section of your schema.xml file:

<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="name" type="text" indexed="true" stored="true" multiValued="false"/>
<field name="type" type="string" indexed="true" stored="true" multiValued="false"/>

For the purpose of the recipe, we assume that we have the following data stored in the data.xml file:
```
<add>
 <doc>...
```

Apache Solr 4 Cookbook

By : Rafał Kuć

Apache Solr 4 Cookbook

By: Rafał Kuć

Overview of this book

Related Content you might be interested in

Current Title:

Apache Solr 4 Cookbook

Detecting and omitting duplicate documents

How to do it...