Now that we know what text tagging is and have seen some algorithms that can be used for text tagging, let us learn how text tagging is done using Solr. There is an open source library, Solr Text Tagger, that can be used for text tagging in Solr.
Note
The library can be referred to at the following link: https://github.com/OpenSextant/SolrTextTagger.
Text tagging via this library involves two layers of FSTs. A word dictionary FST is used to hold each unique word. This enables integers to be used as substitutes for a word (char[]). For example, the word New will be mapped to 13452 and another word Delhi will be mapped to 5223316:
New => 13452 Delhi => 5223316
The call to Lucene's FST library Util.getByOutput(<fst object>, 13452) will yield the word New.
The second layer is a word phrase FST comprising word ID string keys. In the case of New Delhi, the word phrase FST will be:
New Delhi => [13452, 5223316]
The tagging algorithm used in the Solr text tagger...