Building a search engine from scratch is a task for seasoned programmers. We have touched on many topics in this book and combined most of them in this chapter by designing a search engine.
We have learned that web search engines are complex systems consisting of several components, such as the crawler, the indexer, and user interface. The crawler is responsible for regularly checking the web to download web pages for the search engine to index. Indexing results in the production of a big data structure called an inverted index. An inverted index, or just an index, is a data structure that maps words with the documents in which they have occurred.
Next, we defined what a recommendation engine is and tried to design a simple one for our search engine. The recommendation engine was connected to the dialog-based features of the search engine discussed in this chapter. A...