Imagine a situation where you have a user constantly paging through the search results. For example, one of the clients I was working for was struggling with the performance of his website. His users tend to search for a word and then page through the result pages – the statistical information gathered from the application logs showed that typical users changed the page about four to seven times. Apart from improving the query relevance (which isn't what we will talk about in this recipe), we decided to optimize the paging. How do we do that? This recipe will show you.
So, let's get back to my client deployment. As I mentioned, typical users typed a word into the search box and then used the paging mechanism to go through a maximum of seven pages. My client's application was showing 20 documents on a single page. So, it can be easily calculated that we need about 140 documents in advance, apart from the first 20 documents returned by the query.
So...