Let's look at an example dynamic web page. The example website has a search form, which is available at http://example.webscraping.com/search, that is used to locate countries. Let's say we want to find all the countries that begin with the letter A:
If we right-click on these results to inspect them with Firebug (as covered in Chapter 2, Scraping the Data), we would find that the results are stored within a
div element of ID "results":
>>> import lxml.html >>> from downloader import Downloader >>> D = Downloader() >>> html = D('http://example.webscraping.com/search') >>> tree = lxml.html.fromstring(html) >>> tree.cssselect('div#results a') 
The example scraper here has failed to extract results. Examining the source code of this web page...