Chapter 2. Acquiring Intelligence Data

We're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW), and this is what we'll focus on in this chapter. Using our own cameras or recording devices is the subject of the next chapter.

Important espionage targets include natural resources, popular opinion, and strategic economic strengths. This kind of background information is useful in a number of ways. A great deal of the world's data is already on the Web, and the rest will get there eventually. Any modern search for intelligence starts with the Web.

We can use Python libraries such as http.client and urllib to get data from remote servers and transfer files to other servers. Once we've found remote files of interest, we're going to need a number of Python libraries to parse and extract data from these libraries.

In Chapter 1, Our...