Having read this chapter, you should now understand the fundamentals of web scraping, such as performing an HTTP GET request and searching for a string using string matching or regular expressions to find HTML comments, emails, and other keywords. You should also understand how to extract the HTTP headers and set custom headers to set cookies and custom user agent strings. Moreover, you should understand the basic concepts of fingerprinting and have some idea of how to gather information about a web application based on the source code provided.
Having worked through this chapter, you should also understand the basics of using the goquery package to find HTML elements in the DOM in a jQuery style. You should feel comfortable finding links in a web page, finding documents, listing title and headers, finding JavaScript files, and finding the difference between breadth-first...
