This book has so far introduced scraping techniques using a custom website, which helped us focus on learning particular skills. Now, in this chapter, we will analyze a variety of real-world websites to show how these techniques can be applied. Firstly, we will use Google to show a real-world search form, then Facebook for a JavaScript-dependent website, Gap for a typical online store, and finally, BMW for a map interface. Since these are live websites, there is a risk that they will have changed by the time you read this. However, this is fine because the purpose of these examples is to show you how the techniques learned so far can be applied, rather than to show you how to scrape a particular website. If you choose to run an example, first check whether the website structure has changed since these examples were made and whether their current terms and conditions prohibit scraping.

Web Scraping with Python
By :

Web Scraping with Python
By:
Overview of this book
Table of Contents (16 chapters)
Web Scraping with Python
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Introduction to Web Scraping
Scraping the Data
Caching Downloads
Concurrent Downloading
Dynamic Content
Interacting with Forms
Solving CAPTCHA
Scrapy
Index
Customer Reviews