In this chapter, we have learned how to write a scraping script using the rvest library. First, we worked on how to collect URLs, and then we worked on how to create XPath rules. After that, we wrote our first web scraping script using the rvest library. We have applied basic text and manipulation techniques. Once we had the final data ready, we calculated the daily average comment counts, and at the final step, we created a line plot using basic R plots and plotly plots to visualize the average comment count, article counts for each categories, and the summary statistics of the comment counts.
In the next chapter, we will talk about the fundamentals of Selenium and how to do web scraping using the RSelenium library. At the end of the next chapter, we will be writing R script to collect data from the web using Selenium.