This section is dedicated to introducing you to the ideas and tools that have been around during several decades of linguistics. The most traditional way to introduce this is to take an idea, talk about it at length, and then put all of this together.
Here, I am going to do this the other way around. We will solve two problems and, in the process, look at the tools we will be using. Instead of talking to you about a number 8 spanner, I am giving you a car engine and the tools, and I will introduce the tools as I use them.
Most NLP tasks are solved in a sequential pipeline, with the results from one component feeding into the next.
There is a wide variety of data structures that are used to store pipeline results and intermediate steps. Here, for simplicity, I am going to use only the data structures that are already in spaCy and the native Python ones like...