Now that we have developed some natural language query interfaces with natural language responses, we should address some techniques for evaluating whether the interfaces work.
First, we should note that developing good training examples for Rasa can be challenging due to the flexibility and ambiguity of natural language. For example, the question What classes should I take? might mean the student wants a schedule for the next semester or a schedule for their entire college career. Likewise, the question What's needed for MATH442? might mean the student wants to know just this course's prerequisites or the courses just this student needs to take before taking MATH442 or a complete multi-semester schedule that ends in the student taking MATH442.
Not only is language sometimes ambiguous, but it is typically far more varied than one expects. Chatbot developers might discover that after just a few minutes of interaction by users other than the developers themselves, a flurry...