Now that we have our final model, we need to validate its results by testing it with unseen data. This will give us the confidence that our model is well trained and will probably produce similar results where new data is handed to use.
A careful reader should have noticed that we used the TF-IDF data frame when creating our sentiment analysis data, and not any of the ones we create later with combinations of bigrams, SVDs, and cosine similarities, which operate in a different semantic space due to the fact they are transformations of the original DFM. Therefore, before we can actually use our trained model to make predictions on the test data, we need to transform it into an equivalent space as our training data. Otherwise, we would be comparing apples and oranges, which would give us nonsense results.
To make sure that we're...