-
Book Overview & Buying
-
Table Of Contents
Advanced Natural Language Processing with TensorFlow 2
By :
While building models for the SMS message spam detection thus far, only aggregate features based on counts or distributions of lexical or grammatical features have been considered. The actual words in the messages have not been used thus far. There are a couple of challenges in using the text content of messages. The first is that text can be of arbitrary lengths. Comparing this to image data, we know that each image has a fixed width and height. Even if the corpus of images has a mixture of sizes, images can be resized to a common size with minimal loss of information by using a variety of compression mechanisms. In NLP, this is a bigger problem compared to computer vision. A common approach to handle this is to truncate the text. We will see various ways to handle variable-length texts in various examples throughout the book.
The second issue is that of the representation of words with a numerical quantity or feature. In computer vision, the smallest unit is...
Change the font size
Change margin width
Change background colour