-
Book Overview & Buying
-
Table Of Contents
Hands-On Machine Learning with C++ - Second Edition
By :
This chapter introduced the Transformer architecture, a powerful model used in NLP and other fields of machine learning. We discussed the key components of the Transformer architecture, which include tokenization, embeddings, positional encoding, encoder, decoder, attention mechanisms, multi-head attention, cross-attention, residual connections, normalization layers, feedforward layers, and sampling techniques.
Finally, in the last part of this chapter, we developed an application so that we could perform a sentiment analysis of movie reviews. We applied the transfer learning technique to use the features learned by the pre-trained model in a new model designed for our specific task. We used the BERT model to produce a embedding representation of input texts and attached a linear layer classification head to classify review sentiments. We implemented a simple version of the tokenizer and dataset loader. We also developed the full training cycle of our classification head...