- In the continuous bag-of-words (CBOW) model, we try to predict the target word given the context word, and in the skip-gram model, we try to predict the context word given the target word.
-
The loss function of the CBOW model is given as follows:
- When we have millions of words in the vocabulary, we need to perform numerous weight updates until we predict the correct target word. It is time-consuming and also not an efficient method. So, instead of doing this, we mark the correct target word as a positive class and sample a few words from the vocabulary and mark it as a negative class, and this is called negative sampling
-
PV-DM is similar to a continuous bag of words model, where we try to predict the target word given a context word. In PV-DM, along with word vectors, we introduce one more vector, called the paragraph vector. As the...
Hands-On Deep Learning Algorithms with Python
By :
Hands-On Deep Learning Algorithms with Python
By:
Overview of this book
Deep learning is one of the most popular domains in the AI space that allows you to develop multi-layered models of varying complexities.
This book introduces you to popular deep learning algorithms—from basic to advanced—and shows you how to implement them from scratch using TensorFlow. Throughout the book, you will gain insights into each algorithm, the mathematical principles involved, and how to implement it in the best possible manner. The book starts by explaining how you can build your own neural networks, followed by introducing you to TensorFlow, the powerful Python-based library for machine learning and deep learning. Moving on, you will get up to speed with gradient descent variants, such as NAG, AMSGrad, AdaDelta, Adam, and Nadam. The book will then provide you with insights into recurrent neural networks (RNNs) and LSTM and how to generate song lyrics with RNN. Next, you will master the math necessary to work with convolutional and capsule networks, widely used for image recognition tasks. You will also learn how machines understand the semantics of words and documents using CBOW, skip-gram, and PV-DM. Finally, you will explore GANs, including InfoGAN and LSGAN, and autoencoders, such as contractive autoencoders and VAE.
By the end of this book, you will be equipped with all the skills you need to implement deep learning in your own projects.
Related Content you might be interested in
Current Title:
Hands-On Deep Learning Algorithms with Python
Table of Contents (17 chapters)
Preface
Free Chapter
Section 1: Getting Started with Deep Learning
Introduction to Deep Learning
Getting to Know TensorFlow
Section 2: Fundamental Deep Learning Algorithms
Gradient Descent and Its Variants
Generating Song Lyrics Using RNN
Improvements to the RNN
Demystifying Convolutional Networks
Learning Text Representations
Section 3: Advanced Deep Learning Algorithms
Generating Images Using GANs
Learning More about GANs
Reconstructing Inputs Using Autoencoders
Exploring Few-Shot Learning Algorithms
Assessments
Other Books You May Enjoy
Customer Reviews