Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying The Regularization Cookbook
  • Table Of Contents Toc
The Regularization Cookbook

The Regularization Cookbook

By : Vincent Vandenbussche
4.3 (7)
close
close
The Regularization Cookbook

The Regularization Cookbook

4.3 (7)
By: Vincent Vandenbussche

Overview of this book

Regularization is an infallible way to produce accurate results with unseen data, however, applying regularization is challenging as it is available in multiple forms and applying the appropriate technique to every model is a must. The Regularization Cookbook provides you with the appropriate tools and methods to handle any case, with ready-to-use working codes as well as theoretical explanations. After an introduction to regularization and methods to diagnose when to use it, you’ll start implementing regularization techniques on linear models, such as linear and logistic regression, and tree-based models, such as random forest and gradient boosting. You’ll then be introduced to specific regularization methods based on data, high cardinality features, and imbalanced datasets. In the last five chapters, you’ll discover regularization for deep learning models. After reviewing general methods that apply to any type of neural network, you’ll dive into more NLP-specific methods for RNNs and transformers, as well as using BERT or GPT-3. By the end, you’ll explore regularization for computer vision, covering CNN specifics, along with the use of generative models such as stable diffusion and Dall-E. By the end of this book, you’ll be armed with different regularization techniques to apply to your ML and DL models.
Table of Contents (14 chapters)
close
close

Regularization using a word2vec embedding

In this recipe, we will use a pre-trained word2vec embedding to improve the results of a task thanks to transfer learning. We will compare the results to the initial Training a GRU recipe from Chapter 8, on the IMDb dataset for review classification.

Getting ready

A word2vec is a rather old type of word embedding in the NLP landscape and has been widely used in many NLP tasks. While recent techniques are sometimes more powerful, the word2vec approach remains efficient and cost-effective.

Without getting into the details of word2vec, a commonly used model is a 300-dimensional embedding; each word in the vocabulary is embedded into a vector of 300 values.

word2vec is usually trained on a large corpus of texts. There are two main approaches for training a word2vec that can be roughly described as follows:

  • Continuous bag of words (CBOW): Uses the context of surrounding words in a sentence to predict a missing word
  • skip-gram...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
The Regularization Cookbook
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon