Training a custom GPT-2 language model
We will continue our top-to-bottom approach in this section by exploring an example with a GPT-2 custom model that we will train on a specific dataset. The goal remains to determine the level of abstract reasoning a GPT model can attain.
This section describes the interaction with a GPT-2 model for text completion trained on a specific dataset. We will focus on Step 12 of the Training_OpenAI_GPT_2.ipynb notebook described in detail in Appendix IV, Custom Text Completion with GPT-2.
You can read this section first to see how an example with a custom GPT-2 model will improve responses. Then read Appendix IV, Custom Text Completion with GPT-2, to understand how to train a GPT-2 to obtain specific responses.
You can also decide to read Appendix IV directly, which also contains the interaction of Step 12 described below.
First, let’s understand how the interaction with GPT-2 improved by training it.