Book Image

Transformers for Natural Language Processing - Second Edition

By : Denis Rothman
5 (1)
Book Image

Transformers for Natural Language Processing - Second Edition

5 (1)
By: Denis Rothman

Overview of this book

Transformers are...well...transforming the world of AI. There are many platforms and models out there, but which ones best suit your needs? Transformers for Natural Language Processing, 2nd Edition, guides you through the world of transformers, highlighting the strengths of different models and platforms, while teaching you the problem-solving skills you need to tackle model weaknesses. You'll use Hugging Face to pretrain a RoBERTa model from scratch, from building the dataset to defining the data collator to training the model. If you're looking to fine-tune a pretrained model, including GPT-3, then Transformers for Natural Language Processing, 2nd Edition, shows you how with step-by-step guides. The book investigates machine translations, speech-to-text, text-to-speech, question-answering, and many more NLP tasks. It provides techniques to solve hard language problems and may even help with fake news anxiety (read chapter 13 for more details). You'll see how cutting-edge platforms, such as OpenAI, have taken transformers beyond language into computer vision tasks and code creation using DALL-E 2, ChatGPT, and GPT-4. By the end of this book, you'll know how transformers work and how to implement them and resolve issues like an AI detective.
Table of Contents (25 chapters)
18
Other Books You May Enjoy
19
Index
Appendix I — Terminology of Transformer Models

Explainable AI (XAI)

You can add XAI to your programs if you are interested in implementing advanced prompt engineering with OpenAI’s state-of-the-art models that can explain outputs. ChatGPT can explain source code. It can also explain its own outputs to a certain extent.

We went through some of the main aspects of explainable AI in Chapter 14, Interpreting Black Box Transformer Models.

To go further, you can try using ChatGPT to explain ChatGPT outputs and other tools by running XAI_by_ChatGPT_for_ChatGPT.ipynb, which is in the Bonus directory of the GitHub repository of this book. The program runs a ChatGPT XAI analysis of a ChatGPT output and also shows how to explain outputs with SHAP.

The notebook is self-contained and can help you, the advanced reader, build XAI on top of the tools in this notebook.

Let’s add audio to our dialogue with ChatGPT.

Speech-to-text with Whisper

In this section, we will run a speech-to-text model...