Pretrain Vision and Large Language Models in Python
By :
Pretrain Vision and Large Language Models in Python
By:
Overview of this book
Foundation models have forever changed machine learning. From BERT to ChatGPT, CLIP to Stable Diffusion, when billions of parameters are combined with large datasets and hundreds to thousands of GPUs, the result is nothing short of record-breaking. The recommendations, advice, and code samples in this book will help you pretrain and fine-tune your own foundation models from scratch on AWS and Amazon SageMaker, while applying them to hundreds of use cases across your organization.
With advice from seasoned AWS and machine learning expert Emily Webber, this book helps you learn everything you need to go from project ideation to dataset preparation, training, evaluation, and deployment for large language, vision, and multimodal models. With step-by-step explanations of essential concepts and practical examples, you’ll go from mastering the concept of pretraining to preparing your dataset and model, configuring your environment, training, fine-tuning, evaluating, deploying, and optimizing your foundation models.
You will learn how to apply the scaling laws to distributing your model and dataset over multiple GPUs, remove bias, achieve high throughput, and build deployment pipelines.
By the end of this book, you’ll be well equipped to embark on your own project to pretrain and fine-tune the foundation models of the future.
Table of Contents (23 chapters)
Preface
Part 1: Before Pretraining
Free Chapter
Chapter 1: An Introduction to Pretraining Foundation Models
Chapter 2: Dataset Preparation: Part One
Chapter 3: Model Preparation
Part 2: Configure Your Environment
Chapter 4: Containers and Accelerators on the Cloud
Chapter 5: Distribution Fundamentals
Chapter 6: Dataset Preparation: Part Two, the Data Loader
Part 3: Train Your Model
Chapter 7: Finding the Right Hyperparameters
Chapter 8: Large-Scale Training on SageMaker
Chapter 9: Advanced Training Concepts
Part 4: Evaluate Your Model
Chapter 10: Fine-Tuning and Evaluating
Chapter 11: Detecting, Mitigating, and Monitoring Bias
Chapter 12: How to Deploy Your Model
Part 5: Deploy Your Model
Chapter 13: Prompt Engineering
Chapter 14: MLOps for Vision and Language
Chapter 15: Future Trends in Pretraining Foundation Models
Index
Customer Reviews