Compiling models for GPU devices with Training Compiler
SageMaker Training Compiler is a capability that allows you to automatically optimize NLP DL models to run on GPU instances. For supported model architectures and frameworks, no code changes are required in your training scripts. You will only need to enable Training Compiler in your SageMaker training job configuration. Training Compiler can both reduce training speed time and memory requirements without this having any impact on model accuracy. For instance, according to AWS benchmarks, the training time and cost for the RoBERTa-based model are reduced by 30% when using Training Compiler.
Let’s review how SageMaker Training Compiler works under the hood and how to use it in training jobs.
Introducing the XLA optimization library
Accelerated Linear Algebra (XLA) is a domain-specific compiler that accelerates model training and execution with little to no changes in model code. At the time of writing, XLA is supported...