Amazon SageMaker Training Compiler and Neo
If you use Hugging Face language models today, such as BERT, GPT, RoBERTa, AlBERT, DistiliBERT, or hundreds of others, then you are in luck! Without much work, you can easily speed up the run-time of your jobs by up to 50%. This is because of SageMaker Training Compiler (SMTC). As we learned earlier, compilation generally has the potential to increase the speed of your training. With SMTC, we provide a managed compilation feature within SageMaker training to easily enable this for your own models and scripts.
As you can see in the visual provided, enabling this is quite simple. Here we use the Hugging Face AWS-managed deep learning container and simply add TrainingCompilerConfig(). If you’re using a model with the Hugging Face Trainer API, this will automatically trigger Training Compiler:
Figure 9.3 – Configure SageMaker Training Compiler
How does it work? SMTC uses a variety of compilation methods...
