Summary

In this chapter, you learned about multilingual and cross-lingual language model pretraining and the difference between monolingual and multilingual pretraining. CLM and TLM were also covered, and you gained knowledge about them. You learned how it is possible to use cross-lingual models on various use cases, such as semantic search, plagiarism, and zero-shot text classification. You also learned how it is possible to train using a dataset from a language and test the model on a completely different language using cross-lingual models. Fine-tuning the performance of multilingual models was evaluated, and we concluded that some multilingual models could be a substitute for monolingual models, remarkably keeping performance loss to a minimum. We also took a look at the models that can translate at a massive scale; M2M100, for example, can translate in 9,900 directions in 100 languages, and we learned how to use this model.

In the next chapter, you will learn how to deploy...