Transformers 2.0: NLP library with deep interoperability between TensorFlow 2.0 and PyTorch, and 32+ pretrained models in 100+ languages

Last week, Hugging Face, a startup specializing in natural language processing, released a landmark update to their popular Transformers library, offering unprecedented compatibility between two major deep learning frameworks, PyTorch and TensorFlow 2.0.

Transformers (formerly known as pytorch-transformers and pytorch-pretrained-bert) provides general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between TensorFlow 2.0 and PyTorch.

Transformers 2.0 embraces the ‘best of both worlds’, combining PyTorch’s ease of use with TensorFlow’s production-grade ecosystem. The new library makes it easier for scientists and practitioners to select different frameworks for the training, evaluation and production phases of developing the same language model.

“This is a lot deeper than what people usually think when they talk about compatibility,” said Thomas Wolf, who leads Hugging Face’s data science team. “It’s not only about being able to use the library separately in PyTorch and TensorFlow. We’re talking about being able to seamlessly move from one framework to the other dynamically during the life of the model.”

https://twitter.com/Thom_Wolf/status/1177193003678601216

“It’s the number one feature that companies asked for since the launch of the library last year,” said Clement Delangue, CEO of Hugging Face.

Notable features in Transformers 2.0

8 architectures with over 30 pretrained models, in more than 100 languages

Load a model and pre-process a dataset in less than 10 lines of code

Train a state-of-the-art language model in a single line with the tf.keras fit function

Share pretrained models, reducing compute costs and carbon footprint

Deep interoperability between TensorFlow 2.0 and PyTorch models

Move a single model between TF2.0/PyTorch frameworks at will

Seamlessly pick the right framework for training, evaluation, production

As powerful and concise as Keras

About Hugging Face Transformers

With half a million installs since January 2019, Transformers is the most popular open-source NLP library. More than 1,000 companies including Bing, Apple or Stitchfix are using it in production for text classification, question-answering, intent detection, text generation or conversational. Hugging Face, the creators of Transformers, have raised US$5M so far from investors in companies like Betaworks, Salesforce, Amazon and Apple.

On Hacker News, users are appreciating the company and how Transformers has become the most important library in NLP.