Raw data-based pipelines

Creating a full pipeline can be a daunting task and requires creating customized tools for all models and all kinds of data. It allows us to optimize how we use the models, but it requires a lot of effort. The main rationale behind pipelines is that they link two areas of ML – the model and its computational capabilities with the task and the data from the domain. Luckily for us, the main model hubs such as HuggingFace have an API that provides ML pipelines automatically. Pipelines in HuggingFace are related to the model and provided by the framework based on the model’s architecture, input, and output.

Pipelines for NLP-related tasks

Text classification is a pipeline designed to classify text input into predefined categories or classes. It’s particularly useful for tasks such as sentiment analysis (SA), topic categorization, spam detection, intent recognition, and so on. The pipeline typically employs pre-trained models fine-tuned...