Human-in-the-loop modeling

Despite more high-quality annotated data points being more valuable, the cost of annotating data, specifically when domain expertise is of necessity, could be very high. Active learning is a strategy that helps us in generating and labeling data to improve the performance of our models at a lower cost. In an active learning setting, we aim to benefit from a model with a limited amount of data and iteratively select new data points to be labeled, or their continuous value identified, with the aim of achieving higher performance (Wu et al., 2022; Ren et al., 2021; Burbidge et al., 2007). The model queries new instances to be annotated by experts or non-experts, or their labels or continuous values are identified via any computational or experimental technique. However, instead of the instances being selected randomly, there are techniques for new instance selection to help us in achieving better models with a lower number of instances and iterations (Table...