Designing interactive learning systems and workflows
The effectiveness of a human-in-the-loop system depends heavily on how well the labeling interface and workflow are designed. Even with advanced active ML algorithms selecting the most useful data points, poor interface design can cripple the labeling process. Without intuitive controls, informative queries, and efficient workflows adapted to humans, annotation quality and speed will suffer.
In this section, we will cover best practices for optimizing the human experience when interacting with active ML systems. Following these guidelines will enable you to create intuitive labeling pipelines, minimize ambiguity, and streamline the labeling process as much as possible. We will also discuss strategies for integrating active ML queries, collecting labeler feedback, and combining expert and crowd labelers. By focusing on human-centered design, you can develop interactive systems that maximize the utility of human input for your models...
