Generalised Active Learning With Annotation Quality Selection

• In this paper we promote a general formulation of active learning (AL), wherein the typically binary decision to annotate a point or not is extended to selecting the qualities with which the points should be annotated. By linking the annotation quality to the cost of acquiring the label, we can trade a lower quality for a larger set of training samples, which may improve learning for the same annotation cost. To investigate this AL formulation, we introduce a concrete criterion, based on the mutual information (MI) between model parameters and noisy labels, for selecting annotation qualities for the entire dataset, before any labels are acquired. We illustrate the usefulness of our formulation with examples for both classification and regression and find that MI is a good candidate for a criterion, but its complexity limits its usefulness.

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

NATURVETENSKAP  -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Probability Theory and Statistics (hsv//eng)

Nyckelord

noisy labels

mutual information

Active learning

Publikations- och innehållstyp

kon (ämneskategori)

ref (ämneskategori)