Beginner
Data Annotation
Labels need a guide, examples and a way to measure agreement.
Guides beat instinct
Write the label names, one positive example and one near-miss. If two workers can read the guide and disagree, the guide is unfinished.
Agreement
Hidden gold items and a second rater tell you whether the label is stable. A single fast click is not a measurement.
Agreement
Hidden gold items and a second rater tell you whether the label is stable. A single fast click is not a measurement.