Intermediate
NLP Dataset Creation
Splits, labels and language tags that another team can trust.
Language tags
Say the language actually in the text. Nigerian Pidgin is not English and not Hausa. Mixed lines need a rule: primary language, or a mixed tag.
Language tags
Say the language actually in the text. Nigerian Pidgin is not English and not Hausa. Mixed lines need a rule: primary language, or a mixed tag.
Splits
Train, dev and test must not share near-duplicate sentences. Write that check into the card.
Splits
Train, dev and test must not share near-duplicate sentences. Write that check into the card.