Low Resource Ner Transfer Eval

low-resource-ner-transfer-eval

Enhancing Low Resource NER Using Assisting Language And Transfer Learning — Sabane et al. (2023) (arXiv:2306.06477, 2023)

What this evaluates

Evaluates cross-lingual transfer learning for Named Entity Recognition in low-resource Indian languages (Hindi and Marathi) by measuring how well models trained on combined or assisting-language datasets generalize to target language test sets compared to monolingual baselines.

Datasets

• IIT Bombay (Marathi) — total ?; splits: train (-1), test (-1)
• IJCNLP (Hindi) — total ?; splits: train (-1), test (-1)
• Wiki ANN (Hindi and Marathi) — total ?; splits: train (-1), test (-1)

Metrics

• scores (primary) — range: percent
  • Unspecified in the text; refers to standard NER performance metrics (typically F1-score or accuracy) used to compare monolingual vs. cross-lingual/mixed training setups.

Input / output format

Input: Tokenized text sequences with corresponding entity labels (e.g., BIO/IOB tagging) for NER.

Output: Predicted entity labels for each token in the input sequence.

Scoring recipe

def compute_ner_scores(preds, golds):
    pred_spans = extract_spans(preds)
    gold_spans = extract_spans(golds)
    tp = len(pred_spans & gold_spans)
    fp = len(pred_spans - gold_spans)
    fn = len(gold_spans - pred_spans)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return f1

Common pitfalls

• Blindly merging datasets from different domains (e.g., Wiki ANN vs. news/social media) without domain or tag-distribution alignment degrades performance.
• Assuming cross-lingual transfer always improves results; performance gains depend heavily on script similarity, lexical overlap, and data curation.

Evidence (verbatim from paper)

The observations portray that XLM Roberta, RoBERTa Hindi, MahaBERT [9], and Maha Roberta perform better on the mixed dataset than on the monolingual IIT Bombay dataset (Marathi). Similarly, when the mixed data model is tested on IJCNLP (Hindi), the scores observed are better for all the models except with mBERT where monolingual IJCNLP performs better.

Citation

@misc{sabane2023enhancing,
  title={Enhancing Low Resource NER Using Assisting Language And Transfer Learning},
  author={Sabane et al. (2023)},
  year={2023},
  note={arXiv:2306.06477}
}

• arXiv: 2306.06477