# Sib200 Xlt Eval

> This evaluation probes a model's ability to perform zero-shot and fully-supervised cross-lingual text classification across typologically diverse languages. It specifically measures how well parameter-efficient soft prompt tuning methods transfer knowledge from high-resource source languages to low-performing or unseen target languages without language-specific fine-tuning. Use when the user wants to benchmark on SIB-200, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/sib200-xlt-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/sib200-xlt-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/sib200-xlt-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/sib200-xlt-eval

---


# sib200-xlt-eval

> Cross-Prompt Encoder for Low-Performing Languages — Mikaberidze et al. (2025) (arXiv:2508.10352, 2025)

## What this evaluates

This evaluation probes a model's ability to perform zero-shot and fully-supervised cross-lingual text classification across typologically diverse languages. It specifically measures how well parameter-efficient soft prompt tuning methods transfer knowledge from high-resource source languages to low-performing or unseen target languages without language-specific fine-tuning.

## Datasets

- **SIB-200** — total ?; splits: train (-1), test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - The proportion of correctly predicted class labels out of the total number of instances. Calculated as (number of correct predictions) / (total number of predictions).

## Input / output format

**Input**: Raw text samples in one of ~200 target languages, processed by a frozen XLM-R large encoder with a lightweight classification head.

**Output**: A single predicted topic class label per text instance.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)
```

## Common pitfalls

- Confusing zero-shot cross-lingual transfer (ZS-XLT), which involves training on source languages and evaluating on targets, with pure zero-shot prompting of LLMs that receive no task-specific tuning.
- Using shared or language-specific verbalizers instead of a lightweight classification head, which introduces cross-lingual bias or fails for languages outside the tokenizer's vocabulary.
- Aggregating performance across language groups (Seen, Unseen, Low-Performing) without accounting for their strict subset relationships, which can mask transfer dynamics.

## Evidence (verbatim from paper)

> We evaluate on the SIB-200 benchmark, a multilingual topic classification dataset covering 200 typologically diverse languages. ... Table 1: ZS-XLT performance (accuracy) across different target groups.

## Citation

```bibtex
@misc{mikaberidze2025crossprompt,
  title={Cross-Prompt Encoder for Low-Performing Languages},
  author={Mikaberidze et al. (2025)},
  year={2025},
  note={arXiv:2508.10352}
}
```

- arXiv: 2508.10352