Bangla Key2text Eval

bangla-key2text-eval

Bangla Key2Text: Text Generation from Keywords for a Low Resource Language — Talukder et al. (2026) (arXiv:2604.19508, 2026)

What this evaluates

Evaluates a model's ability to generate coherent, faithful Bangla text conditioned on a set of extracted keywords, testing sequence-to-sequence generation capabilities in a low-resource language setting.

Datasets

• Bangla Key2Text — total 2600000; splits: train (-1), val (-1), test (-1); repo https://github.com/TonmoyTalukder/Bangla-Key2Text

Metrics

• generation_quality (primary) — range: qualitative
  • Not explicitly defined in the provided section; assessed via keyword-text alignment, fluency, and faithfulness during inference.

Input / output format

Input: A list of keywords extracted from a source text using a pre-trained BERT model.

Output: A generated Bangla text sequence, truncated to a maximum of 64 tokens.

Scoring recipe

def evaluate(keywords, gold_text, model):
    preds = model.generate(
        input_ids=keywords,
        max_length=64,
        num_beams=2,
        top_k=50,
        top_p=0.95,
        repetition_penalty=2.5,
        length_penalty=1.0
    )
    # Coherence and faithfulness scoring (human or automated)
    score = compute_alignment_and_fluency(preds, gold_text)
    return score

Common pitfalls

• Input sequences are truncated to 512 tokens, which may discard relevant context for keyword extraction.
• Decoding hyperparameters (beam size, top-k, top-p, repetition penalty) vary across experiments and must be fixed for reproducible comparison.
• Keyword extraction uses default BERT parameters without fine-tuning, potentially introducing noise into the conditioning signal.

Evidence (verbatim from paper)

During inference, we experimented with different decoding techniques i.e. greedy decoding, beam search with beam size 2, top-k sampling with k=50 and top-p sampling with p=0.95. The repetition penalty and length penalty were set to 2.5 and α=1.0 respectively. For 4-bit quantized LLMs, we utilized model-specific default text generation parameters during inference.

Citation

@misc{talukder2026banglakey2text,
  title={Bangla Key2Text: Text Generation from Keywords for a Low Resource Language},
  author={Talukder et al. (2026)},
  year={2026},
  note={arXiv:2604.19508}
}

• arXiv: 2604.19508