Eu Taxonomy Kpi Eval

eu-taxonomy-kpi-eval

Automated Analysis of Sustainability Reports: Using Large Language Models for the Extraction and Prediction of EU Taxonomy-Compliant KPIs — Schmoll et al. (2025) (arXiv:2512.24289, 2025)

What this evaluates

Evaluates LLMs on extracting and predicting EU Taxonomy-compliant Key Performance Indicators from corporate sustainability reports. It probes two capabilities: multi-label classification of economic activities against regulatory definitions, and zero-shot regression of financial KPI percentages (Turnover, CapEx, OpEx) from unstructured text.

Datasets

• EU Taxonomy Sustainability Reports Dataset — total 190; splits: test (-1)

Metrics

• multi-label text classification (primary) — range: 0-1
  • Not explicitly defined in the provided section; modeled as a multi-label text classification problem where the model predicts a set of relevant Taxonomy activities from operational descriptions.
• regression — range: percent
  • Not explicitly defined in the provided section; modeled as a regression problem where the model predicts the numerical percentage for Turnover, CapEx, and OpEx KPIs based on report text.

Input / output format

Input: Unstructured corporate sustainability reports (annual reports) containing operational descriptions and financial narrative text.

Output: Task 1: A predicted set of relevant EU Taxonomy economic activities. Task 2: Numerical percentage values for Turnover, CapEx, and OpEx KPIs.

Scoring recipe

# Task 1: Multi-label text classification
predicted_activities = model.predict(report_text)
gold_activities = gold_labels[report_id]
classification_score = compute_multilabel_f1(predicted_activities, gold_activities)

# Task 2: Regression
predicted_kpis = model.predict_kpis(report_text) # Turnover, CapEx, OpEx
gold_kpis = gold_kpi_values[report_id]
regression_error = mean_absolute_error(predicted_kpis, gold_kpis)

Common pitfalls

• Zero-shot setting means models are not fine-tuned on this dataset, making performance highly sensitive to prompt design and context window limits.
• The 'paradox of context' indicates that providing full unstructured reports may hurt performance compared to concise metadata, contrary to typical LLM expectations.
• Model confidence scores are poorly calibrated, so high confidence does not guarantee accurate KPI predictions.

Evidence (verbatim from paper)

We model this as a multi-label text classification problem, where the input is a company’s operational description (e.g., from an annual report) and the output is a predicted set of relevant Taxonomy activities. ... We model this as a regression problem, where the model must predict the numerical percentage for the Turnover, CapEx, and OpEx KPIs based on the textual content of a company’s report.

Citation

@misc{schmoll2025automated,
  title={Automated Analysis of Sustainability Reports: Using Large Language Models for the Extraction and Prediction of EU Taxonomy-Compliant KPIs},
  author={Schmoll et al. (2025)},
  year={2025},
  note={arXiv:2512.24289}
}

• arXiv: 2512.24289