Protocol Element Extraction

Extract evaluation protocol parameters from papers

yogsoth-ai Updated 512 repo stars

File contents

Protocol Element Extraction SOP

Extract all evaluation protocol parameters from a paper's description of how it evaluated on a specific benchmark. Captures the implementation details that affect reproducibility.

Input

  • paper_content: Full text or relevant sections of the paper
  • benchmark_name: The specific benchmark whose evaluation protocol to extract

Procedure

  1. Locate evaluation/experimental setup sections
  2. Extract all protocol parameters across 5 categories
  3. Flag parameters that are ambiguous or unstated
  4. Compare against original benchmark protocol (if known)
  5. Note any deviations or innovations in protocol

Output

Structured protocol specification with completeness assessment.

yogsoth-ai/knowledge-acquisition commit 5a6e36bda9

Frequently asked questions

npx get-skill add yogsoth-ai/protocol-element-extraction-2