# Tabular Data Field Comparison > Use when you have multiple independent implementations of the same data format reader (e.g., Rust, Python, R versions) and need to verify they produce identical or equivalent output. - Skill: `holobiomicslab/tabular-data-field-comparison` (Agent Skill) - Install (CLI): `npx skillmds@latest add holobiomicslab/tabular-data-field-comparison` - Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/tabular-data-field-comparison/raw - Safety review: PASS (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - License: CC-BY-4.0 - Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/holobiomicslab/tabular-data-field-comparison --- # Tabular Data Field Comparison ## Summary Compare structured tabular outputs (CSV, Parquet) exported from independent implementations of the same file reader to detect field-level discrepancies in names, types, row counts, and numerical values. This skill validates implementation interoperability and identifies divergences in data interpretation across language/library variants. ## When to use Apply this skill when you have multiple independent implementations of the same data format reader (e.g., Rust, Python, R versions) and need to verify they produce identical or equivalent output. Use it to validate that a newly written reader implementation (in one language) produces the same spectrum table structure and numerical content as an existing trusted implementation. ## When NOT to use - Input is a single implementation or vendor output; comparison requires at least two independent readers. - Field definitions or data types are intentionally different across implementations by design (e.g., different precision levels for intentional lossy compression). - Input file format is already validated or certified by a reference standard; this skill is for early-stage interoperability testing, not final certification. ## Inputs - mzPeak file (ZIP archive containing Parquet files) - Three or more independent reader implementations (Rust library, Python pyarrow implementation, R arrow implementation) ## Outputs - CSV export of spectrum metadata table from each implementation - CSV export of spectrum signal data table from each implementation - Consistency report documenting field-level agreement and discrepancies - Validation status summary (pass/fail per implementation pair) ## How to apply Load the same input file (e.g., an mzPeak archive) sequentially using each independent implementation—Rust, Python/pyarrow, and R/arrow—and export the resulting spectrum metadata or signal tables to CSV format for easy textual comparison. Compare the three CSV outputs systematically: first check schema agreement (field names, declared data types, row counts match), then validate numerical equality field-by-field, paying attention to floating-point precision and integer representation. Generate a structured consistency report documenting which fields agree, which diverge, and whether divergences are due to implementation bugs, format interpretation differences, or acceptable numerical tolerances. Use field-level agreement as the primary validation criterion, since row count and schema matching alone are insufficient to confirm correctness. ## Related tools - **Rust mzPeak library** (Primary read/write implementation used to load test file and export reference spectrum table) — https://github.com/HUPO-PSI/mzPeak - **pyarrow** (Python Parquet reader for loading mzPeak table files and exporting to CSV) — https://arrow.apache.org/docs/python/index.html - **R arrow** (R Parquet reader for loading mzPeak table files and exporting to CSV) — https://arrow.apache.org/docs/r/ - **OpenMS** (Reference mass spectrometry toolkit; mzPeak name held in trust by OpenMS Inc.) ## Evaluation signals - Field names match exactly across all three CSV exports (including order and case sensitivity). - Data types inferred from CSV content (integer, float, string, null) agree across implementations for each column. - Row counts are identical across all three CSV outputs. - Numerical values in floating-point columns agree within an acceptable precision tolerance (e.g., within machine epsilon or a domain-specific threshold like 1e-6 relative error). - No spurious nulls or NaN values appear in one implementation but not others (unless intentional per format design, such as null marking for zero-run-stripped regions). ## Limitations - The mzPeak format is work-in-progress with no stability guarantee at this point; field definitions or semantics may change. - Python and R implementations support reading only; if testing write capability, only Rust can produce reference outputs. - Floating-point precision differences (e.g., f32 vs. f64 representation) may introduce small numerical divergences that are valid but require tolerance tuning. - The skill does not validate semantic correctness (e.g., whether a m/z value makes physical sense); it only checks consistency across implementations. ## Evidence - [other] Do the Rust, Python/pyarrow, and R/arrow implementations of mzPeak file readers produce field-level agreement when loading the same input file?: "Do the Rust, Python/pyarrow, and R/arrow implementations of mzPeak file readers produce field-level agreement when loading the same input file?" - [other] Compare the three CSV outputs field-by-field to identify any discrepancies in field names, data types, row counts, and numerical values.: "Compare the three CSV outputs field-by-field to identify any discrepancies in field names, data types, row counts, and numerical values." - [other] Three independent mzPeak reader implementations exist: a Rust library, a Python implementation using pyarrow, and an R implementation using arrow, all capable of reading mzPeak files.: "Three independent mzPeak reader implementations exist: a Rust library, a Python implementation using pyarrow, and an R implementation using arrow, all capable of reading mzPeak files." - [readme] The primary work shown here is written in Rust at the repository root, including a library for reading and writing mzPeak files, as well as command line tools for converting existing formats into mzPeak.: "The primary work shown here is written in Rust at the repository root, including a library for reading and writing mzPeak files, as well as command line tools for converting existing formats into" - [readme] There is a separate Python implementation in `python/` which is a complete re-implementation for _reading_ mzPeak files using [`pyarrow`](https://arrow.apache.org/docs/python/index.html), and the PyData stack.: "There is a separate Python implementation in `python/` which is a complete re-implementation for _reading_ mzPeak files using [`pyarrow`]"