Old Code Data Detection

Old-Code Data Detection

Experiment Overview

Context

KINTSUGI had several critical bugs fixed in Jan–Feb 2026 (deconvolution Tukey window, intensity scaling, BaSiC flatfield, EDF sigma, registration non-rigid params, Frangi Ra formula). Some projects had processed data that was bulk-copied from a local workstation where the old code ran, mixed in with data freshly processed on HiPerGator with the fixed code. No processing version metadata existed in sentinel files or manifests, so provenance had to be inferred from file timestamps.

Verified Workflow

Step 1: Establish Fix Dates

git log --oneline --format='%ai %s' --all | grep -iE '(tukey|apodiz|intensity.scal|flatfield|edf.*sigma|edf.*variance|fallback.copy|non.rigid|frangi)'

Critical fix dates:

Step 2: Timestamp Forensics — Detect Bulk Copies vs Real Processing

Key insight: Real HPC processing produces files over hours (decon: 22h for 624 files). Bulk copies (Globus/rsync) produce hundreds of files within seconds.

# Measure time span of all TIFs in a stage directory
find "$dir" -name "*.tif" -printf '%T@\n' | sort -n | \
  awk 'NR==1{first=$1} END{print NR " files, span = " $1-first " seconds"}'

Thresholds:

• < 120 seconds for 100+ files → bulk copy (provenance unknown)
• Minutes to hours → real HPC processing (timestamp = processing date)

Step 3: Cross-Reference Timestamps Against Fix Dates

For each project+stage combination:

1. If timestamp span indicates bulk copy → flag as unknown provenance (assume old code)
2. If real processing AND all files dated after the relevant fix → clean
3. If real processing AND some files before fix date → flag specific pre-fix files

Step 4: Check Re-Registration Status

Registration had a specific fix date (Feb 18). Cross-check registered projects against the known re-registration list:

# List projects validated with re-registration
# From memory: CX_19-001, CX_19-002_R1, CX_19-002_R3, CX_19-003_R1
# Any project NOT in this list with registered/ timestamps as bulk copies = old registration

Step 5: Generate Deletion Script

#!/usr/bin/env bash
# Pattern: dry-run first, then execute
delete_dir() {
    local dir="$1" reason="$2"
    if [[ -d "$dir" ]]; then
        local human=$(du -sh "$dir" | awk '{print $1}')
        if $DRY_RUN; then
            echo "  WOULD DELETE: $dir ($human) — $reason"
        else
            echo "  DELETING: $dir ($human) — $reason"
            rm -rf "$dir"
        fi
    fi
}

Failed Attempts (Critical)

Final Parameters

# Timestamp span thresholds
BULK_COPY_THRESHOLD=120  # seconds — 100+ files in <2 min = copy

# Fix dates (epoch seconds for comparison)
FIX_BASIC=$(date -d '2026-01-14' +%s)
FIX_DECON=$(date -d '2026-01-17' +%s)
FIX_EDF=$(date -d '2026-01-21' +%s)
FIX_REGISTRATION=$(date -d '2026-02-18' +%s)
FIX_FRANGI=$(date -d '2026-02-21' +%s)

# Stage-to-fix mapping
# stitched/     → FIX_BASIC
# deconvolved/  → FIX_DECON
# edf/          → FIX_EDF
# registered/   → FIX_REGISTRATION
# vessel_3d/    → FIX_FRANGI
# signal_isolated/ → depends on registered/ (if registered is old, signal isolation is invalid)

Key Insights

• 590 GB of old-code data found across 3 legacy projects (1904CC1-1L, 1901CC2A, 1901CC3C) — all bulk-copied from local workstation
• Timestamp span is the definitive signal: 469 files in 59 seconds = copy; 624 files in 22 hours = real processing
• Signal isolation built on old registered data is also invalid — cascading invalidation
• No version metadata exists in KINTSUGI sentinels — future improvement would be to add algorithm_version and git_commit fields to all sentinel files
• Legacy project names (1901CC2A, 1904CC1-1L) vs CX_ naming helped identify likely locally-processed datasets
• Mixed timestamps within registered/ indicate partial re-registration — check individual files, not just the directory

Results

Total deleted: 590 GB

References

• KINTSUGI CLAUDE.md Critical Fixes table — lists all bug fixes with dates
• scripts/reprocess_striped_zplanes.py — targeted reprocessing for stripe artifacts
• reregister_all.sh — batch re-registration with tuned non-rigid parameters
• multi-project-batch-isolation skill — multi-project signal isolation after clean reprocessing