Weighted Autofluorescence Subtraction

Multi-Range Weighted Autofluorescence Subtraction

Experiment Overview

Algorithm

1. Clip blank channel (same as global method)
2. Segment signal histogram into N intensity ranges (default 5: background, very_dim, dim, medium, bright)
3. For each range, compute signal-to-AF ratio → per-range weight:
   • blank_mean / signal_mean > 1.5 → weight 0.3-0.5 (AF dominates, protect signal)
   • ratio 0.8-1.5 → weight 0.5-0.8 (mixed)
   • ratio 0.3-0.8 → weight 0.8-1.0 (signal moderate)
   • ratio < 0.3 → weight 1.0-1.15 (signal dominant)
   • Correlation within range boosts weight slightly (+0.05 if corr > 0.3)
4. Build smooth per-pixel weight map with cosine transitions at range boundaries
5. Apply: result = signal - min(signal, blank * base_scale * weight_map)
6. Post-processing (smoothing, erosion — same as global method)
7. Compute per-range quality metrics for learning

Key insight: In dim signal regions where blank > signal, the per-range weight drops to 0.3-0.5 (gentle subtraction), preserving real positive cells. In bright signal regions, weight rises to 1.0+ (aggressive AF removal).

Key Parameters

Failed Attempts (Critical)

What Worked

Core Algorithm

• Cosine transitions between ranges prevent visible discontinuities in subtracted image
• smooth_membership() function using (1 - cos(π*t))/2 for rising edges, (1 + cos(π*t))/2 for falling edges
• Weight map is float32 — pixel-local computation, no neighbor dependencies (trivially parallelizable with Dask map_blocks)
• Near-uniform fallback: When dynamic_range < 100, falls back to global method (avoids creating meaningless ranges)
• Sparse signal adaptation: When <1% nonzero pixels, reduces n_ranges to min(n_ranges, 3)

Test Design

• Fixture with realistic proportions: 30% dim signal (rows 0:77), 20% bright signal (rows 180:230), 50% background
• Dim signal values: 3000-6000 with bright AF: 7000-9000 in same region
• Behavioral verification: dim_global=0 (destroyed) vs dim_weighted=432 (preserved)
• Backwards compatibility: Uniform weights=1.0 produces max_diff=0.0 vs global method

Learning Integration

• algorithm_version column in parameter_learning.py via ALTER TABLE migration
• Bootstrap records with user_approved=False (lower weight in recommendations)
• Range aggregation: average per-range weights across matching records with same n_ranges

Architecture

• IntensityRange and WeightedSubtractionParameters dataclasses with to_dict()/from_dict() for serialization
• AutofluorescenceSubtractor.method parameter dispatches to _process_global() or _process_weighted()
• MCP analyze_weighted_subtraction() tool previews ranges without performing subtraction
• CLI kintsugi mcp pretrain scans EDF outputs for batch bootstrapping

Implementation Files

Data Structures

@dataclass
class IntensityRange:
    lower_bound: float       # Inclusive lower boundary
    upper_bound: float       # Exclusive upper boundary
    weight: float            # Scale multiplier (0.1-1.5) for blank in this range
    label: str = ""          # "background", "dim", "medium", "bright"

@dataclass
class WeightedSubtractionParameters:
    blank_clip_factor: int = 0
    base_scale_factor: float = 1.0
    n_ranges: int = 5
    range_method: str = "percentile"   # "percentile" | "otsu" | "manual"
    ranges: list[IntensityRange] = field(default_factory=list)
    transition_width: float = 0.1      # Fraction of range width for blending

Learning Database Schema

ALTER TABLE parameter_records ADD COLUMN algorithm_version TEXT DEFAULT 'v1';

Weighted records store algorithm_version="weighted_v1" with per-range weights in parameters_json. Aggregation averages per-range weights across matching records with same n_ranges.

References

• signal/autofluorescence.py — Core weighted subtraction implementation
• signal/bootstrap.py — Batch pretraining from existing processed data
• tests/test_signal_autofluorescence.py — Test classes for weighted subtraction