Mmii Medical Sonification Eval

mmii-medical-sonification-eval

A Framework for Multimodal Medical Image Interaction — Schütz et al. (2024) (arXiv:2407.07015, 2024)

What this evaluates

Evaluates whether physically informed audiovisual feedback improves spatial perception and task performance in medical imaging. Specifically, it measures how well users learn auditory-visual anatomical mappings and their accuracy in localizing brain tumors within a VR environment compared to unimodal baselines.

Datasets

• Medical imaging volumes (unspecified) — total ?; splits: test (-1)

Metrics

• accuracy (primary) — range: percent
  • Proportion of correctly localized brain tumors in the VR environment, measured against ground-truth surgical annotations or radiologist labels.
• usability — range: other
  • Aggregate score from post-task questionnaires assessing perceived ease of use, cognitive load, and system acceptability during multimodal interaction.

Input / output format

Input: 3D medical image volumes rendered in VR, paired with real-time physically modeled auditory feedback that dynamically correlates with tissue properties and spatial position.

Output: User selections via VR controllers indicating tumor location or anatomical correspondence, followed by Likert-scale questionnaire responses.

Scoring recipe

def calc_accuracy(predictions, ground_truth):
    correct = 0
    for pred, gt in zip(predictions, ground_truth):
        if distance(pred.center, gt.center) < spatial_threshold:
            correct += 1
    return (correct / len(predictions)) * 100

def calc_usability(questionnaire_responses):
    return sum(questionnaire_responses) / len(questionnaire_responses)

Common pitfalls

• Audiovisual correspondence is learned rather than innate, requiring careful baseline comparison with unimodal controls to isolate framework benefits.
• VR hardware variability and individual motion sensitivity can confound usability metrics if not standardized across participants.
• Small clinical sample sizes typical in medical VR studies may limit statistical power despite reported significance levels.

Evidence (verbatim from paper)

Study 2 focused on the usability and accuracy of the framework for a medical localization task, more precisely, brain tumor localization. The two studies sought to answer the following research questions: - Can users learn the audiovisual correspondence of the auditory and visual representation of anatomy? (Study 1) - Is physical modeling synthesis a suitable sonification approach to create distinguishable auditory representations of anatomy? (Study 1) - Can audiovisual interaction improve the usability and accuracy of a medical localization task? (Study 2)

Citation

@misc{schutz2024mmii,
  title={A Framework for Multimodal Medical Image Interaction},
  author={Schütz et al. (2024)},
  year={2024},
  note={arXiv:2407.07015}
}

• arXiv: 2407.07015