Shirt Pose Tracking Eval

shirt-pose-tracking-eval

Adaptive Neural Network-based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft — Park et al. (2022) (arXiv:2206.03796, 2022)

What this evaluates

Evaluates the accuracy and robustness of a neural network-integrated Unscented Kalman Filter for monocular pose tracking of tumbling noncooperative spacecraft. It probes the system's ability to maintain steady-state position and orientation accuracy under domain gaps between synthetic training data and real hardware-in-the-loop test images.

Datasets

• SHIRT — total ?; splits: test (-1)

Metrics

• e_pose (primary) — range: other
  • Normalized pose error computed as e_t / ||t|| + e_q, where e_t is translation error, ||t|| is the ground-truth translation norm, and e_q is rotation error in radians. Mean pose error E_pose is the average over N images.

Input / output format

Input: Sequential monocular images of a target spacecraft from a servicer's camera, initial pose predictions from SPNv2, and absolute orbital/attitude states of the servicer.

Output: Time-series estimates of relative translation, relative velocity, and relative attitude/angular velocity states.

Scoring recipe

def compute_mean_pose_error(preds, gts):
    total = 0.0
    for p, g in zip(preds, gts):
        e_t = np.linalg.norm(p.trans - g.trans)
        e_q = rotation_error_rad(p.rot, g.rot)
        norm_t = np.linalg.norm(g.trans)
        total += (e_t / norm_t) + e_q
    return total / len(preds)

Common pitfalls

• Rotation error e_q must be in radians, not degrees or quaternion distance.
• Translation error must be normalized by ground-truth distance ||t||; failing to do so biases evaluation toward close-range encounters.
• Initial process noise covariance Q_o requires manual tuning before ASNC activates; poor initialization causes filter divergence regardless of adaptive updates.

Evidence (verbatim from paper)

For individual images, filter performance is evaluated using the translation error $(e_{\mathrm{t}})$ and rotation error $(e_{\mathrm{q}})$ defined in Eq. 1. Additionally, the pose error from SPEC2021 [16] is used as a singular metric wherever applicable and is given as $$e _ {\text {p o s e}} = e _ {\mathrm {t}} / | \boldsymbol {t} | + e _ {\mathrm {q}}$$ where $t$ is the ground-truth translation vector of the sample, and $e_{\mathrm{q}}$ is in radians. For batches of images, the mean translation, rotation, and pose errors are reported, respectively defined as $$E _ {\mathrm {t}} = \sum _ {i = 1 } ^ {N} e _ {\mathrm {t}} ^ {(i)}$$ $$E _ {\mathrm {q}} = \sum _ {i = 1 } ^ {N} e _ {\mathrm {q}} ^ {(i)}$$ $$E _ {\text {p o s e}} = \sum _ {i = 1 } ^ {N} e _ {\text {p o s e}} ^ {(i)}$$

Citation

@misc{park2022adaptive,
  title={Adaptive Neural Network-based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft},
  author={Park et al. (2022)},
  year={2022},
  note={arXiv:2206.03796}
}

• arXiv: 2206.03796