# Cesped Pose Estimation Eval

> This benchmark evaluates supervised deep learning models for predicting 3D particle orientations (rotation matrices) from 2D Cryo-EM micrographs. It assesses both angular prediction accuracy and the downstream quality of 3D structural reconstructions derived from the predicted poses. Use when the user wants to benchmark on CESPED, or asks about evaluating this task. Reports MAnE.

- Skill: `qhjqhj00/cesped-pose-estimation-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/cesped-pose-estimation-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/cesped-pose-estimation-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/cesped-pose-estimation-eval

---


# cesped-pose-estimation-eval

> CESPED: a new benchmark for supervised particle pose estimation in Cryo-EM — Sanchez-Garcia et al. (2023) (arXiv:2311.06194, 2023)

## What this evaluates

This benchmark evaluates supervised deep learning models for predicting 3D particle orientations (rotation matrices) from 2D Cryo-EM micrographs. It assesses both angular prediction accuracy and the downstream quality of 3D structural reconstructions derived from the predicted poses.

## Datasets

- **CESPED** — total ?; splits: half_0 (-1), half_1 (-1); repo https://github.com/rsanchezgarc/cesped

## Metrics

- `MAnE` **(primary)** — range: degrees
  - Mean Angular Error: average of the minimum angular distance between predicted and ground-truth rotation matrices over a symmetry group G. Formula: MAnE = (1/N) ∑ min_{g_j∈G} arccos((trace(g_j·trueR_i·predR_i^T)-1)/2).
- `wMAnE` — range: degrees
  - Confidence-weighted mean angular error. Weights each angular error by the ground-truth pose confidence (Relion's rlnMaxValueProbDistribution). Used for hyperparameter tuning.
- `PCC` — range: [0, 1]
  - Real space Pearson’s Correlation Coefficient between ground-truth and predicted 3D volumes. Measures voxel-wise linear correlation.
- `FSCR_0.143` — range: Å (resolution)
  - Fourier Shell Correlation Resolution at threshold 0.143. The highest frequency shell where FSC drops below 0.143, indicating resolution where maps agree with SNR=0.5.

## Input / output format

**Input**: 2D Cryo-EM particle images (downsampled to 1.5 Å/pixel, background-normalized to mean 0/std 1, phase-flipped for defocus, and cropped to exclude neighboring particles).

**Output**: Probability distribution over a discretised grid of rotation matrices (SO(3)_grid), optionally expanded for point symmetry groups.

## Scoring recipe

```python
# Calculate MAnE
ang_errors = []
for true_R, pred_R in zip(true_rotations, pred_rotations):
    min_err = min(arccos((trace(g * true_R * pred_R.T) - 1) / 2) for g in symmetry_group)
    ang_errors.append(min_err)
mane = sum(ang_errors) / len(ang_errors)

# Calculate PCC
pcc = pearsonr(volume_gt.flatten(), volume_pred.flatten())

# Calculate FSCR_0.143
fsc_curve = fourier_shell_correlation(volume_gt, volume_pred)
fscr = find_first_index_below_threshold(fsc_curve, 0.143)
```

## Common pitfalls

- Using ground-truth translations during volume reconstruction overestimates performance, as translation accuracy is tightly coupled with angular estimation.
- Symmetry expansion in labels must be handled during evaluation; failing to account for the symmetry group G will incorrectly penalize valid symmetric predictions.
- Ground-truth pose uncertainty is inherent in Cryo-EM; standard MAnE does not account for this, which is why wMAnE and ΔPCC/ΔFSC are recommended for robust benchmarking.

## Evidence (verbatim from paper)

> The most employed metric in pose estimation is the mean angular error, MAnE=∑ angError_i where angError_i=min_{g_j∈G} arccos((trace(g_j·trueR_i·predR_i^T)-1)/2) with G the set of rotation matrices given a point symmetry group, and trueR_i and predR_i the ground-truth and predicted rotation matrices. The quality of the volumes reconstructed from the predicted poses is assessed by comparing them against the ground-truth volumes generated from the original poses. As comparison metrics we use the real space Pearson’s Correlation Coefficient, PCC(X,Y)=... and the Fourier Shell Correlation Resolution at threshold t, FSCR_t(X,Y)...

## Citation

```bibtex
@misc{sanchezgarcia2023cesped,
  title={CESPED: a new benchmark for supervised particle pose estimation in Cryo-EM},
  author={Sanchez-Garcia et al. (2023)},
  year={2023},
  note={arXiv:2311.06194}
}
```

- arXiv: 2311.06194