# Esv Intervention Eval

> Evaluates the psychological and behavioral impact of an AI-generated emotional self-voice intervention compared to text-only and control conditions on goal-related resilience, confidence, motivation, and emotional states. Use when the user wants to benchmark on Custom Human-Subject Intervention Dataset, or asks about evaluating this task. Reports Self-report questionnaire scores.

- Skill: `qhjqhj00/esv-intervention-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/esv-intervention-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/esv-intervention-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/esv-intervention-eval

---


# esv-intervention-eval

> Leveraging AI-Generated Emotional Self-Voice to Nudge People towards their Ideal Selves — Fang et al. (2024) (arXiv:2409.11531, 2024)

## What this evaluates

Evaluates the psychological and behavioral impact of an AI-generated emotional self-voice intervention compared to text-only and control conditions on goal-related resilience, confidence, motivation, and emotional states.

## Datasets

- **Custom Human-Subject Intervention Dataset** — total ?; splits: pre-intervention (-1), post-intervention (-1)

## Metrics

- `VADER compound score` — range: [-1, 1]
  - Normalized, weighted composite score of sentiment polarity for free-text responses, ranging from -1 (negative) to 1 (positive).
- `Self-report questionnaire scores` **(primary)** — range: [1, 7]
  - Likert-scale ratings (1-7) measuring psychological constructs including valence, arousal, resilience, confidence, motivation, commitment, vividness, similarity, and positive affect.

## Input / output format

**Input**: Participants receive scenario prompts (goal failure or habit struggle) and are assigned to Control, Text, or ESV conditions. They complete pre-intervention and post-intervention questionnaires and provide free-text reflections.

**Output**: Numeric Likert-scale ratings for psychological outcomes and free-text responses for sentiment analysis.

## Scoring recipe

```python
# For continuous outcomes (valence, arousal, resilience):
lme_model = lmer(outcome ~ Test * Condition + covariates + (1|Subject), data=df)
p_val_lme = summary(lme_model).p_value

# For collinear outcomes (confidence, motivation, commitment):
df_art = aligned_rank_transform(df, by=['Test', 'Condition'])
anova_model = aov(outcome ~ Test * Condition, data=df_art)
p_val_anova = anova_model.p_value

# For text responses:
vader_score = vader_sentiment(text_response).compound
```

## Common pitfalls

- Ignoring repeated-measures structure: Standard ANOVA/t-tests on pre/post data violate independence assumptions; LME or ART-ANOVA is required.
- Overlooking collinearity: Pre/post and condition effects were perfectly separated for confidence/motivation/commitment, necessitating Aligned Rank Transform instead of standard LME.
- Misinterpreting VADER: VADER captures general sentiment polarity, not specific psychological constructs, so it should not be directly equated with questionnaire scores.

## Evidence (verbatim from paper)

> For the quantitative data, we used a linear mixed effects (LME) model (lme4 package in R [6]) to account for the repeated measurement nature of the data, namely each subject has 2 observations: pre-intervention and post-intervention. ... The main outcomes are valence, arousal, resilience, confidence, motivation, and commitment. ... We also used VADER [41] to quantitatively capture the sentiment in the free responses.

## Citation

```bibtex
@misc{fang2024leveraging,
  title={Leveraging AI-Generated Emotional Self-Voice to Nudge People towards their Ideal Selves},
  author={Fang et al. (2024)},
  year={2024},
  note={arXiv:2409.11531}
}
```

- arXiv: 2409.11531