Profile Image Conjoint Eval

profile-image-conjoint-eval

Smiles in Profiles: Improving Efficiency While Reducing Disparities in Online Marketplaces — Athey et al. (2022) (arXiv:2209.01235, 2022)

What this evaluates

This protocol evaluates the causal impact of specific profile image features (smile, body-shot, and gender) on lender selection preferences in a simulated micro-lending marketplace. It uses a conjoint-style choice experiment with GAN-generated images to isolate how visual cues influence funding decisions independent of borrower creditworthiness.

Datasets

• Custom GAN-generated profile images — total 4920; splits: test (4920)

Metrics

• Average Treatment Effect (ATE) (primary) — range: percent
  • Estimated via logistic regression of a binary choice outcome on indicator variables for smile, body-shot, and male features. Reported as the average marginal effect (change in probability of being chosen) for each feature.

Input / output format

Input: Pairs of GAN-generated profile images, each manipulated to vary in smile, body-shot, and male features, presented to subjects alongside basic borrower context.

Output: Binary selection: subject chooses one profile from each pair.

Scoring recipe

# For each choice instance i:
# y_i = 1 if profile j is chosen, else 0
# X_i = [male_j, smile_j, body_shot_j, profile_fixed_effects]
# Fit logistic regression: P(y_i=1) = sigmoid(alpha*male + beta*smile + gamma*body_shot + mu)
# ATE = marginal effect of each feature on P(y_i=1) at sample means

Common pitfalls

• Subjects are recruited from Prolific and are not actual Kiva lenders, so preferences may not generalize to real platform users.
• Only 6 choice instances per subject may limit the precision of individual-level preference estimates.
• GAN-generated images, while realistic, may lack subtle cues present in real borrower photos.

Evidence (verbatim from paper)

Average treatment effects. Subjects were given a pair of profile pictures and asked which one they preferred. Suppose the preference has a systematic component, which depends on male, smile, and body-shot, and there is also an idiosyncratic random component ε_ij. The utility of subject i choosing the option j is written: u_ij = αmale_j + βsmile_j + γ*body-shot_j + μ_j + ε_ij. ... We find that all the features of interest are statistically significant and have high magnitudes: male and body-shot lead to lower outcomes, while smile leads to higher outcomes. ... The average marginal effect of male is a 31% reduction in the probability of being selected; there is a 17% drop for the body-shot, and a 34% increase for smile.

Citation

@misc{athey2022smiles,
  title={Smiles in Profiles: Improving Efficiency While Reducing Disparities in Online Marketplaces},
  author={Athey et al. (2022)},
  year={2022},
  note={arXiv:2209.01235}
}

• arXiv: 2209.01235