Geometric Matrix Completion Eval

geometric-matrix-completion-eval

Geometric Matrix Completion via Sylvester Multi-Graph Neural Network — Du et al. (2022) (arXiv:2206.09477, 2022)

What this evaluates

Evaluates a model's ability to perform geometric matrix completion on multi-network recommendation datasets. It probes how well graph neural networks and low-rank representations can integrate cross-network and within-network features to predict missing user-item ratings.

Datasets

• Douban — total 136891; splits: train (-1), test (-1)
• Flixster — total 26173; splits: train (-1), test (-1)
• YahooMusic — total 5335; splits: train (-1), test (-1)
• ML-100K — total 100000; splits: train (-1), test (-1)
• ML-1M — total 1000209; splits: train (-1), test (-1)

Metrics

• RMSE (primary) — range: other
  • Root Mean Squared Error: the square root of the average of squared differences between predicted and actual ratings over the test set.

Input / output format

Input: Sparse user-item rating matrix, plus user-user and item-item interaction networks represented as adjacency matrices. For datasets lacking one network type, an identity matrix is used as a placeholder.

Output: Predicted rating values for all unobserved user-item pairs in the test set.

Scoring recipe

import numpy as np
def compute_rmse(y_true, y_pred):
    return np.sqrt(np.mean((y_true - y_pred) ** 2))

Common pitfalls

• The exact train/test split is not specified in the paper; it relies on a fixed partition adopted by prior works (Yao et al. 2018, Monti et al. 2017), making exact reproduction difficult without consulting those citations.
• For ML-100K and ML-1M, user-user and item-item interaction networks are constructed via k-NN search on features with k=10 or k=12, which acts as a critical hyperparameter affecting performance.
• Douban and YahooMusic only provide one type of interaction network (user-user or item-item respectively); the missing network must be replaced with an identity matrix, which is easy to overlook.

Evidence (verbatim from paper)

The metric for comparison is the widely adopted rooted mean squared error (RMSE).

Citation

@misc{du2022geometric,
  title={Geometric Matrix Completion via Sylvester Multi-Graph Neural Network},
  author={Du et al. (2022)},
  year={2022},
  note={arXiv:2206.09477}
}

• arXiv: 2206.09477