Fuelcast Eval

fuelcast-eval

FuelCast: Benchmarking Tabular and Temporal Models for Ship Fuel Consumption — Viga et al. (2025) (arXiv:2510.08217, 2025)

What this evaluates

Evaluates the ability of tabular and time-series regression models to predict ship fuel consumption using operational, environmental, and temporal features. It probes how well models leverage in-context learning, weather covariates, and sequential patterns across different vessel types.

Datasets

• FuelCast — total ?; splits: train (-1), test (-1)

Metrics

• MAE (primary) — range: other
  • Mean Absolute Error: the average of absolute differences between predicted and actual fuel consumption values.
• R² — range: [-1, 1]
  • Coefficient of Determination: the proportion of variance in fuel consumption explained by the model, calculated as 1 - (SS_res / SS_tot).

Input / output format

Input: Tabular or time-series sequences containing vessel operational metrics, environmental/weather covariates, and temporal features. Target variable is continuous fuel consumption.

Output: Single continuous numerical value representing predicted fuel consumption.

Scoring recipe

maes, r2s = [], []
for fold in range(5):
    X_train, y_train = get_fold(fold, 'train')
    X_test, y_test = get_fold(fold, 'test')
    model.fit(X_train, y_train)
    preds = model.predict(X_test)
    maes.append(mean_absolute_error(y_test, preds))
    r2s.append(r2_score(y_test, preds))
return {'MAE': sum(maes)/len(maes), 'R²': sum(r2s)/len(r2s)}

Common pitfalls

• Evaluations must use 5-fold cross-validation; single splits will not capture vessel-specific variance.
• Omitting environmental or temporal features drastically inflates MAE and lowers R², invalidating comparisons with baselines.
• TabPFN results are highly sample-dependent (500 vs 1000); reporting must specify the in-context training size.

Evidence (verbatim from paper)

In task 1 we observe in Table [3] in the two bottom rows that TabPFN consistently achieved the lowest MAE across all vessels. For CPS Poseidon, it reached 0.061 (1000 samples), ahead of MLP (0.066), CatBoost (0.068), and the polynomial baseline (0.086). R² values among the top models ranged from 0.93 to 0.94.

Citation

@misc{viga2025fuelcast,
  title={FuelCast: Benchmarking Tabular and Temporal Models for Ship Fuel Consumption},
  author={Viga et al. (2025)},
  year={2025},
  note={arXiv:2510.08217}
}

• arXiv: 2510.08217