When Forecast Accuracy Fails to Translate into Performance

Financial forecasting models are typically evaluated using statistical metrics such as RMSE or out-of-sample R². However, investors ultimately care about economic performance — e.g., Sharpe ratios, portfolio efficiency, and risk-adjusted returns.

This project studies the disconnect between statistical forecast accuracy and economic value.

Core Contribution

In Lost in Translation? Risk-adjusting RMSE for economic forecast performance (Journal of Forecasting, forthcoming), we:

• Derive the Sharpe-ratio gap between perfect foresight and estimated portfolios
• Decompose it into:
  • Risk-Adjusted Mean Forecast Error (RAFE)
  • Risk-Adjusted Covariance Forecast Error (C-RAFE)
• Show that the widely used RMSE is a special case of a risk-adjusted loss function under highly restrictive assumptions
• Demonstrate that RMSE explains little of the performance shortfall
• Show that risk-adjusted error measures explain the Sharpe-ratio gap across a wide range of portfolio strategies

Key Insight

Forecast models optimized using MSE-based criteria are not aligned with the investor’s objective function.

Improving statistical fit does not necessarily improve economic performance.

Ongoing Research

This project extends toward:

• Risk-adjusted loss functions in regression and machine learning
• Portfolio-aware model selection
• Utility-consistent forecast evaluation
• AI-based asset allocation with economically aligned objectives