Estimating Forecast Accuracy Metrics by Learning from Time Series Characteristics ^†

Accurate forecasts play a crucial role in various industries, where enhancing forecast accuracy has been a major focus of research. However, for volatile data and industrial applications, ensuring the reliability and interpretability of forecast results is equally important. This study shifts the focus from predicting future values to estimating forecast accuracy with confidence when no future validation data is present. To achieve this, we use time series characteristics calculated by statistical tests and estimate forecast accuracy metrics. For this, two methods are applied: Estimation by the euclidean distances between time series characteristic values, and second, estimation by clustering of time series characteristics. In-sample forecast accuracy serves as a benchmark method. A diverse, industrial data set is used to evaluate the methods. The results demonstrate that there is significant correlation between certain time series characteristics and estimation quality of forecast accuracy metrics. For all forecast accuracy metrics, the two proposed methods outperform the in-sample forecast estimation. These findings contribute to improving the reliability and interpretability of forecast evaluations, particularly in industrial applications with unstable data.