Context

This issue serves as a discussion topic, summarizing the current state of reconciliation methods in sktime, identifying which methods are missing, and suggesting potential improvements for user experience.

A Quick Summary of Hierarchical Reconciliation Methods in Literature

Single-Level Approaches

• Bottom-up: Forecast bottom levels and then aggregate to obtain forecasts for the entire hierarchy.
• Top-down:
  • Share forecast: Transform all series (except the total) into shares relative to the total. Forecast independently, then multiply the shares by the total forecast.
  • Forecast proportions [10]: Forecast all series, use the forecasts to compute shares relative to the total forecast, and then multiply by the total forecast to obtain reconciled forecasts.
    Middle-out: Select a level. Keep the forecasts for this level unchanged and use one reconciliation method for the series above and potentially another for the series below.

Reconciliation Approaches

For forecast reconciliation, all series are forecasted independently, and then the base forecasts are projected onto the coherent subspace. This can be viewed as a generalized least squares (GLS) problem with linear constraints. Solutions can typically be computed using a projection matrix approach or constrained GLS. The latter is more efficient, as it requires inverting an $n_a \times n_a$ matrix instead of $n \times n$ (if $n_a$ is the number of non-bottom timeseries, and $n$ is the total number of series in the hierarchy) [9]. There are various methods based on different error covariance matrix assumptions:

• OLS [6]: Simple orthogonal projection, not requiring an error covariance matrix.
• WLS-Struc: Assumes the error variance of each series is proportional to the number of series below it in the hierarchy.
• Optimal reconciliation (and its variants) [2]: Uses the error covariance matrix to compute the optimal solution, assuming constant error covariance matrices across all forecast horizons.
• Reconciliation with Machine Learning [4,5]: Uses base forecasts as inputs for machine learning algorithms, which then forecast the bottom levels. Aggregation follows to obtain the final forecasts.
• Non-negative reconciliation [3]: Solves the optimal reconciliation problem (or any other) with a constraint for non-negative forecasts. Solvable using quadratic programming (QP) algorithms. There's also an alternative heuristic that sets all negative reconciled forecasts to zero.
• Reconciliation with immutable series [1]: This method can be combined with any of the above. First, identify immutable series, then choose a set of $n_b$ basis vectors (potentially non-bottom series, ensuring the corresponding slice of the summing matrix $S$ is invertible). The summing matrix can then be partitioned for all reconciliation methods while keeping the immutable series constant. This method can also be used with non-negative reconciliation (via QP) or analytically with matrix inversion.

Current State in sktime

Currently, the following objects are related to reconciliation:

1. Reconciler (transformer): bu, ols, wls_str, td_fcst
2. ReconcilerForecaster (forecaster): mint_cov, mint_shrink, ols, wls_var, wls_str, bu, td_fcst

The transformer applies all reconciliation methods that do not require the error covariance matrix, except for the top-down share forecasting strategy. To use the other methods, forecasting and reconciliation need to be performed simultaneously.

Possible Improvements

I would appreciate feedback on the suggestions below:

• All reconciliation methods should also be implemented as transformations: In many industrial applications, optimal reconciliation is applied independently of the forecasting step. For example, different teams may generate forecasts, with a separate reconciliation strategy applied afterward using all forecasts. This is also useful in judgmental forecasting scenarios, where the error covariance matrix can be estimated from historical forecasts.
• Implementation of non-negative reconciliation as a hyperparameter: This is crucial for reconciling intermittent time series, where negative forecasts may make it impossible to build a valid product after reconciliation. This feature is essential, in my opinion.
• Middle-out reconciliation transformer: Potentially a transformation that accepts two reconciliation strategies and a level to serve as the "middle."
• Top-down share forecasting: Although conceptually straightforward, this strategy introduces subtle API design challenges. If all methodologies are implemented as transformations, this would be a preprocessing step in a forecasting pipeline where the transform method casts all non-total levels to shares and inverse_transform applies the shares to the total forecast. This contrasts with other methods, which are simple post-processing steps where the transformation applies the reconciliation directly to the base forecasts. We may need to accept this as an outlier.

More Complex Features

The following features may be more challenging to implement and might not be achievable as simple transformations:

• Temporal reconciliation: These algorithms could be extended for reconciliation across different temporal aggregations (similar to the MAPA forecaster). This could involve a transformer that takes panel/hierarchical time series and creates a hierarchical structure based on time components. However, implementing this as a transformer may require a new machine type for temporal hierarchies since the time index might not fully capture different temporal structures.

• Cross-temporal reconciliation [7,8]: The authors in [7] show that sequential hierarchical and temporal reconciliation does not guarantee full reconciliation. In [8], an optimal cross-temporal reconciliation strategy is proposed, where cross-sections and temporal hierarchies are reconciled simultaneously. However, it is shown that in many cases, it does not outperform standard one-way reconciled forecasts [9]. This approach seems more relevant for decision-making contexts than for accuracy improvement.

Further Comments

1. Non-negative reconciliation requires solving an optimization problem. If this constraint is non-existent, an analytic solution can be obtained. This should be invisible to the user. If a user wants an OLS reconciliation (equivalent to an identity error covariance matrix), they should be able to choose non-negativity constraints without needing to understand the internal mechanics.
2. ols, wls_var, wls_str, mint_cov, and mint_shrink are essentially the same methodology given that a covariance matrix is given as input. This could help in creating a modular implementation of transformations.
3. The immutable time series approach translates the reconciliation problem to one where the chosen series remain constant. This viewpoint can be used to add this feature without introducing significant complexity.
4. For reconciliation strategies that require error covariance matrices, the transformations could accept it as an initialization hyperparameter. The ReconcilerForecaster could compute the matrix internally and pass it to an inner transformation if one of these methods is chosen.

Issues

I would like to know if it would be ok to open the following issues:

1. [ENH] Add OptimalReconciler transformation that receives error covariance matrix as input.
2. [ENH] Add SingleLevelReconciler transformation and refactor Reconciler into a wrapper delegating to SingleLevelReconciler or OptimalReconciler.
3. [ENH] Add non-negative optimal hierarchical reconciliation. (implemented after or with 1.)
4. [ENH] Add MiddleOut transformation for hierarchical forecasting.
5. [ENH] Add Topdown (share) transformation for hierarchical forecasting.
6. [ENH] Add ImmutableReconciler transformation to forecast hierarchies with immutable series.

What do you think?

References

[1] Zhang, Bohan, et al. "Optimal reconciliation with immutable forecasts." European Journal of Operational Research 308.2 (2023): 650-660.
[2] Wickramasuriya, S. L., Athanasopoulos, G., and Hyndman, R. J. (2019). "Optimal forecast reconciliation for hierarchical and grouped time series through trace minimization." Journal of the American Statistical Association, 114(526), 804–819.
[3] Wickramasuriya, S. L., Turlach, B. A., and Hyndman, R. J. (2020). "Optimal non-negative forecast reconciliation." Statistics and Computing, 30(5), 1167–1182.
[4] Spiliotis, Evangelos, et al. "Hierarchical forecast reconciliation with machine learning." Applied Soft Computing, 112 (2021): 107756.
[5] Burba, Davide, and Trista Chen. "A trainable reconciliation method for hierarchical time-series." arXiv preprint arXiv:2101.01329 (2021).
[6] Hyndman, Rob J., et al. "Optimal combination forecasts for hierarchical time series." Computational statistics & data analysis 55.9 (2011): 2579-2589.
[7] Kourentzes, Nikolaos, and George Athanasopoulos. "Cross-temporal coherent forecasts for Australian tourism." Annals of Tourism Research 75 (2019): 393-409.
[8] Di Fonzo, Tommaso, and Daniele Girolimetto. "Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives." International Journal of Forecasting 39.1 (2023): 39-57.
[9] Athanasopoulos, George, et al. "Forecast reconciliation: A review." International Journal of Forecasting 40.2 (2024): 430-456.
[10] Athanasopoulos, G., Ahmed, R. A., & Hyndman, R. J. (2009). Hierarchical forecasts for Australian domestic tourism. International Journal of Forecasting, 25, 146–166.