# Types of forecast: ex ante vs ex post

## September 06, 2015

Ex ante forecast is a forecast based solely on information available at the time of the forecast, whereas ex post forecast is a forecast that uses information beyond the time at which the forecast is made. Let’s discuss the two in more detail, as in different contexts the terms may mean slightly different things.

# Introduction

Only one thing is true about forecasts - they are always wrong. Any forecast $$\hat{Y}_t$$ of the actual value $$Y_t$$ is associated with the forecast error $$e_t = Y_t − \hat{Y}_t$$. The art and science of forecasting is all about bringing these forecast errors down to zero, as close as possible, for all future values of $$t$$. Doing this first for some of the past values of $$t$$ is often a useful exercise for developing a better forecasting model. It is this distinction between future and past values of $$t$$ that underlines the distinction between two types of forecast: ex post forecasts and ex ante forecasts.

The terms ex ante and ex post are among those few words in the forecasting literature that a general reader is unlikely to understand without first learning what they in fact mean. To see how the term ex ante have been used in the forecasting literature, check out the following excerpts:

1. Empirical results show that this procedure selects models that give reasonable ex ante forecast accuracy.
2. The models allowing for serial correlation are shown to have the best ex ante forecasting performance.
3. I looked primarily for studies that used real data to compare the ex ante forecasting accuracy of alternative methods.
4. They examined the potential benefits of ex ante rules when contrasted with model selection based on within-sample model fitting.
5. It is not clear to me from reading the paper if the resulting forecasts are true ex ante forecasts.
6. The forecasts were tested on a holdout ex ante sample that was known only to the administrator of the competition.
7. It may be, for example, that forecasters are using future information, perhaps inadvertently, so that forecasts are not ex ante, and I have known several cases where further study showed that a method was not as good as first suggested.
8. The NN3 competition evaluates the ex ante accuracy of forecasting the next 18 observations on two homogeneous sets of 111 or 11 time series of varying length and time series patterns on multiple established error metrics.
9. No matter how honest our efforts, and no matter how generous our colleagues, as long as comparisons were not genuinely ex ante a measure of doubt about our results was inevitable.

In general, ex ante is often used to mean ‘before-the-fact’ and ex post as ‘after-the-fact’. In the forecasting literature, an ex ante forecast is said to be any forecast that uses information available only at the time of the forecast. When a forecast prepared at certain time uses information available after that time, it is said to be an ex post forecast. A typical example of the latter is when known (actual rather than projected) values of external variables (regressors, predictors or drivers) are used in producing forecasts for the hold-out part of a time series. While ex post forecasts may be useful for exploring the properties of the forecasting model, it is ex ante forecasts (and the associated accuracy) that are ultimately important.

# Forecasting univariate time series

In the context of univariate time series forecasting, the difference between ex ante and ex post forecasts is well explained by Chan (2010). The remainder of this paragraph, including the quote below, follows him almost verbatim.

The ex post forecasts are made when the “future” observations are known during the forecasting period. It is used as a means to check against known data so that the forecasting model can be evaluated. The ex ante forecasts are made beyond the present, when the future observations are not available for checking.

Suppose that we observe $$\{Y_1,\dots,Y_n\}$$, so we may use $$\{Y_1,\dots,Y_t\}$$ for $$t < n$$ to estimate a model and use the estimated model to forecast $$\{Y_{t+1},\dots,Y_n\}$$. These are ex post forecasts since we can use them to compare against the observed $$\{Y_{t+1},\dots,Y_n\}$$. The estimation period in this case is $$t$$. On the other hand, when we forecast $$\{Y_{n+1},\dots,Y_{n+h}\}$$ for $$h > 0$$, we are doing ex ante forecasts. After fitting a model, we estimate a future value $$Y_{n+h}$$ at time $$n$$ by $$\hat{Y}_n(h)$$ based on the fitted model, while the actual value of $$Y_{n+h}$$ is unknown.

Somewhat differently from the above, in the context of univariate time series ex post forecast is sometimes used synonymously with the ‘model fit’ (i.e. small ex post forecast errors are associated with models that fit the data well). For example, according to Gardner & McKenzie (1988), in the 1960s and 1970s, the period of early theoretical development in quantitative forecasting, it was expected that

The better the fit (ex post) of the forecasting model, the better the accuracy (ex ante) should be.

Note that it is well known nowadays that the above statement is in general false. Ledolter (2010) also uses the term ex post when referring to the in-sample forecast performance:

If sufficient observations are available, one should divide the series into two parts, derive estimates of the parameters that are necessary to construct the forecasts from the first part, and evaluate the accuracy of the ex ante forecasts from the observations in the holdout period. This is important, since models that fit the data well (i.e., have small ex post forecast errors) sometimes perform badly in forecasting, and vice versa.

# Forecasting with external variables

In the context of time series with external variables ex post forecasts most commonly are forecast made with knowledge of the external variables. This type of forecast can be useful in quantifying the sources of forecast uncertainty, when answering questions like what part of a forecast error is due to a poor forecast of temperature and what part of it should be attributed to the forecasting model itself being imperfect. Quantifying the effect of errors due to the need to use estimated future values of external variables rather than their actual or observed values can often help improve the forecasting model. For example, Hyndman & Fan (2010) write:

Specifically, ex ante forecasts are the forecasts made in advance using whatever information is available at the time. On the other hand, ex post forecasts are those that are made using information on the “driver variables” that is only known after the event being forecast. The difference between the ex ante forecasts and ex post forecasts will provide a measure of the effectiveness of the model for forecasting (taking out the effect of the forecast errors in the input variables).

It is in this latter context that the distinction between the ex post and ex ante forecast is most useful. The reader of forecasting literature, however, should be aware that some well established names in the forecasting literature may use the terms simply to distinguish between the in-sample forecasts (done as part of the model fitting process) and the out-of-sample forecasts (done as part of the forecast evaluation process).

# References

Chan, N. H. (2010). Time series applications to finance with R and S-Plus (2nd ed.). John Wiley & Sons. http://www.sta.cuhk.edu.hk/nhchan

Gardner, E. S., & McKenzie, E. (1988). Model identification in exponential smoothing. Journal of the Operational Research Society, 39(9), 863–867. http://doi.org/10.2307/2583529

Hyndman, R. J., & Fan, S. (2010). Density forecasting for long-term peak electricity demand. IEEE Transactions On Power Systems, 25(2), 1142–1153. http://doi.org/10.1109/TPWRS.2009.2036017

Ledolter, J. (2010). Prediction and forecasting. Encyclopedia of Statistical Sciences. http://doi.org/10.1002/0471667196.ess2046.pub3

### Time series models: VAR model definition

The Vector AutoRegression (VAR) family of models has been widely used for modelling and forecasting since the early 1980s. A VAR model is...… Continue reading

#### Time series models: ARIMA model definition

Published on September 30, 2015

#### Outliers and the correlation coefficient

Published on September 05, 2015