Aggregation of Non-Binary Predictions

The author would be happy about help on this article.

Combining several predictions from several forecasters or models consistently improves the accuracy of forecasts. Methods may differ depending on whether forecasts are for a binary (yes/no) or a non-binary target (like world GDP in year X). This page only deals with aggregating probabilistic forecasts (i.e. predictive distributions) for non-binary outcomes. Aggregation techniques for binary forecasts can be found here.

Ways of combining probability distributions

Combination of the CDF of different forecasts

Probability distributions can be combined either based on their probability density functions (PDF) or their cumulative density functions (CDF). Usually, forecasts are combined using the CDF.

Several CDF can be either combined horizontally or vertically. A horizontal combination of several CDF is equal to a combination of the quantiles of the CDF. A vertical combination of the CDF is equal to a mixture distribution that combines the cumulative densities of the individual forecasts.

When combining forecasts based on their PDF, then only a vertical combination is sensible. When combining using the mean, then it does not matter whether we combine functions based on their PDF or CDF, as the sum of integrals is the same as the integral of a sum of two distributions. For combinations based e.g. on the median of the cumulative or non-cumulative density at a given point, differences may occur (although these will typically not be very large).

The forecast combination puzzle

Empirically, it is very difficult to improve on unweighted ensembles by estimating weights for individual forecasters from the data.

References

<comments />