Forecasting Best Practices

This article is work in progess. The author is working on it and it is not yet ready for review.

The most accurate forecasters commonly follow a set of best practices. While there may not be universal agreement on best practices, this section describes some forecasting practices that are generally considered to be helpful.

Basic Principles

Update Often

It's good forecasting practice for a forecaster to keep up to date on new information and change their mind based on the evidence. Stale forecasts often become more inaccurate over time because they don't incorporate the most recent evidence. It can also be easy for forecasters to become biased toward their initial view, a form of choice-supportive bias, and because of this bias it may be easier to do more frequent smaller updates than it is to do larger less frequent updates.

Describe Your Rationale

Whether publicly or privately, it's good practice for forecasters to document their rationale when making a forecast. This helps avoid hindsight bias, the tendency for people to believe they were correct all along. Even if a forecaster's initial assigned probability turns out to be incorrect, it can be easy for them assume that they were wrong but for the right reasons, or that their reasoning was well-calibrated but just happened to be wrong this time (for example, a well-calibrated forecast of 60% will be directionally wrong 40% of the time). A documented and detailed rationale helps a forecaster to confront themself with what they thought at the time and allows them to learn lessons from previous mistakes. A documented rationale is crucial for correcting previous forecasting inaccuracies and learning from mistakes in the underlying reasoning behind a forecast.

Engage with the Community

Other forecasters are often happy to share the rationale behind the forecasts, and a forecaster can learn a lot by asking others why their forecasts differ from theirs or how they are incorporating new information.

Use Base Rates

The base rate is the frequency of occurrence in the reference class, or in other words how often you would expect something to occur based only on how common it is in the population. Base rates provide a valuable starting point for forecasts and is often one of the first steps experienced forecasters make when making a forecast.

Ignoring base rates in populations is commonly known as base rate neglect and can lead to inaccurate assessments.

References

<comments />