Introduction to Forecasting: Difference between revisions

From Forecasting Wiki
No edit summary
(Added Kalshi's introduction to prediction market terminology, etc. and linked to the newly created post-mortems page.)
Line 1: Line 1:
Political experts are on average worse than chance at political predictions<sup>[citation needed from that famous Tetlock paper]</sup>. There is no reason to believe that we, as random members of the public, are any better. On the other hand, prediction markets and aggregators are a decentralised way of getting information earlier. Forecasting is valuable the following reasons:
+
Political experts are on average worse than chance at political predictions<sup>[citation needed from that famous Tetlock paper]</sup>. There is no reason to believe that we, as random members of the public, are any better. On the other hand, prediction markets and aggregators are a decentralised way of getting information earlier. Forecasting is valuable for the following reasons:
   
 
* Many people in this space hold views on many topics. It's good to hold ourselves accountable - how accurate are we?
 
* Many people in this space hold views on many topics. It's good to hold ourselves accountable - how accurate are we?
* Forecasting questions allow people to anonymously give information to the community in exchange for money or reputation
+
* Forecasting questions allow people to anonymously give information to the community in exchange for money or reputation.
* When holding discussions, prediction markets and concretise them
+
* When holding discussions, prediction markets and concretise them.
   
 
== If you don't like reading wikis, you should read ==
 
== If you don't like reading wikis, you should read ==
   
* [https://en.wikipedia.org/wiki/Superforecasting:_The_Art_and_Science_of_Prediction Superforecasting]. ''The'' book to read on forecasting. It explains the research that led to forecasting becoming trusted as a decision making mechanism and some ways that we can improve as forecasters
+
* [https://en.wikipedia.org/wiki/Superforecasting:_The_Art_and_Science_of_Prediction Superforecasting]. ''The'' book to read on forecasting. It explains the research that led to forecasting becoming trusted as a decision making mechanism and some ways that we can improve as forecasters.
* [https://www.youtube.com/watch?v=e6Q7Ez3PkOw Intro to Forecasting (video series)]. A great series of videos introducing forecasting
+
* [https://www.youtube.com/watch?v=e6Q7Ez3PkOw Intro to Forecasting (video series)]. A great series of videos introducing forecasting.
* [https://twitter.com/MichaelTrazzi/status/1380974774717976581 Tweet summary] of Intro to Forecasting
+
* [https://twitter.com/MichaelTrazzi/status/1380974774717976581 Tweet summary] of Intro to Forecasting.
  +
* [https://forecasting.wiki/wiki/Introduction_to_Forecasting Prediction market terminology] (by Kalshi).
* Something on prediction markets <sup>[please someone add one]</sup>
 
   
 
== Two frames for forecasting: Self-Improvement vs Information Gathering ==
 
== Two frames for forecasting: Self-Improvement vs Information Gathering ==
Line 20: Line 20:
 
[[Prediction markets]] involves betting on outcomes. Some users will be paid if they happen and others if they don't. The current price of the bets indicates a level of confidence in the outcome. Kalshi, polymarket etc.
 
[[Prediction markets]] involves betting on outcomes. Some users will be paid if they happen and others if they don't. The current price of the bets indicates a level of confidence in the outcome. Kalshi, polymarket etc.
   
Prediction aggregators (like Metaculus and Good Judgement Open) gather forecasts and score how well individuals did according to various [https://forecasting.wiki/wiki/Category:Forecast_evaluation scoring methods] which incentivise accuracy, regular forecasting or both
+
Prediction aggregators (like Metaculus and Good Judgement Open) gather forecasts and score how well individuals did according to various [https://forecasting.wiki/wiki/Category:Forecast_evaluation scoring methods] which incentivise accuracy, regular forecasting or both.
   
 
== Hard problems within forecasting ==
 
== Hard problems within forecasting ==
Line 34: Line 34:
 
** Metaculus
 
** Metaculus
 
** Good Judgement Open
 
** Good Judgement Open
  +
**Discord servers of Metaculus and various prediction markets
* Do some forecasts
 
  +
* Do some forecasts and do [[Post-Mortem|post-mortems]] once they resolve.
   
 
== How do I forecast as an org ==
 
== How do I forecast as an org ==

Revision as of 11:33, 13 April 2022

Political experts are on average worse than chance at political predictions[citation needed from that famous Tetlock paper]. There is no reason to believe that we, as random members of the public, are any better. On the other hand, prediction markets and aggregators are a decentralised way of getting information earlier. Forecasting is valuable for the following reasons:

  • Many people in this space hold views on many topics. It's good to hold ourselves accountable - how accurate are we?
  • Forecasting questions allow people to anonymously give information to the community in exchange for money or reputation.
  • When holding discussions, prediction markets and concretise them.

If you don't like reading wikis, you should read

Two frames for forecasting: Self-Improvement vs Information Gathering

Forecasting as self improvement. You (or your organisation) make forecasts about the future. You keep a track record of your progress. Looking back, you spot the repeated kind of errors and work on making your future decision better. You become wary of making forecasts in future that you don't think will hold up. Your behaviour is better linked to reality and you become a more accurate source of information.

Forecasting as information gathering. There is an issue you want to know about. You create a prediction aggregator or market. People make forecasts/bets. This provides early indicators on this issue. Both prediction markets aggregators have a great track record[citation needed]. You learn what will happen earlier than you otherwise would.

Prediction Markets vs. Prediction Aggregators

Prediction markets involves betting on outcomes. Some users will be paid if they happen and others if they don't. The current price of the bets indicates a level of confidence in the outcome. Kalshi, polymarket etc.

Prediction aggregators (like Metaculus and Good Judgement Open) gather forecasts and score how well individuals did according to various scoring methods which incentivise accuracy, regular forecasting or both.

Hard problems within forecasting

How do I forecast as an individual

  • Join a community of forecasters
    • Follow forecasters on twitter @nikosbosse @fianxu @peterwildeford
    • Metaculus
    • Good Judgement Open
    • Discord servers of Metaculus and various prediction markets
  • Do some forecasts and do post-mortems once they resolve.

How do I forecast as an org

  • If you have effective altruism questions talk to @nathanpmyoung or [blanka]
  • If you have general questions, write them ["how to write question on metaculus" page required]
  • Set up prediction markets
  • Cultivate Labs