Noise: Difference between revisions

Content added Content deleted

Inline

Revision as of 17:26, 16 April 2022

Noise has at least two related but different meanings in the context of forecasting. The first type of noise is related to the issue of signal detection. The second type of noise is variability in judgment and is related to the idea of Cognitive Biases. The two types of noise are closely related.

Noise in signal detection

Making accurate forecasts requires that one is able to find valid evidence indicating what will be the true state of the world at the close of the question. Valid evidence is called signal, while everything that is not valid, and which could get in the way of identifying valid evidence, is called noise. Thus the goal of accurate forecasting is to find the signal in the noise, and then to make reasonable inferences based on the signal.

Noise in judgment

Noise, in this second sense, is all variability in judgment that is not directional. In forecasting, noise can arise due to differences in information, or differences in reasoning between or within forecasters.

Noise is preferable to bias in collective forecasting as noise tends to cancel out, and may even at times decrease bias due to the Bias-Variance Tradeoff. However, reducing noise is still an effective method for improving forecasts, and de-biasing techniques tend to work by decreasing noise more so than by decreasing bias.^[1] Noise is different than bias in one important way, which is that you do not need to know what the correct answer is in order to detect noise.

Noise can be caused by Cognitive Biases. Cognitive biases are directional deviations away from normative reasoning which sometimes lead to errors in judgment. This can lead to noise if one person exhibits a cognitive bias, and another does not, or if two people exhibit different cognitive biases. This could also happen within a single individual. For example, someone using the Affect Heuristic may give two different answers at different points in time due to variability in mood.

References

↑ [1] - Satopää, V., Salikhov, M., Tetlock, P., & Mellers, B. (2020). Bias, Information, Noise: The BIN Model of Forecasting. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3540864

[1] [1] - Satopää, V., Salikhov, M., Tetlock, P., & Mellers, B. (2020). Bias, Information, Noise: The BIN Model of Forecasting. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3540864

[1]

@@ Line 9: / Line 9: @@
 Noise, in this second sense, is all variability in judgment that is not directional. In forecasting, noise can arise due to differences in information, or differences in reasoning between or within forecasters.
-Noise is preferable to [[bias]] in collective forecasting as noise tends to cancel out, and may even at times decrease bias due to the [[Bias-Variance Trade-Off]]. However, reducing noise is still an effective method for improving forecasts, and [[de-biasing]] techniques tend to work by decreasing noise more so than by decreasing bias.<ref>[https://www.wu.ac.at/fileadmin/wu/d/i/statmath/Research_Seminar/SS_2020/Satop%C3%A4%C3%A4_BIN_Main_Article.pdf] - Satopää, V., Salikhov, M., Tetlock, P., & Mellers, B. (2020). Bias, Information, Noise: The BIN Model of Forecasting. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3540864</ref> Noise is different than bias in one important way, which is that you do not need to know what the correct answer is in order to detect noise.
+Noise is preferable to [[bias]] in collective forecasting as noise tends to cancel out, and may even at times decrease bias due to the [[Bias-Variance Tradeoff]]. However, reducing noise is still an effective method for improving forecasts, and [[de-biasing]] techniques tend to work by decreasing noise more so than by decreasing bias.<ref>[https://www.wu.ac.at/fileadmin/wu/d/i/statmath/Research_Seminar/SS_2020/Satop%C3%A4%C3%A4_BIN_Main_Article.pdf] - Satopää, V., Salikhov, M., Tetlock, P., & Mellers, B. (2020). Bias, Information, Noise: The BIN Model of Forecasting. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3540864</ref> Noise is different than bias in one important way, which is that you do not need to know what the correct answer is in order to detect noise.
 Noise can be caused by [[Cognitive Biases]]. Cognitive biases are directional deviations away from normative reasoning which sometimes lead to errors in judgment. This can lead to noise if one person exhibits a cognitive bias, and another does not, or if two people exhibit different cognitive biases. This could also happen within a single individual. For example, someone using the [[Affect Heuristic]] may give two different answers at different points in time due to variability in mood.