For each day of the LivingStandardsNZ workshop (3-5 December 2013) we are giving the participants a ‘Thought for the Day’. These thoughts are from a range of sources and are designed to encourage the participants to think broadly about some of the issues they will be tackling over the course of the week.
Today’s Thought for the Day is the chapter ‘Try, and Err’ from Nate Silver’s The Signal and the Noise:
This is perhaps the easiest Bayesian principle to apply: make a lot of forecasts. You may not want to stake your company or your livelihood on them, especially at first. But it’s the only way to get better.
Bayes’s theorem says we should update our forecasts any time we are presented with new information. A less literal version of this idea is simply trial and error. Companies that really ‘get’ Big Data, like Google, aren’t spending a lot of time in model land. They’re running thousands of experiments every year and testing their ideas on real customers.
Bayes’s theorem encourages us to be disciplined about how we weigh new information. If our ideas are worthwhile, we ought to be willing to test them by establishing falsifiable hypotheses and subjecting them to a prediction. Most of the time, we do not appreciate how noisy the data is, and so our bias is to place too much weight on the newest data point. Political reporters often forget that there is a margin of error when polls are reported, and financial reporters don’t always do a good job of conveying how imprecise most economic statistics are. It’s often the outliers that make the news.
But we can have the opposite bias when we become too personally or professionally invested in a problem, failing to change our minds when the facts do. If an expert is one of Tetlock’s hedgehogs, he may be too proud to change his forecast when the data is incongruous with his theory of the world. Partisans who expect every idea to fit on a bumper sticker will proceed through the various stages of grief before accepting that they have oversimplified reality.
The more often you are willing to test your ideas, the sooner you can begin to avoid these problems and learn from your mistakes. Staring at the ocean and waiting for a flash of insight is how ideas are generated in the movies. In the real world, they rarely come when you are standing in one place. Nor do the ‘big’ ideas necessarily start out that way. It’s often with small, incremental, and sometimes even accidental steps that we make progress.