Since the US election result there have been many articles about how Nate Silver got it right. It has been triumphed as a victory for science and Bayesian statistics. There’s only one problem – it’s not quite science! Or at least, we don’t know if it’s science or not.
Before I go on, I am the first to say congratulations to Nate Silver and Sam Wang and all others who use polls and statistical modelling to give a prediction (with probability) of an election outcome. I think it’s great.
The reason it’s not science is that we don’t know how Nate Silver makes his predictions. The methods he uses are proprietary and are kept secret. Everybody seems to be assuming that he used some kind of statistical modelling methods to predict the outcome. This may well be true, he probably did, but we don’t know. For this reason it cannot be called science. The scientific method contains experiments where all the methods and conditions are made public, so that other scientists can attempt to replicate the experiment if they want to. In the case of computer usage, the source code is made public. The scientific community doesn’t accept what other scientists say unless it has been confirmed by other scientists. The cold fusion debacle is a good example.
In this case, we cannot replicate Nate Silver’s predictions because he keeps the model secret. True, there is an explanation of sorts here. He assigns weightings to various things. We don’t know the precise weights though. Perhaps he assigns weightings based on the star signs of the election candidates. Maybe he’s just really good at guessing. Who knows. Some people can guess at how he might have done it, and those guesses may well be right, but we don’t know. We just have to have faith.