Data Torturing

In the 1960s the British economist Ronald Coase gave a talk at the University of Virginia. He said “if you torture the data enough, nature will always confess.”

Data torturing is the practice of repeatedly interpreting source data until it reveals a desirable result.

Occasionally data torturing is used with good intentions. For example, the discovery of the planet Neptune by Urbain Le Verrier was acheived by making hundreds of calculations based on the astronomical observations of Uranus.

Data torture has also been used extensively in advertising. The slogan “8 out of 10 cats agree” (coined by Whiskas) has become a euphemism for a dubious marketing claim.

Data torturing is sometimes the unexpected result of wishful thinking or confirmation bias. It is human nature to see a pattern in something that affirms our beliefs, and sometimes the outcome of data processing will produce an answer which appeals and is then accepted as proof. For example you may see a cloud in the sky or an ink spot in the shape of something familiar. In 1976 the Viking 1 orbiter took a photo of Mars that resembled a face. Many people had to wait until it was re-photographed 18 years later at a higher resolution by the Mars Global Surveyor before they would believe it wasn’t what they first thought to be.

However data torturing in the political arena tends to come under the greatest scrutiny. Unfortunately, politicians can make claims that can be shown to be correct by referring to source data but the voracity of their claim only becomes doubtful if you actually look at how the data was collated and sampled, or by the particular method of interpretation used.