|
|
|
‘Lies, damned lies, and statistics’ |
|
|
|
Written by Dan Johnson
|
|
Friday, December 11 2009, 10:50 PM |
Why are scientists always so concerned with data and statistics? Don’t we have enough facts that are already known in the world?
On the contrary, continual fact-checking and the assumption that only part of the story is known are what give scientists the confidence to be able to make any claims at all.
There are several reasons why scientists trust evidence even over their own beliefs and common sense. One is that human thought and even our language are often inaccurate, for example when we say things like a spacecraft “escaped Earth’s gravity” or experienced “zero-g.” (Spacecraft orbit the Earth because of gravity, not because they have escaped it.)
Worse yet, the human mind has a tendency to miscalculate risks and misattribute causes. To illustrate, assume that you are given a medical lab test to see if you have a disease that is known to exist in the population at a rate of one case per 1,000 people. The test method is certified 99 per cent correct, whenever it tests “positive.” It incorrectly indicates the presence of the disease in only one per cent of test cases. So, if you test positive, what is the likelihood that you have the disease?
Most people would probably assume that the chances of actually having the disease were high, at least better than even. Actually, if you test positive, you only have a nine per cent chance of having the disease. Of 1,000 people, 10 would be falsely indicated as having the disease and one person would actually have it. One out of 11 is nine per cent. In the case of a rarer disease, say one in 10,000, a “positive” test with this same 99 per cent precise method would mean you have less than a one in 100 chance of actually having the disease.
Apply this same clarity to polygraph machines and it becomes obvious that they are worthless as “lie detectors.” Studies have shown that the “false positive” rate of polygraphs used in crime detection is between 10 per cent and 40 per cent. Such testing fingers far more honest people as liars, than actual liars.
Another reason that data is still central to science is that we know from experience that any claim needs to be backed up with a comparison to reality. The quip “lies, damned lies and statistics,” attributed to 19th-century British Prime Minister Benjamin Disraeli, refers to the annoying abuse of facts and figures to support a weak claim. The irony is that the proper use of statistical analysis is the best method for preventing this from happening. The main purpose of statistical analysis is not to prove whether something is true or false, but to determine the likelihood of being wrong by claiming an effect.
Because of the importance of evidence, you don’t have to be a creative genius to be a scientist. What is required is a willingness to keep working and step back at each point and ask “what is the evidence, and how sure are we about the conclusion?”
This is why armchair deniers (those who say “it isn’t so” without doing the work of checking the facts with measurements) are anti-science. For example, for a long time it was popular among deniers to say that chlorofluorocarbon (CFC) refrigerants could not possibly affect the ozone in the upper atmosphere, because CFCs are heavier than air. Anyone who has ever gone outside knows that the air is moving and mixing, and that heavier gases and aerosols do not just lie on the ground. It is simple to measure atmospheric composition, but the myth persisted, out of lack of interest in evidence. Sometimes denial is just denial.
Dan Johnson is a professor of environmental science, and teaches applied statistical methods at the University of Lethbridge.
|
|
|
|
