Lies, Damned Lies, and Statistics

This is a wonderful webpage explaining all about how to lie with statistics and the display of them in chart form. It is easy to understand, and fun to read. It is always worth remembering:

Figures don't lie
But Liars figure!

It is worth knowing this stuff for a number of reasons:

1) You don't feel so bad when you have to sort of make rough guesses for a report (blush);

2) You read other people's statistics with a less trusting eye. This is good. Whether you are looking at library statistics or government statistics, it is a very good thing to keep in mind that everybody gets tired when they have to count every little thing, and sometimes they sort of fudge a little. Sometimes they fudge a lot. Sometimes it's all smoke and mirrors. If you read this webpage, you will feel empowered to tell the difference a little better.

3) When you ask other people for statistics, you will understand better what they are giving you. :)

4) When you want to mislead somebody with statistics, you will have some good ideas about how to do it. ;)

Pay special attention to the 3-D bar chart -- these are not good ideas, unless you intend to confuse people. The froggie bar chart is another bad idea. Both things are misleading.

Here is another page that adds a bit more to thinking about ways that statistics can mislead, sometimes even unconsciously. See link here.

Some people subconsciously tend to depict data in the most favorable light. Others intentionally try to deceive the reader by selective data use, extrapolation from the data, using creative graphics, or by making faulty assumptions.

Following are just a few examples of how the data may be presented to cloud the actual result:

Consider the news headline quoting a study, “At lunchtime, men network, women run errands.” In reality, researchers at the University of California - Irvine, found that 51 percent of women said they shopped or ran errands at lunch. Only 39 percent of men claimed to do the same. No question asked men what they were doing instead of errands. The writer used a creative extrapolation of the data and a false assumption of activity to derive the headline.

From SRA Research Group website.

Here is another nice website that analyzes the misuse of statistics to sow fear:

From: Ed Chenel, a police officer in Australia.

Hi Yanks,

I thought you all would like to see the real figures from Down Under.

It has now been 12 months since gun owners in Australia were forced by a new law to surrender 640,381 personal firearms to be destroyed by our own government, a program costing Australia taxpayers more than $500 million dollars.

The first year results are now in: Australia-wide, homicides are up 3.2 percent, Australia-wide, assaults are up 8.6 percent; Australia-wide, armed robberies are up 44 percent (yes, 44 percent!). In the state of Victoria alone, homicides with firearms are now up 300 percent. (Note that while the law-abiding citizens turned them in, the criminals did not and criminals still possess their guns!)

While figures over the previous 25 years showed a steady decrease in armed robbery with firearms, this has changed drastically upward in the past 12 months, since the criminals now are guaranteed that their prey is unarmed.

There has also been a dramatic increase in break-ins and assaults of the elderly. Australian politicians are at a loss to explain how public safety has decreased, after such monumental effort and expense was expended in "successfully ridding Australian society of guns."

You won't see this data on the American evening news or hear your governor or members of the state Assembly disseminating this information.

The Australian experience proves it. Guns in the hands of honest citizens save lives and property and, yes, gun-control laws affect only the law-abiding citizens.

Take note Americans, before it's too late!

This piece of urban legend was collected from the web in 2001 according to the snopes website. The analyst then goes on to look at the way the author mis-used the percentages in the little message to fear-monger. I will snip considerably; please see the original website for a more in-depth analysis of the problems.

Origins: Although the old adage says that "Figures don't lie, but liars figure," those who seek to influence public opinion often employ a variety of means to slant statistical figures into seemingly supporting their point of view:

(snip)

* Percentages by themselves often tell far from a complete story, particularly when they involve small sample sizes which do not adequately mask normal fluctuations or the potential influence of a number of extraneous factors affecting the phenomenon under study.

(snip)

* Context is especially important, and percentages alone don't provide context.

(snip)

* Most importantly, percentages don't establish cause-and-effect relationships — at best they highlight correlations which may be due to any number of factors.

(snip)

In the specific case offered here, context is the most important factor. The piece quoted above leads the reader to believe that much of the Australian citizenry owned handguns until their ownership was made illegal and all firearms owned by "law-abiding citizens" were collected by the government through a buy-back program in 1997. This is not so. Australian citizens do not (and never did) have a constitutional right to own firearms — even before the 1997 buyback program, handgun ownership in Australia was restricted to certain groups, such as those needing weapons for occupational reasons, members of approved sporting clubs, hunters, and collectors. Moreover, the 1997 buyback program did not take away all the guns owned by these groups; only some types of firearms (primarily semi-automatic and pump-action weapons) were banned. And even with the ban in effect, those who can demonstrate a legitimate need to possess prohibited categories of firearms can petition for exemptions from the law.

Given this context, any claims based on statistics (even accurate ones) which posit a cause-and-effect relationship between the gun buyback program and increased crime rates because "criminals now are guaranteed that their prey is unarmed" are automatically suspect, since the average Australian citizen didn't own firearms even before the buyback. But beyond that, most of the statistics offered here are misleading and present only "first year results" where long-term trends need to be considered in order to draw valid cause-and-effect conclusions.

For example, the first entry states that "Homicides are up 3.2%." This statistic is misleading because it reflects only the absolute number of homicides rather than the homicide rate. (A country with a rapidly-growing population, for example, might experience a higher number of crimes even while its overall crime rate decreased.) An examination of statistics from the Australian Institute of Criminology (AIC) reveals that the overall homicide rate in Australia has changed little over the past decade and actually dipped slightly after the 1997 gun buy-back program. (The chart found at this link also demonstrates how easily statistics based on small sample sizes can mislead, as when the homicide rate in Tasmania increased nearly eight-fold in one year based on a single incident in which 35 people were killed.)

(snip)

The main point to be learned here is that determining the effect of changes in Australia's gun ownership laws and the government's firearm buy-back program on crime rates requires a complex long-term analysis and can't be discerned from the small, mixed grab bag of short-term statistics offered here. And no matter what the outcome of that analysis, the results aren't necessarily applicable to the USA, where laws regarding gun ownership are (and always have been) much different than those in Australia.

That might be enough different ways for now to think about being lied to with numbers. Everybody likes to think that numbers are bedrock solid and objective. But they are only as solid and objective as the people who compiled them and as the methodology they used both to collect and to display them. Be skeptical of numbers -- they can be meaningful and useful, or they can be lies, damned lies and statistics!