“Ah-hah! Statistics”

In his book “The Vision of the Anointed” Thomas Sowell wrote about what he called “Ah-hah! Statistics.” This is where one starts from a proposition, then search through mounds of data until one finds something, anything, that seems to support that proposition.  One then holds up that set of figures and goes “Ah hah!”

It has been said that one can prove anything if one is allowed to make up ones data.  This should be obviously true with very little thought indeed.  However, it has also been said that one can prove almost anything if one is allowed to simply exclude data that does not fit.  Indeed, if the situation is at all complicated, one is almost certain to find something that fits a given proposition, any given proposition.  You can always, if you wish, pare away the data that doesn’t fit until you have something that does.  Then, it becomes a matter of saying “Ah-hah!” and you’ve “proven” your case.  Except you haven’t.

Take, for instance, proponents of gun control.  They point to other countries with strict gun control that have strict gun control and that also have low violent crime (or even more restricted, low “gun crime”) and go “Ah-hah!  Gun control saves lives!” They ignore the many countries with strict gun regulation that, nevertheless, have very high violent crime rates.  Ninety-three countries, for instance, have higher intentional homicide rates than the US.  The one with the highest rate.  Wikipedia (I know, I know, but it’s just a publicly available source for a first cut) provides listings of countries both by homicide rate and by civilian gun ownership. Take the ranking from both those listings, exclude only the ones where a country or region isn’t included in both lists, making as complete a list as is readily available, and plotting the ranking of gun ownership against the ranking of homicide rates, and one would expect–at least so the anti-gun folk would tell us–that the fewer guns in civilian hands the lower the homicide rate.  However:

You can “prove” nearly anything if you exclude data that don’t fit.  Oh, they’ll claim reasons why this data that doesn’t fit isn’t “relevant.” They’ll say they’re limiting to the “developed world” (except some of those nations over on the right hand side are quite well developed), but that’s only another way of saying that other factors are more important than gun ownership in driving homicide rate.  If that were not the case, there would be no need to exclude nations because they’re not “developed.” And if there are differences between those on the right hand side from those on the left causing them to have much higher homicide rates, then how can they say that there are no other differences between the US and those on the left likewise causing the US to have higher homicide rates than those other nations.

This is a simple illustration.  I could have gone further and used actual homicide rates and actual estimated gun ownership rates but the basic principle would not have changed.  The point remains that “statistics” can be used to paint a false picture depending on what is included, and equally if not more important, what is left out.

The whole point of statistics is to take large amounts of data and render it into an understandable form.  The “exceptions” the data that don’t fit, they are an important part of the overall picture.  They represent how much data can vary or possible places to look for experimental or data collection error.  Simply dismissing them because they don’t fit the theory is bad science and bad math.

But it does seem to be “good politics” in many people’s view.  At least, it’s a weapon they can use to hoodwink the unwary and push their own agenda ahead.