Spurious correlations and you

One of the interesting things in life is in how we make use of observations. We all try to understand why things happen and when we observe a connection – or more accurately, what we BELIEVE is a connection – we feel as if we have gained a sense of control over our lives. Knowing why something happens allows us to take corrective steps. When I was young my father started experiencing Angina. One Sunday he had a particularly bad day. Given the traditional pasta meal, he believed that the sauce was what made him feel worse. For many months following that observation, he always ate his pasta with a big hunk of melted butter instead of tomato sauce. Needless to say, he eventually discovered that he butter treatment just wasn’t working.

My March 23 column explored the human tendency to draw conclusions in the absence of substantial data. This tendency is fed in part by our belief that unusual events are an indication of an underlying pattern. It was inspired by a demonstration by my computer science professor when I was in university. At the end of his first lecture he asked half the class flip a coin fifty times when they got home. The other half was asked to make up a random list of flips. Not knowing his students yet, he read off the names in the second class and blew us away at how accurate he was in telling us if we flipped or just made it up. He then told us the secret. All he did was look for a series of five identical results. Statistically, it is almost guaranteed of happening. Psychologically however, very few people thought it was likely. The made up lists contained no such string.

There is a book called Innumeracy by John Allen Paulos. Innumeracy is the mathematical equivalent of illiteracy. Paulos’ book brilliantly explores some of the consequences of a mathematical naiveté. It focuses on the science of statistics and probabilities. We all know about odds, or at least we all believe we do. Almost everyone plays the lottery or plays the occasional betting game, but very few of us truly understand the implications of statistical probabilities and variance. The higher the variance, the more likely it is that unusual events are NOT significant. When true patterns exist, these red herrings make them tougher to detect.

Here is the column.

Heads or Tails

(Source: Pile…ou face? Journal Métro, March 23, 2010)

Want to impress your friends with your “psychic” abilities? Try this.

Ask some of them to flip a coin fifty times and write down the results. Give them the option of just making it up and writing down a random series of heads or tails on a piece of paper without telling you what they chose to do. When you look at their papers, you can figure out whether they really flipped or not with almost 100% accuracy!

How can you do it? Quite easily, just look for a string of at least five heads or tails in a row. Those who have such a string on their lists really flipped the coin. The rest of them made it up.

The reason for this is because flipping a coin fifty times will almost certainly result in the same outcome several times in a row. It is a simple fact of probability. Yet, if you were to make up a list that looked random in your head, most of you would assume it is highly unlikely to get the same result five straight times. The made up lists rarely have more than four heads or tails in a row.

This exercise tells us a lot about probability and how it affects our beliefs and attributions. The laws of chance interact with the human tendency to want to explain and understand things. We want to have a sense of control over out lives in order to avoid tragic consequences. Thinking we have something figured out gives us a sense of security, even if it is a false security.

What are the odds of that?

Things happen all the time. Some of them appear to be out of the ordinary and make us wonder if there is an underlying pattern. How often have you heard about a number of similar cancer deaths on a street or in a city and heard speculation that there must be something in the water or in the soil that must have caused it?

It does seem hard to believe, but unusual things do happen all the time. Being dealt four aces in poker is certainly unlikely, but it can happen. An unusual outcome is not evidence of an underlying pattern.

Important and significant patterns do exist in our lives. Unfortunately the laws of probability keep throwing us curveballs that make them hard to detect. Just because we want to understand why something happens does not always mean that we actually do.


Tagged as , , , .

Posted in Human nature.

Posted on 18 Apr 2010

Leave a comment

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

25