Simpson's paradox makes a donkey of us

By Burkard Polster and Marty Ross

The Age, 7 April 2008

Mathematics is not merely beautiful and fun. It is also a powerful tool to answer very important questions. Such as: who is the better full forward, Matthew Lloyd or Brendan Fevola? Answering such a question is not only important for bragging rights. It can also influence critical decisions on who to include in a fantasy football team.

So, who is the better full forward? In 2007, Lloyd and Fevola both played 19 games, with Lloyd’s 52 goals edging out Fevola’s 49. In 2006, Fevola played many more games than the injured Lloyd, and it makes more sense to compare their average goals per game played. By this comparison, Lloyd again had the edge, averaging 4.3 goals per game to Fevola’s 4. It seems clear, at least from goal averages in the last two years, Lloyd has been the superior spearhead.

But now we can make a very puzzling observation. Suppose we consider 2007 and 2006 combined. Over the two years, Lloyd kicked 75 goals in 22 matches, for an average of 3.4 goals per game. And, by comparison, Fevola kicked 143 goals over 40 games, for an average of 3.6 goals per game. So, though Lloyd had a better average in each year, Fevola had a better average overall!

In fact, if we include the data from 2005, the puzzle continues. In 2005, Lloyd averaged more goals than Fevola. But, over the years 2005 to 2007 combined, Fevola still had the higher average.

This strange phenomenon, where comparing averages of separate portions of some data contradicts an overall comparison, is known as Simpson’s Paradox. The possibility of this occurring was first recognized over 100 years ago, and there are now striking examples known in many real-world areas: sport of every kind; tuberculosis infection rates; sex discrimination claims; and on and on.

In fact, in a sense, wherever there is data there is Simpson’s paradox. In a recent paper, the mathematicians O. E. Percus and J. K. Percus proved that essentially any set of data can be cunningly split in a way to give rise to Simpson’s Paradox.

At first glance it seems very peculiar, but Simpson’s paradox is actually easy to explain. In our goalkicking example, the key is the year 2006. In that year both Lloyd and Fevola had a high average, with Lloyd’s slightly higher. However, Lloyd only played 4 games, which had little effect on his long-term average. On the other hand, Fevola’s sustained form over 21 games had a significant effect.

Undoubtedly, averages can be meaningful and informative. So, what is the moral of this story? Don’t let a high average over a short period mislead you. And, there’s only one Tony Lockett.