Getting rid of the predictablePosted: August 1, 2011
Here is a very striking example of how adding some simple analysis to a raw metric can be incredibly important.
When businesses talk about their ‘analytics’, most of the time what they’re actually talking about is plain graphs of business metrics. Maybe if you’re lucky they will also include some information on averages or on last year’s comparable values.
Business leaders get used to eyeballing these graphs and drawing their own conclusions about what’s going on. As I discussed in my last post, most of the time they want to know (a) does any point stick out and (b) what’s the overall trend. The problem is that humans are far better at spotting some types of pattern than others. Good basic data visualisation helps enormously, but still not everything you need to know is immediately visible. Take a look at the chart below – this shows online sales by day for a real business.
So far so uneventful, right? The data obviously follows a weekly cycle – looks like Mondays are always the highest (pretty normal for most businesses), and there’s a nice upward spike about day 21 – if I was the chief executive looking at this graph, I’d feel pretty comfortable.
Now take a look at this chart, which shows the week on week change in sales.
Ouch! A big fat negative outlier on day 28. Something worrying did happen after all.
The outlier happened on a Monday, so didn’t stick out on the original graph – the eye misses the fact that there are three low days in a row instead of two – but the missing sales amount in this case (caused by a technical problem with processing sales, which fortunately was spotted and fixed) was big enough to be important for the outcome of the whole month.
There’s also some more subtle but useful information that comes out more easily by looking at the week on week – for example that spike on day 21 was also preceded by a strong 5 previous days. That doesn’t stick out on the raw graph, as we mostly notice the biggest peak on the Monday, rather than the fact that the preceding weekend is also very good relative to a normal weekend.
The general principle here is that if you can see a way to strip out the predictable and uninformative elements of the numbers – in this case, the regular weekly cycle – then the trend and the outliers show up much more easily. That’s exactly what the readers of a simple dashboard are doing in their minds when they look at the plain metrics and mentally discount for the ups and downs of the regular weekly pattern, but we can make it a lot easier for them by actually doing the calculation behind the scenes.