Standard deviations for fun and profit (or, why I heart statistics)Posted: September 29, 2011
Unless you use the hero of the tale, standard deviation. In the example above, the bleep count has a standard deviation of 5 and mean of 100. So 2 standard deviations is equivalent to 10% of the mean in this case, which we decided was our starting comfort level. About 5% of events will fall outside 2 s.d. in a normal distribution, so in our timeline of 50 events here we would expect 2 alerts. If that starts to feel like too many alerts, we could extend the limits out to 3 s.d., which will only pick up even more unusual events (~0.3% of the time).
Now if we’re happy with the sensitivity level, we can use 2 s.d. for the blip count as well. This will be totally consistent with our bleep alerting, and really easy to implement.
Ahh, that’s better. So now we only get alerted when something genuinely unusual happens, within the parameters of standard blip variation, and we don’t get hounded by false alarms.
Maybe statistics is not so nasty and smelly after all.
bonus material: code to generate the graphs in R is below:
hist(bell,100,main=”some normally distributed data”,xlab=”value”)
plot.ts(blip,ylim=c(60,140),ylab=”blips”,main=”blip count with revised alert threshold”)