As we continue to bask in the post Olympic glow of national achievement and the “2012 effect” it seems strange to remember the dim dark days at the start of the games when team GB went a whole 3 days without winning a gold medal. As the press shrieked that we were heading for disaster, unable to meet our target of 20 golds despite massive investment, I asked myself what parallels were there with risk management and what really were Mo Farah’s chances?
Well, as we all now know, actually pretty good. Of course only an idiot would assume that winning 29 medals over 16 days should equate to 2 every day with Sundays off, but even so, how likely should a medal-less day be. Well if you assume a Poisson distribution – commonly used for estimating event frequency – and take an average of 1.8 golds a day, the chance of a day with no medals is 16%. The chances of a super Saturday were actually 7%.
The bad news is that, as you can see from the chart the Poisson doesn’t quite fit what actually happened. The good news is that a day without any golds was actually more likely at 38% of all Olympic days. The least likely (below 5) was a single gold day, which only happened once. The last day of the boxing since you ask.
So why does any of this matter? Because it shows we are very bad at estimating how frequently things happen even when its quite straightforward. We assume that events are evenly distributed and get confused when they’re not. Not much of a problem with gold medals, but quite a big problem when you’re tying to understand access rights, detect fraud, and regulate access to our highly valuable systems and data. And that goes double for those trying to write the regulation.
We assume that because failures are relatively unlikely they are also uniformly infrequent. Having spent the best part of a decade working on access control, risk and regulation, its clear to me that an approach that defines controls by exception management, otherwise known as – the boss checks my work – will perform splendidly with “frequent but not disastrous” but does nothing to stop the “very infrequent but quite awful”.
So a strange lesson from the Olympics is that risk management and regulation is going to consistently fail until we stop managing with our intuition, educate ourselves about understanding big data and start really using automatic analysis to predict and analyse.
So next time you ask yourself how can I protect myself from those with inappropriate access to systems and data, think automation and analysis and that way maybe the Olympic legacy can be more robust access security as well as more kids playing sport..