The return of the New Model Army
The sexy job this decade is supposed to be a statistician. This brings a smile to my face as I recall sitting in my statistics lectures at University twenty years ago surrounded by greasy, spotty men (91.7%) and very few women (8.3%). Statistics in those days was anything but sexy.
But it was worthy. Many great things have come from statistics, not least in medicine with the design of clinical trials and drug development. Politics, business and, of course, market research agencies all use statistics. Nate Silver’s much celebrated US election prediction was built on the bedrock of political polling data. It is now the volume of data and, more importantly, its accessibility that has made it sexy. Having people who can model, analyse, interpret and visualise data are very much in demand. But while data visualisation takes centre stage and all the adulation, it is statistics with its algorithms and models that look on paternally, sometimes wistfully, from the wings.
Applying statistics to trends to create predictive models has been around for a long time. The arrival of “big data” is supposed to enhance our modelling capabilities. I’m a bit wary of this. We have to be careful of the lies, damned lies and statistics. You can make the data say anything you want to, which is a worry. We must never accept data at face value and should always verify our findings. It is all too tempting to extract something from a dataset that looks interesting and shout about it. I come from the school of researchers that say if something in the data looks really interesting then it is likely to be wrong.
So, the time for statisticians to rejoice is now. I will shout on the train that I’m a statistician and watch all the ladies swoon.