The Age of Big Data

Is it just me, or everywhere you turn do you hear the phrase “Big Data”?  So what does this mean anyway?  (And isn’t this the kind of phrase that should be introduced by the guy who does the overly-dramatic voices for X-Factor?)

“Data” has come of age for several reasons…the Internet, Smartphones and the increasingly connected way in which we work, shop, seek out information and socialise.  Social networking in particular has created forums in which we share online, and make public, more personal data than ever before.

The economic crisis also prompted clients to start looking more closely at the data streams they have available to them and how these could be better used to inform decision making.

Some of the speakers at the recent WARC conference warned of the potential pitfalls of having so much data available, including the fact that anyone who can use a spreadsheet can now start crunching data.

Arguably the art of data analysis is something that has to be learned.  There are many different analysis techniques available to researchers; even the simple average or mean score can be misleading if the distribution is not checked for outliers/anomalies.   Representativeness is another key issue often not discussed.

And at Ipsos MORI data analysis is a skill we invest many hours training our graduates in.  In my experience it is something that also needs to be developed “on the job”, as you learn the old adage that if something looks interesting, double check it first to see if it is wrong.

I think there are three key elements to consider in order to get the most out of your data sources:

1. Understand what business outcomes you want to measure.  What are the important questions you need to answer?  What variables could provide information to feed into these?  Map these, prioritise and organise data to source these answers.
2. Future proof.  Look back to look forward – what was important to your business 10 years ago?  Use this to contextualise and project what could be important in ten years time.
3. Develop hypotheses.  And use these to go looking for answers.

And as a final thought,  I echo Nicola Mendelsohn’s words in her article written for our recent event on [sp]innovation, the research industry needs to re-connect with the advertising and marketing community to pioneer new methods and approaches, without losing the “old skills” of how to interrogate data with due rigour.

Louise Brice is a Research Director at Ipsos ASI