Big data is everywhere at the moment. There’s a lot of talk about it, much of which presents big data as a problem to be solved rather than as an opportunity to be seized. There are many organisations that could be using big data for powerful predictive analytics but aren’t, because they’ve bought into one of the many myths about it that are in common circulation. Here are five of the big data myths that I come across most frequently in my work
1. Big data is new
The terminology is new (and time will tell if it’s a term we’ll still be using in 5 years’ time) but the concept certainly isn’t new. I’ve worked in the analytics space for approaching twenty years now and there’s nothing new under the sun. Whilst the range and amount of data available in most organisations may have expanded significantly, so has computing power. Many years ago the analysis of 100,000 customer records would have been considered big and analysts with a file that size would have faced many similar challenges to those working with much bigger files today. Big is relative.
2. Big data is a problem
Big data is commonly presented as a problem: How do we make the most of our data? What if we’re not doing the right thing with it? Could we be doing more? I really don’t see it that way. Big data has a lot of potential, even if it’s not currently being used properly, or even at all. So long as the data exists and can be used in a helpful way in future, it’s an opportunity for an organisation rather than a problem. The only problem with the data comes when organisations fail to turn it into something which is both legible and practical for analysis, and this is often a big challenge for companies.
3. You need big data to make use of analytics
When I’m out meeting customers and prospects it’s common to hear this concern: we don’t have enough data to be able to do anything with it. The term ‘big data’ implies that you need a vast datamart to be able to do any useful analytics. But as already discussed, that isn’t the case. Twenty years ago people were doing predictive analytics with datasets that would be considered tiny by today’s big data standards, but they still gained useful insights from them. As long as you have a business objective and some data you should be able to use some form of predictive analytics to learn something useful. I’ve also found that it’s common for organisations to have more data than they think they do. Data can be hidden away in some surprising places – tweets, feedback letters, web analytics – it isn’t just about what’s in your customer transaction database. Studies have shown that most organisation which have made use of big data actually already held the information but only made sense of it once they came up with a sensible question and the analytical means to answer it.
4. There are plenty of people who can use big data
Wrong. This is a big problem, and it’s worldwide. Gartner statistics say that there is such a lack of skilled data scientists, up to 75 percent of the positions required for all companies to make use of big data will go unfilled. Competition is rife. In other words, it’s a great job to be doing right now!
5. Big companies know what they’re doing
Gartner research based on case studies of hundreds of companies estimates that in 2016 85% of fortune 500 companies will still not be able to exploit big data for competitive advantage. In my many years of experience I’ve seen any number of big companies and household names who are just beginning to get started with deep analytics. Not everybody knows what they’re doing. There’s a big competitive advantage to be had by starting soon. Don’t assume your competitors are already doing this – there’s a good chance that they’re not.
So what does all this mean for you?
The real challenge around maximising the opportunity of big data is an age old one and not one specifically related to the size and volume of data. If you don’t use it or it’s not useful in solving a business problem then is it really worth anything? As with all valuable analytics projects an organisation should start with a business goal, look for data which can inform against that business goal and from there use it to look for patterns, relationships, indicators which will help inform processes to address the original business goal.
Having access to more history, data about different aspects of behaviour, more granular information and more up to date information will certainly help to make these business goals easier to solve but please don’t get so hung up on the perils of big data. It’s mostly about storage and accessibility of the useful parts of the information contained in this “big data”. Not dissimilar to the worries of our recent technological past where data marts and data warehouses were the thing to spend your money on to get a single view of the customer. Perhaps it will always be that as computing power and storage capacity and the amount or range of data expands the data will always be just a little bit bigger than we can comfortably manage.
