The Big Data Conundrum

Another article appeared last week in Forbes on the future of Big Data. No “v” words this time, but a much more practical discussion: how prepared is your IT organization to support big data?

The article quoted a Gartner webcast poll that answered this question, with 54% responding that they don’t know yet what skills they need but they are researching it. Another 31% said they had no plans and will deal with it when they need to. And 0% – zero percent – said they know what they need to do and their staff is coming up to speed. The results of this poll, statistically valid or not, provide a glimpse into how well IT organizations self-evaluate their readiness for Big Data.

Our own informal poll of attendees at Big Data events showed similar trepidation. Thirty-eight percent said their biggest concern with Big Data is that it will require new technologies to learn and support.

Clearly this is due to a lot of uncertainty about how Big Data can help, what business problems can be addressed by it — or whether your company has it or not. On this point, there was another interesting blog post that suggested you probably don’t have Big Data. In his post, Ben Taub at Dataspace suggests that data volumes are the primary driver and that unless the volumes are so large and complex, then most of us probably only have “just plain data.” While he may or may not be correct about this, one thing I do agree with him on is you must start with the user and the business use case. In our survey, 20% said that a line of business was pushing the organization to investigate or deploy Big Data; 40% said IT was the driving force. I was really hoping we’d see the reverse of this, and perhaps we will over time as the Big Data concept matures. Furthermore, the Forbes article predicts that the business units will soon come begging for help from IT, and that Big Data could also have a negative impact on existing governance models.

At Kalido we have encouraged people to think about how to get ready for Big Data. In our survey, 61.5% agreed or strongly agreed that the data warehouse would be a critical foundation for delivering Big Data Analytics. So whether Big Data turns out to be just more data, or it really does include more data sources and data types, there is no time like the present to think about how you’ll model this new data and its relationships, prepare your data warehouse to be more scalable as it plays its role in delivering analytics, and to document your governance policies so they can be ready for the Big Data wave.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply