Bill Franks from Teradata touched on this topic in a recent article I came across on the Agile Data Warehousing site. New technologies that enable the processing of more data in a shorter amount of time are only part of the Big Data equation. You still need the capability to make sense of it all.
While technologies such as MapReduce and Hadoop allow companies to consider larger and broader data sets for analysis, you still need a way to ask the right questions. That means that companies need to employee teams of data scientists, right? Ah, but therein lies the rub. There is an extreme shortage of that valuable asset in the market. So, as companies spend money and IT resources to collect and sift through reams of data, they are doing so without any real, business-value based guidance. It’s like feeling around in the dark to find the light switch – you’re just guessing at what illumination the next query might bring.
What options do companies have as they run headlong into the Big Data fray? Well, they can send out their recruiters to look for the same thing that everyone else is looking for, the rare and elusive data scientist. Or, they can arm their current staff with better tools and methods to begin to learn how to ask the right questions and produce value. I’m not known to be a sit back and wait type of person, and I imagine that most business leaders aren’t known for their patience in waiting for technology to produce value. Taking action now and enabling existing data analysts to take better advantage of current and past investments in data management technologies just makes sense to me.
That Teradata warehouse is loaded with value. That value can be significantly enhanced when the right information from Big Data analysis is merged with it to provide the right context to drive business action. But, data analysts must have the ability to model their analysis, refine the algorithms that are required to ask and answer the tough questions, and to do so at a cadence that outpaces your competition.
Business model driven, complex operations unlock this capability and bring the needed agility to make better use of existing resources and develop the skills of your most valuable assets, your knowledge workers. Or, you can just wait…