What on earth does MDM have to do with Big Data?

Well, actually rather a lot. In a recent webinar in which I participated with Virgin Media, the question was raised by one of the attendees as a current requirement. It surprised me a little, as although I have had many theoretical conversations about the relationship between the two disciplines, it was refreshing to hear customers looking to take real action on the subject. This combined with no less than a dozen conversations with large financial services customers, analysts and industry alumni alike, has inspired me to share.

At its highest level, we intuitively know that the majority of the “Big Data” sources tend to capture high volumes of transactions or interactions, often from new and/or 3rd party sources. But in order to make them relevant to one’s organization, the data requires context. That is to say, we need to relate it back to the core entities within one’s organization, such as: products, customers, partners, markets, households, policy holders, agents, etc.

Drilling down a level, here are some of the common patterns of thinking that I believe are being sought after. Firstly, structured data can often be thought of as one (or more) of “Master Data,” “Reference Data,” “Hierarchies,” and “Observances.” An observance typically represents a record (persisted or not) of an activity, event, state or other form of measurement. There are many, many different types of observances, ranging from the traditional sales transactions to a website hit, a tweet, a sensor reading in a car, a cellphone ping of a cell tower, a cable box recording a user flicking channels. When we consider new types of analysis and data, we are most typically concerned with analyzing this exploding number of observances all around us.

But if you have every tried to perform some analysis of these observances, you quickly find you most often need to combine it with other structured master, reference and hierarchical data to “give it context.” Put simply, we need to “link” the observances to master and reference data (sound familiar to any of you Data Warehouses gurus out there?). But if MDM is truly targeted at helping an analyst or data scientist perform some complex analysis, that’s really just the beginning of the requirement.

Every day I see a new class of analytics appear on the market and each time I see a new clustering model or recommendation algorithm, I’m reminded of the old saying “garbage in, garbage out.” The analytic will only ever be as good as the context from which it is derived. The age of analytics is just beginning, we will undoubtedly see some monumental revolutions in this space, but what I’m increasingly hearing from forward-looking customers is an urgency to establish a set of clean context to make these analytics useful and relevant to their organizations.

Big Data might be the elephant in the room, but MDM will be the enabler that makes it useful in the average enterprise.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply