With the release of Magnitude MDM 10 SP1 on October 4, 2018, we have completely rearchitected the way we manage and process data. What does this mean to the user? Read on….
Usually, when a software product is as extensively revised as MDM 10 SP1, users are faced with a choice: upgrade and face having to change many procedures to work with the new release or pass on the upgrade and forgo the new features. This is not true of Magnitude MDM – despite a major architectural change, there has been NO loss or change in basic function. In other words, what worked before, will still work without the need for any rework…it will just be faster.
To understand what we’ve changed and why, we need to go back to the early days of Kalido MDM, circa 2004. One of the great things about being in this business for so many years is the opportunity to learn from our customers about what they need and how they want to work. But back when we were first building the product, almost no one knew exactly how MDM would be used. In fact, the term ‘MDM’ was not yet in broad use. One of the features we built allowed users to load data without fully describing it. All users needed to do was point us at the name and a unique identifier, and we would parse and load the full record (sort of a simple data survey tool).
To accommodate this ability, we converted all incoming data into an XML string. This allowed us to effective reverse the normal “model then load” process: you could load the data first, and then change the model to match the data.
However, as time went by, MDM evolved from validating and maintaining reference data to more of a data governance function, where incomplete data became more important, and what made up “valid” data changed from being a purely technical definition into a more business context driven metric. This meant not only loading the available data, but also identifying critical attributes that were not a single source load, turning the modeling focus from ‘what do we have?’ to ‘what do we need or want?’.
Over the intervening years, we’ve laid the groundwork to eliminate the use of XML throughout the product. In MDM 10 SP1 it is finally gone. Now, working context data is stored and managed in the output tables, regardless of how it is loaded. When a validation error occurs, we simply flag the invalid record instead of converting it to XML and storing it in a completely different table. For the user, this greatly improves the response time of any activity that involves the working context (loads, validation, edits, workflow, etc.). Most importantly, this enables our customer to implement real-time MDM and supports real time monitoring of your MDM repository.
For more details (and a real-world example) regarding performance improvements, you can watch this 5 minute video
More than just performance
MDM 10 SP1 includes more than just a performance upgrade. For example, SP1 features the production release of our revised Reference Matching feature. This matching solution is designed to link records together, populating the reference attribute defined between the categories. This is different from harmonization, which is designed to match and MERGE records into a single category. Reference matching is designed to match and LINK records in two different categories.
We haven’t forgotten about Harmonization matching either! New in SP1 is Manual Overrides – the ability to change a value in a merged, golden copy record, and keep that value even if the category is re-harmonized. Each override is tracked at the attribute level, meaning that multiple overrides may be placed on a record, but they can be added or cleared on a case by case basis as needed. Of course, you can also clear them in bulk (more on that below). This addresses the situation when a harmonization results in records having missing, incomplete or invalid data (i.e. NO source had a good value). The user can now enter a better value and mark as an override, keeping that value until later changed again, or the override is cleared. Once cleared, the value will update automatically for the next time when survivorship is calculated.
Besides the changes to the matching processes themselves, we have revised the matching stewardship UI to allow users to process pending matches in any order, allow records to be resorted, skip matches and accept or reject matches in bulk. This bulk operation allows overrides to be cleared in bulk as well.
For more information, and to see these new features in action, watch this short video that explains and demonstrates overrides and reference matching.
We hope you find this additional functionality useful! As always, if you have any questions or concerns, please contact your Magnitude account representative or Magnitude Support.