What types of common data problems are found in your master data? January 13, 2008
Posted by Peter Benza in Data Analysis, Data Assessment, Data Governance, Data Hygiene, Data Metrics, Data Profiling, Data Quality.Tags: Data Governance, Data Profiling, Data Quality
trackback
Master Data exists across your entire enterprise. Companies today are assessing what is the best way to consolidate all their information assets (data sources) into a “single customer view”.
What types of data problems exist in your organization today or the future with the move towards managing data at the enterprise level?
[Be first to answer this question]
Enterprise Data Quality Weblog 
I usually divide master data quality problems into 3 categories: Operational, tactical and strategic.
Operational data quality problems are often found in ERP systems and other individual core business systems databases. The problems are things as returned post due to incorrect or outdated addresses, multiple direct mails to same person due to duplicates, wrong credit limits due to missing data and fraud entries. The cost of these problems may often actually be measured – these are direct costs, and you may establish the ROI for fixing these problems. For some companies these costs are very small – for other companies these costs could be very high if not addressed.
Tactical data quality problems are often found in CRM and SCM systems databases covering enterprise wide data. Problems are e.g. failing single view on customer due to duplicates and duplicate definition issues, wrong segmentation again due to duplicates, missing and incorrect data leading to not choosing the best dialogue with customers and prospects. Here the costs can’t be measured directly – the costs are hidden all around in missing sales, missing rebates, complaint handling, rework and other extended costs. Some suggest these costs could be as high as 10-20% of revenue.
Strategic data quality problems are related to BI (Business Intelligence). If you take decisions based on data which actually have data quality problems your decisions may be wrong. You can only guess on the impact from that – but it could be a question of life or dead for a company.
Henrik Liliendahl Sørensen, as you are experiencing data quality problems, you could use a data profiler.
I have been looking for an interesting open source profiler and so far, I have found a few. Talend has an interesting tool: Their open source Talend Open Profiler makes it possible to perform data profiling without being an expert. It is also easy to use, fast to operate and free to download. Talend has an active community able to solve problems and a very dynamic R&D team to quickly implement new features and fix bugs.
You can download Talend Open Profiler from http://www.talend.com/download.php. Hope this helps you on your data quality problems.
With regard to the data problems that exist today, I’d say that the proliferation of web-based applications has produced a data fiefdom effect. Consequently there’s been a dispersion of mission critical customer data throughout, and even external, to the enterprise.
With regard to the data problems that exist in the future, I see the internet again playing a significant role. The rich and varied format of the data being made available via web based mediums will present an even greater challenge to maintaining the unified perspective of a customer.