jump to navigation

Today’s Linkedin Discussion Thread: Enterprise Data Quality April 28, 2009

Posted by Peter Benza in Data Analysis, Data Elements, Data Governance, Data Optimization, Data Processes, Data Profiling, Data Quality, Data Sources, Data Standardization, Data Synchronization, Data Tools, Data Verification.
Tags: ,
add a comment

Here is my most recent question I just added to my Linkedin discussion group = Enterprise Data Quality.

QUESTION: What master data or existing “traditional” data management processes (or differentiators) have you identified to be useful across the enterprise regarding data quality?

MY INSIGHTS: Recently, I was able to demonstrate (and quantify) the impact of using an NCOA updated address for match/merge accuracy purposes when two or more customer “names and addresses” from three disparate source systems were present. The ultimate test approach warrants consideration especially when talking about the volume of customer records for big companies today number “hundreds” of millions of records. It is ideal to apply this test to the entire file not just a sample set. But, we all know today its about: money, time, value, resources, etc.

For testing purposes, I advised all individual customer address attributes were replaced (where information was available) with NCOA updated addresses and then loaded and processed through the “customer hub” technology. If you are not testing a piece of technology, then constructing your own match key or visually checking sample sets of customer records before and after is an alternative. Either way, inventory matches and non-matches from the two different runs – once with addresses (as-is) and once with addresses that leverage the NCOA information.

My goal was to establish a business process that focused on “pre-processing customer records” using a reliable third party source (in this case NCOA) instead of becoming completely dependent on a current or future piece of technology that may offer the same results, especially when the methodology (matching algorithms) are probalistic. My approach reduces your dependency, as well, and you can focus on “lift” the technology may offer – if your are comparing two or more products.

Where as, inside a deterministic-based matching utility (or off-the-shelf solution) adding extra space or columns of data to the end of your input file to store the NCOA addresses will allow you to accomplish the same results. But, for test purposes, the easier way may be to replace addresses where an NCOA record is available.

Remember, based on the volume of records your client may be dealing with, a pre-process (business process) may be ideal, rather than loading all the customer names and addresses into the third party customer hub technology and processing it. Caution: This all depends on how the business is required (i.e. compliance) to store information from cradle to grave. But, the rule of thumb of the MDM customer hub is to store the “best/master” (single customer view record) with the exception of users with extended search requirements. The data warehouse (vs. MDM solutions) now becomes the next challenge… what to keep where and how much. But, that is another discussion.

The percentage realized in using the updated customer address was substantial (over 10%) on the average based on all the sources factored into the analysis. This means several 10’s of millions of customer records will match/merge more effectively (and efficiently) followed by the incremental lift – based on what the “customer hub” technology enables using its proprietary tools and techniques. This becomes the real differentiator!

Advertisements

Dots On A Map Improve Data Quality April 18, 2009

Posted by Peter Benza in Data Accuracy, Data Hygiene, Data Integrity, Data Management, Data Mining, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Types, Data Visualization, Linkedin.
Tags: , , ,
add a comment

This was a presentation I originally prepared back in 2005, but is probably even more applicable in 2009 given the impact using a GIS tool can have on visualizing data quality – customer addresses on  a map! The next time you conduct a customer “data” assessment – try this!

Data Quality and Master Data Initiatives March 31, 2009

Posted by Peter Benza in Data Accuracy, Data Integration, Data Integrity, Data Profiling, Data Quality, Data Sources.
Tags: , , , , , ,
1 comment so far

Initiatives related to master data continues to be on the radar of major corporations especially as it relates to data quality and other mission critical business processes across the enterprise that impact or relies on the quality of data being complete, accurate, and up-to-date.

What other MDM initiatives (besides Data Quality) are also paramount as part of centralizing master data for single customer view purposes.

Lets start a list:

1.) Data Profiling

2.) Data Integration

3.) Match Accuracy

4.) MDM Tools

5.) ???

Where Is Cognizant Showing This Year? October 23, 2008

Posted by Peter Benza in 1, Data Governance, Data Hygiene, Data Integration, Data Management, Data Profiling, Data Quality.
add a comment

Here are some of tradeshows Cognizant has been attending:

http://www.cognizant.com/html/news/events.asp

Do you use Linkedin ? April 23, 2008

Posted by Peter Benza in Data Governance, Data Hygiene, Data Management, Data Profiling, Data Quality, Data Tools.
Tags: , , ,
add a comment

If you are interested in Enterprise Data Quality and want to network with other people that have similar professional interests or skills… Click on the link below and submit your name for review.  A linkedin account is required to join this network group.

http://www.linkedin.com/e/gis/67375/1F86E04EE32D

 

Enterprise Information Management Institute (EIMI) April 23, 2008

Posted by Peter Benza in Data Governance, Data Integration, Data Management, Data Profiling, Data Quality, Data Tools.
Tags: , , ,
add a comment

What to learn more about EIM?

http://www.eiminstitute.org/

Here is an extract from the website – ABOUT EIMI

About EIMI

The Enterprise Information Management Institute (EIMI)’s purpose is to provide data management professionals with the most comprehensive knowledge portal and access to the industry’s most respected thought leaders on managing enterprise information assets. EIMI features a monthly electronic magazine, EIMInsight, including regular monthly columns by David Marco, John Zachman, Sid Adelman, Len Silverston, Anne Marie Smith, Larrisa Moss, Mike Jennings, and Richard Wang, with contributions by Bill Inmon.

Human Inference – an international data quality solution provider February 11, 2008

Posted by Peter Benza in Data Governance, Data Hygiene, Data Integrity, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality.
add a comment

From the website:

Human Inference discovered that to reach the desired results, mathematical logic is not sufficient. The knowledge about the language and culture of a country was necessary as well. Human Inference proved to be right, since today the largest companies of the world are using our knowledge-based software to improve the quality of their data.

http://humaninference.com/services/professional-services/data-quality–assessment/

Cognos data quality rapid assessment service January 17, 2008

Posted by Peter Benza in Data Accuracy, Data Analysis, Data Governance, Data Integration, Data Management, Data Metrics, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Tools.
add a comment

http://www.cognos.com/performance-management/technology/data-quality/pdfs/fs-cognos-data-quality-rapid-assessment-service.pdf

DataFlux positioned in leaders quadrant for data quality according to Gartner January 17, 2008

Posted by Peter Benza in Data Assessment, Data Consolidation, Data Governance, Data Hygiene, Data Integration, Data Integrity, Data Management, Data Profiling, Data Quality, Data Standardization, Data Templates, Data Tools.
Tags: , , ,
add a comment

Compliance, Data Governance, Master Data Management, Data Profiling

http://www.dataflux.com/

BusinessObjects data quality XI January 17, 2008

Posted by Peter Benza in Data Accuracy, Data Analysis, Data Architecture, Data Assessment, Data Consolidation, Data Hygiene, Data Integrity, Data Profiling, Data Quality, Data References, Data Strategy, Data Templates, Data Tools.
Tags: , , ,
add a comment

Standardize, Identify Duplicates, Correct, Improve Match, Append, Consolidate, and more.    

http://www.businessobjects.com/products/dataquality/data_quality_xi.asp

SOA Governance At Bea: Essential to your enterprise transformation strategy January 17, 2008

Posted by Peter Benza in 1, Data Analysis, Data Architecture, Data Governance, Data Integration, Data Management, Data Optimization, Data Profiling, Data Security, Data Stewardship.
Tags: , ,
1 comment so far

Effective SOA governance is an essential element in any enterprise transformation strategy. It can help your organization achieve measurable, sustainable business value.

Read about this and other webcasts, whitepapers, etc… at Bea.

http://www.bea.com/framework.jsp?CNT=index.jsp&FP=/content/solutions/soa_governance/

Innovative Systems, Inc – data quality assessment tool January 14, 2008

Posted by Peter Benza in Data Assessment, Data Errors, Data Governance, Data Hygiene, Data Metrics, Data Processes, Data Profiling, Data Tools.
Tags: , ,
1 comment so far

http://www.innovativesystems.com/services/data_quality_assessment.php

The Innovative Data Quality Assessment provides a quick and economical evaluation of the quality of your customer information. It identifies areas where your information may be enhanced or improved, and quantifies the impact of the defined data quality issues in terms of costs, customer service, lost revenues, etc. It also benchmarks your organization’s data quality against industry standards, showing how your data quality compares to others in your industry.

TDWI’s class outline on data quality assessment January 13, 2008

Posted by Peter Benza in Data Aggregates, Data Assessment, Data Consistancy, Data Profiling.
Tags: , , ,
add a comment

http://www.tdwi.org/education/conferences/lasvegas2008/sessions2.aspx?session_code=4

This course gives comprehensive treatment to the process and practical challenges of data quality assessment. It starts with the systematic treatment of various data quality rules, and proceeds to the results analysis and building of an aggregated data quality scorecard. Special attention is given to the architecture and functionality of the data quality metadata warehouse.

What types of common data problems are found in your master data? January 13, 2008

Posted by Peter Benza in Data Analysis, Data Assessment, Data Governance, Data Hygiene, Data Metrics, Data Profiling, Data Quality.
Tags: , ,
3 comments

Master Data exists across your entire enterprise.  Companies today are assessing what is the best way to consolidate all their information assets (data sources) into a “single customer view”.

What types of data problems exist in your organization today or the future with the move towards managing data at the enterprise level?

[Be first to answer this question]

What type of data quality reports does your organization publish on a regular basis? January 13, 2008

Posted by Peter Benza in Data Assessment, Data Hygiene, Data Management, Data Profiling, Data Quality.
1 comment so far

[be first to answer this question]

MDM Accelerator® by Zoomix January 9, 2008

Posted by Peter Benza in Data Accuracy, Data Aggregates, Data Analysis, Data Assessment, Data Consolidation, Data Dictionary, Data Formats, Data Governance, Data Hygiene, Data Integration, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality, Data References, Data Sources, Data Standardization, Data Stewardship, Data Synchronization, Data Templates, Data Tools.
add a comment

To learn more about or post your comments about MDM Accelerator®

by Zoomix.

http://www.zoomix.com/mdm.asp

Teradata – Master Data Management January 9, 2008

Posted by Peter Benza in Data Assessment, Data Consolidation, Data Dictionary, Data Governance, Data Hygiene, Data Integration, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Strategy, Data Templates, Data Tools, Data Types.
add a comment

To learn more about Teradata and their MDM solution offering:

http://www.teradata.com/master-data-management

Incorporating data quality into your business, technical, and regulatory standards September 18, 2007

Posted by Peter Benza in Data Profiling, Data Quality, Data Research, Data Warehouse.
add a comment

This white paper describes how application developers can incorporate data quality into their Microsoft SQL Server 2005 Integration Services solutions. (22 printed pages)

Here is an excerpt from the beginning of this paper:

The quality of the data that is used by a business is a measure of how well its organizational data practices satisfy business, technical, and regulatory standards. Organizations with high data quality use data as a valuable competitive asset to increase efficiency, enhance customer service, and drive profitability. Alternatively, organizations with poor data quality spend time working with conflicting reports and flawed business plans, resulting in erroneous decisions that are made with outdated, inconsistent, and invalid data.

For the rest of this article:

http://technet.microsoft.com/en-us/library/aa964137.aspx

10 Critical factors for successful enterprise data quality September 16, 2007

Posted by Peter Benza in Data Governance, Data Integration, Data Management, Data Profiling, Data Quality, Data Strategy, Data Warehouse.
add a comment

These 10 points were written by Len Dubois, VP of Marketing for Trillium Software over five years ago and his guidance still holds true today in September 2007. 

I recommend, if you need a refresher or want to read this article for the first time – its time well spent.  I have met Len in person – be sure to look for him at the next data quality trade show – you will be pleased you did.

http://www.tdwi.org/research/display.aspx?ID=6341

Cognos to resell Informatica’s data quality software September 16, 2007

Posted by Peter Benza in Data Assessment, Data Management, Data Metrics, Data Profiling, Data Quality, Data Strategy.
1 comment so far

The services are grounded on a data quality assessment summary that comprises quality reports, best practices, and recommendations.

http://www.cbronline.com/article_news.asp?guid=63BF0934-3590-4C07-9E4D-02D7BC7CB060

Data quality connector for Siebel by Group 1 Software September 16, 2007

Posted by Peter Benza in Data Hygiene, Data Integration, Data Management, Data Profiling, Data Quality, Data Standardization.
1 comment so far

See this podcast immediately by linking to the demo below:

http://www.g1.com/Resources/Demos/DQC/index.html

Upcoming information quality and data management tradeshows August 25, 2007

Posted by Peter Benza in Data Integration, Data Management, Data Profiling, Data Quality, Data Research, Data Tools.
add a comment

Europe’s Most Authoritative
Data Management and Information Quality Conferences

29 October – 1 November 2007 • London, UK
Victoria Park Plaza Hotel

This year there are three major shows in one: Information Quality, DAMA International, and Meta Data.  (October 29, 2007 – November 1, 2007)

http://www.irmuk.co.uk/dm2007/

White papers on data quality August 25, 2007

Posted by Peter Benza in Data Profiling, Data Quality, Data Tools.
add a comment

Here is a link to a variety of white papers on data quality and data profiling from one of the leaders in the industry.  You will need to register, but it’s easy to do.

http://www.trilliumsoftware.com/site/content/resources/library/index.asp

Please continue to add other company links who have white papers on data quality for easy reference in the future. 

NCDM tradeshow from early 1990’s August 19, 2007

Posted by Peter Benza in Data Elements, Data Management, Data Mining, Data Profiling.
add a comment

ncdm_wagon2.jpg

Here is a old snapshot I found from a database marketing tradeshow I attended back in the early 1990’s in Orlando, FL. 

Data quality in the 1990’s equated to postal name and address information – address standardization, zip correction, phone number verification, first name tables, apartment number enhancements, area code changes, and probably the biggest – National Change of Address.  Today, data quality has expanded to include product data, item data, financial data, and other master data sources across the enterprise.

Service bureau’s like IRA (at that time) were just one of a few bureau’s remaining that were privately held who mass-compiled consumer data on a national basis and collected information like exact age, phone numbers, length of residence, dwelling unit type, dwelling unit size, height/weight information, voting preference… the list goes on!

Today, with the evolution of database technology, consumer data used as reference data, statistical analysis, and advanced data profiling tools – the database marketing industry has truely taken all these subject area’s to the next level. 

Best practices for database management, data quality, and data governance are now prime time and instead of organization just concentrating on how to cut costs (more) – they want to shift to increasing revenues – and to do that it begins with leveraging corporate data sources across the enterprise.

Data quality and plotting customer address data on a map August 19, 2007

Posted by Peter Benza in Data Analysis, Data Hygiene, Data Integration, Data Metrics, Data Profiling, Data Quality, Data Tools.
add a comment

Consider the insights and knowledge your organization will gain about the quality of its customer name/address data prior to centralizing all the desparate data sources into one location.  Here is a actual slide deck I prepared a few years ago using the output from my analysis to illustrate how maps and data profiling can assist in assessing data quality. 

How can data profiling and a meta data repository be leveraged together? August 17, 2007

Posted by Peter Benza in Data Modeling, Data Processes, Data Profiling.
add a comment

Many times the same term used by one department means something totally different to another department.  This can prove to be a challenge as organizations continue to centralize all their customer data into one location.  You may not be able to resolve all the different name variations used by each department, but assembling all the pieces and documenting them in one place is a must.  It may become necessary to follow-up with the appropriate decision makers to resolve any discreptencies.

So, it becomes mission-critical to compile “data about your data” and store it in a meta data repository, plus include some other key attributes about the data source, about each variable, its range of values, record length, and so on.  Ultimately, the data elements need to be analyzed and merged into a single classification system based on all relevant data sources from across the enterprise.

This (meta data) information will also become valuable guide to validate other data-specific activities, such as: customer data modeling, match/merge logic, and even for QC purposes during the integration/execution phase of storing the resulting customer information in one location.

Meta Data: http://en.wikipedia.org/wiki/Metadata

What data quality dimensions can statisticans impact by using a data profiling tool. August 16, 2007

Posted by Peter Benza in Data Accuracy, Data Completeness, Data Profiling, Data Quality, Data Tools.
add a comment

Two data quality dimensions that statisticans can play a role in is data accuracy and data completeness.  A data profiling tool comes in handy to facilitate the actual research required by the organization. 

What data profiling tool does your organization use?

What other data quality dimensions can be analyzed? 

Peter Benza – 1984 graduate of the direct marketing educational foundation – creates enterprise data quality weblog August 13, 2007

Posted by Peter Benza in Data Elements, Data Governance, Data Integrity, Data Management, Data Mining, Data Optimization, Data Profiling, Data Quality, Data Stewardship, Data Strategy, Data Tools, Data Variables, Data Visualization.
add a comment