jump to navigation

What types of common data problems are found in your master data? January 13, 2008

Posted by Peter Benza in Data Analysis, Data Assessment, Data Governance, Data Hygiene, Data Metrics, Data Profiling, Data Quality.
Tags: , ,
3 comments

Master Data exists across your entire enterprise.  Companies today are assessing what is the best way to consolidate all their information assets (data sources) into a “single customer view”.

What types of data problems exist in your organization today or the future with the move towards managing data at the enterprise level?

[Be first to answer this question]

Advertisements

Today’s Linkedin Discussion Thread: Enterprise Data Quality April 28, 2009

Posted by Peter Benza in Data Analysis, Data Elements, Data Governance, Data Optimization, Data Processes, Data Profiling, Data Quality, Data Sources, Data Standardization, Data Synchronization, Data Tools, Data Verification.
Tags: ,
add a comment

Here is my most recent question I just added to my Linkedin discussion group = Enterprise Data Quality.

QUESTION: What master data or existing “traditional” data management processes (or differentiators) have you identified to be useful across the enterprise regarding data quality?

MY INSIGHTS: Recently, I was able to demonstrate (and quantify) the impact of using an NCOA updated address for match/merge accuracy purposes when two or more customer “names and addresses” from three disparate source systems were present. The ultimate test approach warrants consideration especially when talking about the volume of customer records for big companies today number “hundreds” of millions of records. It is ideal to apply this test to the entire file not just a sample set. But, we all know today its about: money, time, value, resources, etc.

For testing purposes, I advised all individual customer address attributes were replaced (where information was available) with NCOA updated addresses and then loaded and processed through the “customer hub” technology. If you are not testing a piece of technology, then constructing your own match key or visually checking sample sets of customer records before and after is an alternative. Either way, inventory matches and non-matches from the two different runs – once with addresses (as-is) and once with addresses that leverage the NCOA information.

My goal was to establish a business process that focused on “pre-processing customer records” using a reliable third party source (in this case NCOA) instead of becoming completely dependent on a current or future piece of technology that may offer the same results, especially when the methodology (matching algorithms) are probalistic. My approach reduces your dependency, as well, and you can focus on “lift” the technology may offer – if your are comparing two or more products.

Where as, inside a deterministic-based matching utility (or off-the-shelf solution) adding extra space or columns of data to the end of your input file to store the NCOA addresses will allow you to accomplish the same results. But, for test purposes, the easier way may be to replace addresses where an NCOA record is available.

Remember, based on the volume of records your client may be dealing with, a pre-process (business process) may be ideal, rather than loading all the customer names and addresses into the third party customer hub technology and processing it. Caution: This all depends on how the business is required (i.e. compliance) to store information from cradle to grave. But, the rule of thumb of the MDM customer hub is to store the “best/master” (single customer view record) with the exception of users with extended search requirements. The data warehouse (vs. MDM solutions) now becomes the next challenge… what to keep where and how much. But, that is another discussion.

The percentage realized in using the updated customer address was substantial (over 10%) on the average based on all the sources factored into the analysis. This means several 10’s of millions of customer records will match/merge more effectively (and efficiently) followed by the incremental lift – based on what the “customer hub” technology enables using its proprietary tools and techniques. This becomes the real differentiator!

Keynote for MDM SUMMIT Fall 2008 in NYC April 25, 2009

Posted by Peter Benza in 1, Data Governance, Data Hygiene, Data Integration, Data Management, Data Stewardship, Linkedin.
Tags: , , ,
add a comment

As part of some ongoing research I have been conducting, here is a slide deck I came across from 2008 – the keynote for the CDI-MDM Summit this past fall in NYC. Re: Key Trends

Dots On A Map Improve Data Quality April 18, 2009

Posted by Peter Benza in Data Accuracy, Data Hygiene, Data Integrity, Data Management, Data Mining, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Types, Data Visualization, Linkedin.
Tags: , , ,
add a comment

This was a presentation I originally prepared back in 2005, but is probably even more applicable in 2009 given the impact using a GIS tool can have on visualizing data quality – customer addresses on  a map! The next time you conduct a customer “data” assessment – try this!

Data Quality and Master Data Initiatives March 31, 2009

Posted by Peter Benza in Data Accuracy, Data Integration, Data Integrity, Data Profiling, Data Quality, Data Sources.
Tags: , , , , , ,
1 comment so far

Initiatives related to master data continues to be on the radar of major corporations especially as it relates to data quality and other mission critical business processes across the enterprise that impact or relies on the quality of data being complete, accurate, and up-to-date.

What other MDM initiatives (besides Data Quality) are also paramount as part of centralizing master data for single customer view purposes.

Lets start a list:

1.) Data Profiling

2.) Data Integration

3.) Match Accuracy

4.) MDM Tools

5.) ???

What is the major difference between structured and unstructured data? December 27, 2008

Posted by Peter Benza in Data Dictionary, Data Elements, Data Formats, Data Types.
add a comment

A good rule of thumb is structured “tabular” data fits into rows and columns and unstructured data are things like web pages, presentations, survey’s, and images.

[Add more examples here.]

What tool or service do you license/subscribe to for geocoding? December 27, 2008

Posted by Peter Benza in Data Accuracy, Data Tools.
add a comment

Where Is Cognizant Showing This Year? October 23, 2008

Posted by Peter Benza in 1, Data Governance, Data Hygiene, Data Integration, Data Management, Data Profiling, Data Quality.
add a comment

Here are some of tradeshows Cognizant has been attending:

http://www.cognizant.com/html/news/events.asp

Do you use Linkedin ? April 23, 2008

Posted by Peter Benza in Data Governance, Data Hygiene, Data Management, Data Profiling, Data Quality, Data Tools.
Tags: , , ,
add a comment

If you are interested in Enterprise Data Quality and want to network with other people that have similar professional interests or skills… Click on the link below and submit your name for review.  A linkedin account is required to join this network group.

http://www.linkedin.com/e/gis/67375/1F86E04EE32D

 

Enterprise Information Management Institute (EIMI) April 23, 2008

Posted by Peter Benza in Data Governance, Data Integration, Data Management, Data Profiling, Data Quality, Data Tools.
Tags: , , ,
add a comment

What to learn more about EIM?

http://www.eiminstitute.org/

Here is an extract from the website – ABOUT EIMI

About EIMI

The Enterprise Information Management Institute (EIMI)’s purpose is to provide data management professionals with the most comprehensive knowledge portal and access to the industry’s most respected thought leaders on managing enterprise information assets. EIMI features a monthly electronic magazine, EIMInsight, including regular monthly columns by David Marco, John Zachman, Sid Adelman, Len Silverston, Anne Marie Smith, Larrisa Moss, Mike Jennings, and Richard Wang, with contributions by Bill Inmon.

Human Inference – an international data quality solution provider February 11, 2008

Posted by Peter Benza in Data Governance, Data Hygiene, Data Integrity, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality.
add a comment

From the website:

Human Inference discovered that to reach the desired results, mathematical logic is not sufficient. The knowledge about the language and culture of a country was necessary as well. Human Inference proved to be right, since today the largest companies of the world are using our knowledge-based software to improve the quality of their data.

http://humaninference.com/services/professional-services/data-quality–assessment/

Cognos data quality rapid assessment service January 17, 2008

Posted by Peter Benza in Data Accuracy, Data Analysis, Data Governance, Data Integration, Data Management, Data Metrics, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Tools.
add a comment

http://www.cognos.com/performance-management/technology/data-quality/pdfs/fs-cognos-data-quality-rapid-assessment-service.pdf

DataFlux positioned in leaders quadrant for data quality according to Gartner January 17, 2008

Posted by Peter Benza in Data Assessment, Data Consolidation, Data Governance, Data Hygiene, Data Integration, Data Integrity, Data Management, Data Profiling, Data Quality, Data Standardization, Data Templates, Data Tools.
Tags: , , ,
add a comment

Compliance, Data Governance, Master Data Management, Data Profiling

http://www.dataflux.com/

BusinessObjects data quality XI January 17, 2008

Posted by Peter Benza in Data Accuracy, Data Analysis, Data Architecture, Data Assessment, Data Consolidation, Data Hygiene, Data Integrity, Data Profiling, Data Quality, Data References, Data Strategy, Data Templates, Data Tools.
Tags: , , ,
add a comment

Standardize, Identify Duplicates, Correct, Improve Match, Append, Consolidate, and more.    

http://www.businessobjects.com/products/dataquality/data_quality_xi.asp

SOA Governance At Bea: Essential to your enterprise transformation strategy January 17, 2008

Posted by Peter Benza in 1, Data Analysis, Data Architecture, Data Governance, Data Integration, Data Management, Data Optimization, Data Profiling, Data Security, Data Stewardship.
Tags: , ,
1 comment so far

Effective SOA governance is an essential element in any enterprise transformation strategy. It can help your organization achieve measurable, sustainable business value.

Read about this and other webcasts, whitepapers, etc… at Bea.

http://www.bea.com/framework.jsp?CNT=index.jsp&FP=/content/solutions/soa_governance/

Innovative Systems, Inc – data quality assessment tool January 14, 2008

Posted by Peter Benza in Data Assessment, Data Errors, Data Governance, Data Hygiene, Data Metrics, Data Processes, Data Profiling, Data Tools.
Tags: , ,
1 comment so far

http://www.innovativesystems.com/services/data_quality_assessment.php

The Innovative Data Quality Assessment provides a quick and economical evaluation of the quality of your customer information. It identifies areas where your information may be enhanced or improved, and quantifies the impact of the defined data quality issues in terms of costs, customer service, lost revenues, etc. It also benchmarks your organization’s data quality against industry standards, showing how your data quality compares to others in your industry.

TDWI’s class outline on data quality assessment January 13, 2008

Posted by Peter Benza in Data Aggregates, Data Assessment, Data Consistancy, Data Profiling.
Tags: , , ,
add a comment

http://www.tdwi.org/education/conferences/lasvegas2008/sessions2.aspx?session_code=4

This course gives comprehensive treatment to the process and practical challenges of data quality assessment. It starts with the systematic treatment of various data quality rules, and proceeds to the results analysis and building of an aggregated data quality scorecard. Special attention is given to the architecture and functionality of the data quality metadata warehouse.

What type of data quality reports does your organization publish on a regular basis? January 13, 2008

Posted by Peter Benza in Data Assessment, Data Hygiene, Data Management, Data Profiling, Data Quality.
1 comment so far

[be first to answer this question]

Summit 2008 – San Francisco January 10, 2008

Posted by Peter Benza in Data Accuracy, Data Governance, Data Integrity, Data Metrics, Data Processes, Data Quality, Data Stewardship, Data Strategy, Data Templates, Data Verification, Data Warehouse.
Tags: , ,
add a comment

If you have not attended a Summit then mark your calendars for:

CDI-MDM Summit Spring 2008

Please post and share your comments about this upcoming summit or if you have not attended and want to learn more then link using the above reference.

MDM Accelerator® by Zoomix January 9, 2008

Posted by Peter Benza in Data Accuracy, Data Aggregates, Data Analysis, Data Assessment, Data Consolidation, Data Dictionary, Data Formats, Data Governance, Data Hygiene, Data Integration, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality, Data References, Data Sources, Data Standardization, Data Stewardship, Data Synchronization, Data Templates, Data Tools.
add a comment

To learn more about or post your comments about MDM Accelerator®

by Zoomix.

http://www.zoomix.com/mdm.asp

Teradata – Master Data Management January 9, 2008

Posted by Peter Benza in Data Assessment, Data Consolidation, Data Dictionary, Data Governance, Data Hygiene, Data Integration, Data Management, Data Metrics, Data Processes, Data Profiling, Data Quality, Data Standardization, Data Stewardship, Data Strategy, Data Templates, Data Tools, Data Types.
add a comment

To learn more about Teradata and their MDM solution offering:

http://www.teradata.com/master-data-management

International association of information and data quality website September 19, 2007

Posted by Peter Benza in Data Governance, Data Hygiene, Data Management, Data Quality, Data Stewardship.
add a comment

www.iaidq.org

Firstlogic’s data quality blog launches in 2005 September 19, 2007

Posted by Peter Benza in Data Hygiene, Data Management, Data Quality.
add a comment

http://findarticles.com/p/articles/mi_pwwi/is_200507/ai_n14753212

Purisma delivers breakthrough solutions for MDM September 19, 2007

Posted by Peter Benza in Data Governance, Data Hygiene, Data Integration, Data Management, Data Quality, Data Variables, Data Warehouse.
1 comment so far

Here is an partial extract from this announcement:  

(see link below to read the entire article)

Bob Hagenau, Cofounder and VP of Products and Corporate Development for Purisma, recently participated in an exclusive interview with Claudia Imhoff and the Business Intelligence Network ( http://www.BeyeNETWORK.com ). In this interview, Hagenau discusses Purisma’s solutions-driven approach to master data management (MDM), new features of Purisma Data Hub 3.0 and Purisma Business Data Appliances — preconfigured MDM applications that allow businesses to solve specific point problems.

http://findarticles.com/p/articles/mi_pwwi/is_200709/ai_n19509669

What other data aggregate functions are useful besides averages and means? September 19, 2007

Posted by Peter Benza in Data Aggregates, Data Consolidation, Data Elements, Data Errors, Data Research.
1 comment so far

(Be first to author an article on this topic.)

What is the first piece of advice you would give someone starting a data governance program? September 18, 2007

Posted by Peter Benza in Data Governance, Data Management, Data Metrics, Data Stewardship, Data Strategy.
1 comment so far

(Be first to author an article on this topic.)

Incorporating data quality into your business, technical, and regulatory standards September 18, 2007

Posted by Peter Benza in Data Profiling, Data Quality, Data Research, Data Warehouse.
add a comment

This white paper describes how application developers can incorporate data quality into their Microsoft SQL Server 2005 Integration Services solutions. (22 printed pages)

Here is an excerpt from the beginning of this paper:

The quality of the data that is used by a business is a measure of how well its organizational data practices satisfy business, technical, and regulatory standards. Organizations with high data quality use data as a valuable competitive asset to increase efficiency, enhance customer service, and drive profitability. Alternatively, organizations with poor data quality spend time working with conflicting reports and flawed business plans, resulting in erroneous decisions that are made with outdated, inconsistent, and invalid data.

For the rest of this article:

http://technet.microsoft.com/en-us/library/aa964137.aspx

The link between Data Quality and Data Governance September 18, 2007

Posted by Dan Power in Data Governance, Data Quality, Data Stewardship.
3 comments

I suppose it’s possible to have an enterprise data quality program without having a data governance program.  But I think having an enterprise-wide data governance program which includes data quality as one of its core focus areas makes more sense.

Data quality without data governance means having to correct the same errors over and over again, and not fixing the problem at the source. But data quality with data governance means bringing the data governance council up to speed on the issue, inviting the different business owners to present their views, and then defining a solution that will be funded, supported and enforced.

Data quality without data governance can be a “solution in search of a problem” – where an IT department adopts data quality tools without directly involving the business, and applies them “in the back room” so to speak.  But data quality tools within the context of a corporate data governance program means the business is fully involved from the beginning, from forming the data governance council, to staffing the data management organization with business data stewards, to providing some or all of the funding, etc.

I’ve been encouraging clients lately to start by defining a data governance program, then to build the data governance council and the attending processes, then to start thinking about individual projects underneath the banner of the data governance program.  That could include projects like implementing specific data quality tools or a Master Data Management hub.

The holistic approach that I generally recommend recognizes that for most business problems, there’s going to be:

  • a People aspect (requiring organizational change, with impacts on corporate culture, accompanied by a political dimension)
  • a Process aspect (as the organization realizes that treating information as a true corporate asset means some new work is going to have to be done by someone)
  • a Technology aspect (usually requiring new platforms, applications or tools)
  • an Information aspect (enriching internal data with trusted external content, for example)

This holistic approach to data governance usually involves wholesale improvements in data quality and in integration. If you want to better capture, store, manage and use information, you’re going to need better data entry procedures, better integration between applications and databases, and an efficient, automated way to find & correct incorrent data.

But you’re never going to be able to get away from the People side of things.  People need to be convinced of the importance of data quality as an issue, and of the need for a comprehensive data governance framework as the best long term way to fix data quality problems.

Spatial data layers and conflation September 18, 2007

Posted by Peter Benza in Data Accuracy, Data Elements, Data Formats, Data Types, Data Visualization.
add a comment

Conflation is more than matching features from different spatial sources.  A good spatial-matching technology that includes conflation as a parameter should also be defined by location, the shapes attributes, and its relationships to other objects. 

A good example of this is when two or more road networks have conflicting views – how do you proceed, if you end up only wanting to display one of the sources? 

What geometrical matching techniques or advice do you have on this topic?

10 Critical factors for successful enterprise data quality September 16, 2007

Posted by Peter Benza in Data Governance, Data Integration, Data Management, Data Profiling, Data Quality, Data Strategy, Data Warehouse.
add a comment

These 10 points were written by Len Dubois, VP of Marketing for Trillium Software over five years ago and his guidance still holds true today in September 2007. 

I recommend, if you need a refresher or want to read this article for the first time – its time well spent.  I have met Len in person – be sure to look for him at the next data quality trade show – you will be pleased you did.

http://www.tdwi.org/research/display.aspx?ID=6341

Cognos to resell Informatica’s data quality software September 16, 2007

Posted by Peter Benza in Data Assessment, Data Management, Data Metrics, Data Profiling, Data Quality, Data Strategy.
1 comment so far

The services are grounded on a data quality assessment summary that comprises quality reports, best practices, and recommendations.

http://www.cbronline.com/article_news.asp?guid=63BF0934-3590-4C07-9E4D-02D7BC7CB060

Data quality connector for Siebel by Group 1 Software September 16, 2007

Posted by Peter Benza in Data Hygiene, Data Integration, Data Management, Data Profiling, Data Quality, Data Standardization.
1 comment so far

See this podcast immediately by linking to the demo below:

http://www.g1.com/Resources/Demos/DQC/index.html

What metrics do you use to monitor data quality in your organization? September 16, 2007

Posted by Peter Benza in Data Metrics, Data Quality, Data Variables.
add a comment

There are many different metrics to consider, but maybe the first question to ask yourself is – did you talk to the business users?

The results of these discussions should be captured and used to formulate business rules.  These business rules will hold the clue to what data variables (or metrics) are most important to monitor/track/graph.

Online data gathering – great resource for surveys and business forms August 25, 2007

Posted by Peter Benza in Data Analysis, Data References, Data Templates, Data Tools, Data Warehouse.
add a comment

I came across this website and after reading what it does I just had to share it.  The responses can be exported to a excel or word file – even add your response and form data into a data warehouse – an Access file will be downloaded that will allow you to do further analysis, if you desire.  

Visit www.askget.com to learn more about this online data gathering tool. 

Malcolm Chisholm, President, Askget.com, Holmdel, NJ

Upcoming information quality and data management tradeshows August 25, 2007

Posted by Peter Benza in Data Integration, Data Management, Data Profiling, Data Quality, Data Research, Data Tools.
add a comment

Europe’s Most Authoritative
Data Management and Information Quality Conferences

29 October – 1 November 2007 • London, UK
Victoria Park Plaza Hotel

This year there are three major shows in one: Information Quality, DAMA International, and Meta Data.  (October 29, 2007 – November 1, 2007)

http://www.irmuk.co.uk/dm2007/

How do you define/differentiate between CDI, PIM, and MDM? August 25, 2007

Posted by Peter Benza in Data Integration, Data Management, Data Modeling, Data Strategy.
1 comment so far

(Be first to author your answer to this subject.)

What data variable(s) are useful to determine when a customer record should be classified as active or inactive? August 25, 2007

Posted by Peter Benza in Data Analysis, Data Management, Data Metrics, Data Mining, Data Processes, Data Research, Data Variables.
2 comments

Be the first to author a comment on this subject.

White papers on data quality August 25, 2007

Posted by Peter Benza in Data Profiling, Data Quality, Data Tools.
add a comment

Here is a link to a variety of white papers on data quality and data profiling from one of the leaders in the industry.  You will need to register, but it’s easy to do.

http://www.trilliumsoftware.com/site/content/resources/library/index.asp

Please continue to add other company links who have white papers on data quality for easy reference in the future. 

NCDM tradeshow from early 1990’s August 19, 2007

Posted by Peter Benza in Data Elements, Data Management, Data Mining, Data Profiling.
add a comment

ncdm_wagon2.jpg

Here is a old snapshot I found from a database marketing tradeshow I attended back in the early 1990’s in Orlando, FL. 

Data quality in the 1990’s equated to postal name and address information – address standardization, zip correction, phone number verification, first name tables, apartment number enhancements, area code changes, and probably the biggest – National Change of Address.  Today, data quality has expanded to include product data, item data, financial data, and other master data sources across the enterprise.

Service bureau’s like IRA (at that time) were just one of a few bureau’s remaining that were privately held who mass-compiled consumer data on a national basis and collected information like exact age, phone numbers, length of residence, dwelling unit type, dwelling unit size, height/weight information, voting preference… the list goes on!

Today, with the evolution of database technology, consumer data used as reference data, statistical analysis, and advanced data profiling tools – the database marketing industry has truely taken all these subject area’s to the next level. 

Best practices for database management, data quality, and data governance are now prime time and instead of organization just concentrating on how to cut costs (more) – they want to shift to increasing revenues – and to do that it begins with leveraging corporate data sources across the enterprise.

What department should data stewards reside in… IT or business? August 19, 2007

Posted by Peter Benza in Data Governance, Data Stewardship.
1 comment so far

Be first to author an article on this subject.