Data Quality & the CDO

Data Quality is a concern of everyone in the Big Data space. The quantity of data is growing at a geometric pace (increased volume, velocity and variety — the very description of Big Data). According to IBM, 90% of all data was created in the past 2 years. Therefore, making sense of all of this data starts with the need for assurance that the data is accurate — a very real concern.

And in the past few years, there’s a new sheriff in town — a new position that has risen in the corporate ranks at the C‑level to deal with data. It is the CDO.

According to Wikipedia, “the Chief Data Officer (CDO) is a corporate officer responsible for enterprise wide governance and utilization of information as an asset, via data processing, analysis, data mining, information trading and other means. CDOs report mainly to the Chief Executive Officer (CEO).” And according to to NewVantage Partners 2016 Big Data Executive Survey, 54% of firms surveyed now report having appointed a Chief Data Officer.

The Chief Data Officer now has the critical job of setting policies and implementing procedures around data strategy, data governance, and regulatory compliance. And therefore, for providing a strategy for assuring data quality organization-wide.

Gartner just released its much anticipated Magic Quadrant for Data Quality.

According to Gartner:

“chief data officers (CDOs)…are held to account for information that supports a landscape of changing business processes.

Increasingly we are seeing organizations seek to monetize their information assets, curate external with internal data using a trust-based governance model, and apply machine learning as they explore the value of the Internet of Things (IoT).

Unless adequate focus is given to improving data quality, many of these business opportunities cannot be fully realized.”

Gartner also stated that:

“given the scale and complexity of the data landscape across organizations of all sizes and in all industries, tools to help automate key elements of this discipline continue to attract more interest and grow in value.”

Well, maybe not so much just yet. We recently performed a study of data professionals at 207 companies interested in improving the level of data quality in their projects. We found that many companies have no holistic view of the health of their corporate data. In other words, they have no view of the status of the data (verified and unverified) that is flowing through their applications. And corporate data is a strategic asset that is their company’s lifeblood.

In the study, we also found that:

29% of companies validated less than 1 percent of their data
33% had validated < 5 percent
58% had validated < 25 percent
A whopping 84% of companies had validated less than 50 percent

There is a real and significant risk — both financially and to a company’s reputation (which will ultimately have a financial impact on it, whether through a stock hit, regulatory fine, law suit, or loss of sales) — to not having an organized, repeatable way to assure a company’s greatest asset — it’s data.

The CDO’s Dream

Listed below is what every CDO needs to improve the health of their data:

A systematic way to validate all of their data
Assurance of the accuracy of all their data quickly – from all the data sources to the target systems – and identification of any issues
The ability to view, manage and improve the health of their extremely valuable corporate data on a continuous basis

According to Gartner, the average organization loses $8.2 million annually through poor data quality, with 22% estimating their annual losses at $20 million and 4% putting that figure as high as an astounding $100 million.

The bottom line is data quality is a huge concern of CDOs — one that is growing at a exponential rate and needs to be addressed quickly. Luckily, there is a growing market of vendors, tools, and service providers to address this need.

So come on Hoss (or CDO), ride roughshod over that data!

(translation for non-western fans: solve the problem!)

Information Management recently published a summary of the vendors in the Gartner Magic Quadrant with a brief write-up on them.

This is a good article on the Forbes web site, Big Data And The Emergence Of The Chief Data Officer by Randy Bean, CEO and managing partner of consultancy NewVantage Partners.

For information on our Big Data validation solution in the Data Quality space, please visit the QuerySurge web site.

About the Author

Bill Hayduk, CEO, RTTS

Bill founded software and services firm RTTS in 1996 (the parent company of QuerySurge). Under Bill’s guidance, RTTS has have had successful engagements on hundreds of projects at over 600 corporations, from Fortune 1,000 to midsize firms. Bill was also the CEO of TOMOS Software, the award-winning ALM software that was incubated at RTTS, and QuerySurge, RTTS’ leading big data testing software solution. Bill lives in Westchester,NY with his wife and 2 sons.

Connection with Bill on LinkedIn»