Data Quality vs Data Condition: The Power of Context

Anmut
Data Condition, Data Quality

Context is everything. Facts, events, statements, and statistics without proper context have little value and only lead to questions and confusion. This is true for life in general, but it’s especially applicable to the data you use to power your business.

Data without context is just meaningless noise, and any effort to improve or extract value from your data without considering the larger business context is doomed to fall short.  Unfortunately, traditional approaches to data remediation often focus on technical data quality in isolation from the broader data and business ecosystem.

In this blog post, we’ll compare traditional data quality vs data condition — the big picture approach to data improvement we use here at Anmut. We’ll start with basic definitions and then take an in-depth look at the characteristics, strengths, and weaknesses of data quality and data condition. Keep reading to find out everything you need to know about data quality vs. data condition and why understanding your data’s context is so powerful.

Data quality vs data condition: basic definitions & differences

Data quality and data condition share the same basic goal of making sure your data is in good shape and ready for its intended use. Both measure and compare the current state of your data against the desired state, but the approaches — and the results — are quite different.

Data quality assesses your data against standard technical performance measures such as completeness, accuracy, timeliness, validity, and uniqueness. These measures can be applied to any dataset at any company regardless of industry.

In contrast, data condition determines whether your data is fit for its intended purpose by going beyond technical data metrics and diving deep into the larger business context. Data condition includes an assessment of how the data is used, the value it brings to the business, and the systems that support its collection, maintenance, and usage.  Since every business uses data differently, data condition assessments are unique to each company and can be tailored to fit your needs. The assessment process is the same, but your data’s fitness for purpose is measured against the specific needs of your business instead of an arbitrary technical standard.

Putting things in perspective: A helpful analogy for understanding data quality vs data condition

Sometimes the easiest way to understand something is to compare it to something else. Before we dive into more details, let’s explore an analogy to help put things in perspective and set the stage for a deeper understanding of the differences between data quality and data condition.  

Think about the dashboard in your car. There are gauges to tell you how fast you’re going, how much gas you have, your engine’s RPM, whether your lights are on, what gear the car is in, and other things you need to know to make sure your car is ready to get you where you need to go.

In a data quality approach to reading your dashboard, you would simply read the gauges and observe that your tank is half full, your lights and windshield wipers are off, and you’re driving along at 70 miles per hour. Useful information to be sure, but without more context, it’s unclear what those observations mean and whether you should take action.

A data condition approach considers the context you need, to put what you’re seeing in perspective. In the car dashboard analogy, this context includes factors such as how far you’re going, the time of day, the weather conditions, and the speed limits. Depending on your destination, half a tank of gas may be all you need. If the sun is shining and the weather is nice, there’s no need to adjust anything, but if it’s dark or raining, you should turn on your lights and wipers. On an open highway, 70 miles per hour is perfectly acceptable, but if you’re near a school, it’s time to slow down.

A closer look at data quality: How it works, how it helps & why it’s not enough

Now that we’ve covered definitions, basic differences, and an analogy, let’s dig into the details and learn more about data quality.

How data quality assessments work

As we mentioned, data quality uses standard technical performance metrics to assess your data. For example, a data quality assessment might tell you that your data is 94% complete and 78% accurate or that 12% of your data is duplicated and 40% is more than five years old.  A data quality assessment might reveal that your customer data includes thousands of invalid addresses or that some of your products are labelled with the wrong description or ID number.

Benefits of data quality

Bad data is bad for business, and 98% of organisations across all industries believe their data is inaccurate. Although its impact is limited, data quality can help you improve some aspects of your data. Here’s how data quality can help you:

Straightforward implementation

If your data has been neglected for a long time, traditional data quality is a relatively easy place to start. The process is the same for every business, and there are tools to help you automate the major steps.

Identify technical issues

Technical data issues are common. It’s easy for databases to become filled with duplicate, inaccurate, old, or invalid data. Data quality helps bring these issues to light so they can be explored and addressed via data cleaning or other remediation activities. Sometimes exposing these issues is a necessary first step towards convincing key decision-makers that further action is needed.

Why data quality falls short

While data quality can be helpful, the benefits are extremely limited. If you’re looking for a solution to truly unlock the value of your data and use it to benefit your business, data quality falls short for several key reasons:

No link to the business

Data quality is often treated as an IT project, with assessments performed by IT personnel in isolation from the broader business. Since the scope of data quality assessments is limited to technical attributes, the scope of your insight and the value of your assessment is limited as well.  Your data is meant to be used by the business, so any data assessment that fails to incorporate the business perspective misses the mark.

No connection to performance

While you may find out your data is incomplete or inaccurate, you won’t find out how these quality issues are affecting your business performance or understand the urgency or value of correcting them.

No optimised remediation

Focusing on data quality often leads to fixing issues just to fix them — without considering what actions have the most value or will have the biggest impact on business performance. This leads to inefficient remediation efforts instead of an organised, optimised strategy based on value.

No root cause analysis

Data quality helps you identify technical issues, but not what’s causing them in the first place. Often the root cause involves the people, processes, and technologies around the data — none of which are considered in data quality. Measuring technical data quality often occurs too late in the data lifecycle and only exposes symptoms of bad data condition instead of what’s causing the issues in the first place.

A closer look at data condition: How it works, how it helps & why you should embrace it

Use cases are the foundation of data condition assessments. They’re meant to keep the focus firmly on value by asking and answering key questions such as:

How does your business use different types of data?
Which data is most critical for your business?
What does your business need from your data to be successful?
What is your business trying to accomplish with your data?
Which business decisions rely on your data?
What needs to be done to improve the fitness for purpose of your data?
How does improving data condition unlock value?
Which data remediation actions will unlock the most value?

The answers to these questions help frame the data in the broader business context and provide a business (and value) centric lens for assessing the data. The goal is to assess your data in a way that includes both technical quality and the people, processes, and technologies that make up the ecosystem around your data. Doing this from a use case perspective keeps business needs and value front and centre.

A data condition assessment starts by building a list of use cases for the data you want to assess. Most data sets have multiple end-users and therefore multiple use cases with unique processes and requirements. It’s important to capture and consider all these use cases. You’ll prioritise your use cases based on the value of each data type and the importance of the business decisions the data supports.

Once you have a prioritised list of use cases, it’s time to interview users to understand how they use the data for each use case and the data-related pain points they encounter. User interviews capture detailed user perspectives on data condition and make sure the voice of the business is heard. The resulting user stories enrich your qualitative assessments and help paint a holistic picture of how well the data and its ecosystem are meeting the needs of your business.

The results of these user interviews are analysed in a series of structured workshops and translated into a set of bespoke, business-centric set of requirements for your data, giving you a clear standard for measuring data condition. As part of Anmut’s data condition process, we walk you through these workshops and help you develop measurable data condition criteria for each use case.

The resulting metrics are all designed to help you measure fitness for purpose and improve your data condition. We use those metrics to measure the condition of your data for each relevant use case.  In our work, we produce a data condition score for each data set to help you understand how your data is performing, what’s preventing best performance, and how to identify and address the root causes of your data problems.

Benefits of data condition — and why it’s a better choice

By seeing your data in the context of your business as a whole, data condition gives you the insight you need to understand the value of your data — and to communicate its value to others. You’ll also see whether your data is meeting your business needs and where it falls short.

No other approach to data improvement that we’ve seen (and we’ve looked at A LOT) gives you the tools to assess your data in a way that’s holistic, relevant to the business, and designed to add value to your organisation.

Here are just a few of the benefits it brings:

Tailored processes lead to results

Data condition considers how you use your data by examining the datasets and use cases in a way that’s unique to your company. This targeted process leads to actionable results and clear direction for improvements to your data and the people, processes, and technology around it.

Links data directly to the business

Data condition frames data in language the business can understand. End users don’t care much about technical completeness or validity scores, but they do care about being able to find, access, and trust the data they need. Data condition helps build a bridge between IT and business teams because it gives a common framework for discussion and cooperation.

Powers efficient remediation efforts

Data condition assessments give you a clear picture of the state of your data. You know exactly what’s ‘broken’ in your data set and how it’s affecting your business. Using this knowledge, you can design efficient remediation efforts focused on the most critical root cause problems.

Enables smart investment for tangible returns

When you understand exactly how your data is used, the value it provides, and the problems preventing optimal use, you can make smart decisions about investing in projects to improve its condition. 

Enables monitoring & tracking of data condition & improvements

Data condition also provides a way to continually track how well your data is supporting the business as well as the success of your data investments. It allows you to measure the value gained from making your data fit for its intended purpose.

What’s next?

We hope you’ve enjoyed reading this article and learning more about data quality and data condition. Here at Anmut, we get to see the benefits of data condition all the time, and we love to share what we’ve learned with others who might be interested in adopting this approach.  

We’ve built our business around helping people like you understand and unlock the enormous value of data, for all stakeholders be they customers, investors, employees, suppliers, communities and the environment. If you’d like to learn more about data condition and what it can do for your business, get in touch. Ask all the questions you want — we’ll be happy to answer them. If not, you might be interested in finding out what your data is worth.