13 Data Quality Metrics You Should Be Monitoring Now
You need reliable information to make sound business decisions. To get information, you need to collect data and transform it into a form that you can use.
However, poor-quality data will result in incorrect information, regardless of your data source. If your business uses low-quality information, it can make poor decisions. These ill-informed decisions, in turn, will result in a dip in performance.
If your organization makes heavy use of data, you need to protect its quality. To protect it, start with defining data quality metrics to ensure that you’re working only with reliable data.
This article will discuss data quality, the importance of measuring data quality, and some metrics to track.
What are data quality metrics?
Data quality metrics are the measurements you use to assess your business data. Data quality refers to the degree of closeness of the data to whatever it measures. Data is also high-quality if your business can depend on it.
These two data quality dimensions reflect the two contexts with which people work with data. First, it needs to be correct in itself. Second, it needs to be useful. Data quality metrics are either objective or subjective.
It’s common for businesses to use both categories of data quality metrics. The secret to maintaining data quality lies in recognizing how different metrics influence each other. You also need to consider how they portray your business.
Why should you measure data quality metrics?
Data quality metrics are essential because your business depends on the correctness of your data. If you use incorrect data in your planning and operations, you might miss out on good opportunities. You may also be forced to repeat work that you’ve already done.
Because of these risks, your organization should take data quality metrics seriously. Quality data doesn’t just reduce your costs. It also helps your business identify high-quality leads, improve your products, and enhance the customer experience.
Before you can use data to improve your business, you need to learn how to measure data quality.
How do you regulate data quality metrics in your organization?
If you have just a handful of data sources, it’s relatively easy to track and ensure data quality in your organization. However, as your business grows and your data sources increase, regulating data quality metrics becomes a challenge.
Even if you take all the steps to prevent errors, some will still fall through the cracks. To account for this, you can specify a set of parameters that you can use to see if your data is still useful and valid.
Most businesses use data governance rules to ensure the quality of the data they collect. Many large companies have data offices that set quality standards. At some smaller companies, data quality metrics and management fall under IT departments.
In either case, whoever is in charge of data quality should know how to spot incorrect data. They should also know how to illustrate the effects of poor data quality.
13 essential data quality metrics you should monitor
While different businesses and industries have different data quality standards, this shouldn’t stop you from creating a data quality policy.
Your data quality policy should include a list of standard metrics that you’ll track constantly. Here are 15 metrics that will give you a good idea of your data quality:
Most data scientists define accuracy as the correctness of data values and how they compare to generally accepted sources.
In other words, accuracy doesn’t just refer to how correct your observations are. It also refers to the frequency with which your data is correct. If your data has a high degree of precision 50% of the time, it’s less accurate than another data that gets it right 80% of the time.
How do you define the benchmarks that you can use to measure data quality? You should use a generally accepted accurate data source to serve as a benchmark. You should also be open to adjusting your standards as your knowledge of the world changes.
Aside from being accurate, your database should also be complete. Completeness means that you have all the data you need to make calculations.
Your data set might have missing records, but it’s still complete if those records don’t affect your ability to answer questions. You can also consider if your data set is biased towards a specific segment.
For example, you can run a survey about the workplace experience during the pandemic. However, your data isn’t complete if your respondents include only those who work from home and not the office. If your intention is to study the remote working experience, then your data is complete.
While completeness means that there are records for all subjects, coverage means that all those records have values. To comply with 100% data coverage, all fields in your records should have a value.
Here’s an example:
While both datasets contain the same number of records and are considered complete, only the first one has 100% coverage. On the other hand, the first one has only 14 out of 20 fields with values — a 70% coverage.
Aside from ensuring that all of your data is completely covered, you need to be sure that it is also consistent. For your data to be consistent, you need to ensure that you measure data the same way all the time.
For example, you cannot mix meters and feet or Celsius and Fahrenheit. You need to stick to one unit. If your data uses the wrong units, you need to convert it to the correct one. Otherwise, you risk making incorrect calculations.
Let’s look at the example above. There seems to be something wrong with the data. First, there are no units for altitude and temperature. Second, the numbers are in Celsius, Fahrenheit, feet, and meters — we just don’t know which is which. This inconsistency can lead to some unreliable results.
Now, let’s take a look at the same data set, but with all the values converted into the correct units:
By ensuring that you use the same units consistently, your data becomes more useful and accurate.
Most pieces of data are influenced by other factors that are recorded in the data. For example, the total of a set of numbers should always be equal to the sum of the individual numbers. If the numbers fail to add up, it might signify manipulated data or incorrect formulas.
To ensure data integrity, you should enforce transparency in generating and collecting data. You should also be aware of the relationships between records and values.
Most people think of timeliness as a metric that tracks whether data arrives on time. However, this is only important if you do your calculations in real-time. Timeliness in a data science context only requires that the data you receive is accurate when generated.
The numbers above tell us the temperature for specific hours. For example, it’s 12 degrees Celsius at 9 PM and 11 degrees at 6 AM. For the data to be timely, it should be 12 degrees at 9 PM and 11 degrees at 6 AM, whether you’re looking at it now or tomorrow at 7 PM.
For data to be useful, you need to eliminate duplicate records. These are records that contain identical values. Duplicate records can affect your computations by skewing them towards a specific outcome.
Let’s look at our example for the section on data coverage, but with duplicates:
The entries for Nikita Grossman and Wilson Cox were duplicated in the example above. If you’re not aware of the duplication, you might think that there are more males and Australians. However, you will get the correct number for each sex and country when you remove the duplicates.
You have to be wary of data duplication especially if you’re using CRM data. Removing duplicates from your CRM database will help you create a more accurate customer persona for more effective marketing campaigns.
8. Data storage cost
Data storage cost is an unusual data quality metric. The storage cost per GB of data doesn’t depend on the data itself but the provider’s quoted rate. The cost of storing data is tied closely to completeness, consistency, timeliness, and duplication.
If your data storage costs keep rising, it is a sign that you might be collecting too much unusable data. Duplicate or incomplete data, for example, are not usable but still consume storage space.
To reduce data storage costs, you need to make an inventory of your data storage needs. If your applications use data only for a specific time frame, it might be time to delete older records. You might also need to remove duplicate or incomplete records before storing them.
9. Accessibility and availability
Accessibility is another data quality metric that is influenced heavily by its users. It refers to the number of users who access the data over a specific period. For example, if five users consistently access the data over 30 days, the accessibility rate is five users/month.
On the other hand, you can compute availability by dividing the number of times the data was accessed by the number of times users needed it. For instance, if a user needs to produce five reports and extract data four times, the data availability is 4/5 or 80%.
Next, we move on to what we call subjective metrics. We measure these metrics by gathering user feedback. While they don’t directly affect the data, they impact how your business uses it.
The metric we call “objectivity” is one of the most subjective in this list. Objectivity measures the impartiality of a data set in the eyes of its users. Different factors influence this metric, such as the data source or the user’s previous experience with similar data sets from the same source.
This perception can change over time. For instance, an objective source can become less objective if enough users start thinking it’s no longer accurate.
A database’s objectivity can also depend on the user’s perception of the source. If a data source has a history of favoring a specific outcome, some users might not consider it objective. Similarly, some users might dismiss a data source as unobjective because it doesn’t provide their desired results.
It’s not possible to appear objective to everyone. All data users have biases, but you need to enforce accuracy, coverage, consistency, timeliness, and integrity. As long as you stick to the basics, your audience will view your data as objective.
Aside from objectivity, believability is another metric that measures end-users’ trust in the data. A data set might be accurate, but if you often find yourself filling in the gaps with data from another source, it might not be believable.
What exactly is believability? One study conducted at MIT proposes that believability has three dimensions: trustworthiness, reasonableness, and temporality. While the MIT believability model is quite complicated, it rewards consistently accurate sources. It also looks favorably on sources that provide timely data. In addition, the model looks at the closeness of the data reporter to the source of the data itself.
Of course, there are certain times where an external party might be more believable. An external auditor might come up with more believable data than an end-user. In the long run, it all comes down to what most people see as believable for a longer time.
Data users should easily understand your raw data. A data extract with clearly defined labels, for example, is more understandable than one with labels that don’t describe the data.
For example, Data Set #1 is not very interpretable:
In contrast, Data Set #2 is a lot more interpretable:
Aside from adding labels to the data set itself, you also need to provide documentation for it. Data documentation includes each data point’s name, format, and unit and how different data points are related.
You may also look into how you collect data. Your online form builder should export data in a structured format with clear labels. If your form builder churns out data that looks more like Data Set #1, you might want to consider another solution.
Finally, usability is a data quality metric that measures how you can use the data to perform a business function. For example, data is not very usable if you need to perform a series of complicated calculations to be useful. On the other hand, if most of your data is straightforward, then it has high usability.
Usability also includes findability, which refers to the ease of finding a data set. Creating a data catalog that includes a description of a data set, a list of its contents, and its location can increase data usability.
Your business relies on quality data to make good decisions. If you’re operating in a competitive market, your data should be high-quality.
However, data quality is more than just accuracy. What we usually call “quality data” consists of data that pass multiple tests. These tests include closeness to accepted values, coverage, consistency, timeliness, and integrity. It is also heavily influenced by your personal opinion of the data source.
By establishing data quality metrics for your business, you can ensure that the data you get tells a true story. The insights you gain from analyzing quality data will allow you to make decisions that benefit your business.