Acuate Data Quality

Welcome To Our Blog

This blog is dedicated to providing information and resources for organisations to improve their data quality and data management. Our aim is to provide practical help in getting the most out of your data.  Having undertaken many data projects in different organisations we see common patterns in a way organisations manage their data. We bring to you our findings and best practices to make your life easier when managing data. 

Double-Dip, Marketing and Data

During the last major economic downturn in 2008/9, we called upon many companies only to find marketing related budgets cut dramatically, marketing individuals made redundant and in one case, a company’s entire marketing department was laid off.

To me this seems short sighted because it’s marketing that can significantly generate sales or leads.

If you are measuring your marketing returns, assuming there is a positive ROI in your campaigns, then increasing the budget may lead to further positive results. Surely, slashing budgets will decrease the return.

Now, if you are not measuring marketing, then I would agree it probably seems like a black hole for your money, and there is a good argument to reduce budgets and headcount.

Not all areas of marketing can be measured effectively, but a great deal can.

As marketers, our job is not to worry about the economy heading south, but to devise tactical changes to take advantage of the current situation. As long we know which direction the economy is going then we can do something about it.

In tough times, the marketers decision is to determine how best to use your budgets: which campaigns perform the best, which campaigns are losing money, which new channels could I initiate right now to boost sales to fight the downward economy.

The great thing about marketing is that you can test campaigns on small budgets and then scale if they are working – this minimises any loses.

Let’s revert to the old adage of marketing about 'The Right Message, To The Right Audience at The Right Time’.

The message can be creative and compelling, but is very dependent on your business offering. Changing messages to accommodate the downward economy can be very profitable because messages can be tested and tweaked until a very positive result is generated.

The ‘Right Time’ can be managed by constant communication through lead nurturing techniques. (Perhaps another blog post about lead nurturing and data at a later date would give this topic justice.)

Data quality isn’t just about cleansing, it’s about having quality data that helps your business perform. Communicating with the ‘Right Audience’ is essential, otherwise the message is being delivered to the wrong person and is simply a waste of time and money.

In a downward economy, focus on

1. Your existing clients for up-sell or cross-sell opportunities; then focus on
2. Your target audience.

This is where the majority of the revenues will come from.

On my next blog post I will run through how to create a target universe so you can be fully prepared.

Marketing or IT?

One of the difficulties in improving data quality is finding who is responsible for the quality of data.

This might sound obvious, it’s the data quality department, but in reality, apart from a few large organisations, a data quality department is hard to find.

Marketers, specifically Database Marketing Managers, are very interested in data quality as they know that each missing email or inaccurate address is a lost opportunity.

However, some of the technical challenges are beyond the average marketer (for example, importing the data back into the CRM application is not always easy.)

Marketing teams often rely on IT to conduct some of the technical work, but if your IT team is very busy, then scheduling the work may not be their priority.

IT departments are good at manipulating the data, but very unlikely to have a stake in its quality.

The competing priorities in IT only delay projects whilst the data continues to degrade.

There are many companies where the data quality is managed by the IT department, but the problem here is that they really do not have a stake in it.

What do I mean? Well the marketer is measured on marketing ROI, poor data directly affects this, hence poor data affects the marketer results and hence their reputation.

Such a clear link is not obvious with IT, not to say that all IT departments don’t care about their data, but simply that there is no obvious link to their performance.

It seems to me, CRM data must be owned by the people who have a clear stake in it, namely, Marketing, Sales and Customer Support.

There will always be cases for other teams to have a stake depending on your business, but I hope you understand the clear link that exists for certain teams.

So what about data quality departments? As long as their performance is linked to the quality of data then there is no conflict.

Data quality departments are unlikely to be the users of the data – so you need to be very careful on how they are measured.

I would love to hear about your thoughts on this subject, if you have a few moments then please leave a comment.

How To End Poor Data Appending

30% Improvements

You can go a long way using data bureaus to append business information, but did you know you can increase their results by up to 30% by simply pre-processing your data.

We are not a data bureau, but over the years we have had numerous requests to append data and often revert to the standard companies providing these services.

We made some interesting observations, when you send the same data to multiple vendors, you get back completely different results.

So we now know who to speak to for certain types of data and get the best results for our clients.

Then we started to think about why these results were all different, after all, it can’t be too difficult to match them consistently.

We concluded there are two key factors.

 

Two Key Factors

Firstly, the quality of the data stored by the vendor had an impact. We noticed if the vendor’s data wasn’t up-to-date or was just untidy then that would affect the results.

Secondly, the way their matching algorithms between the data we sent them and their own database affected results. The results would clearly depend on how clever the algorithm was.

As experts in understanding algorithms for data matching and their complexities; and how poor data is much harder to match, we decided to pre-process data before it reached any bureau.

What we found, to our surprise, was a substantial increase in results with very little effort. Given how expensive it can be to append certain types of data, it was certainly worth the effort.

If your data bureau appends to 70% accuracy, then some pre-processing can increase that to 90%. This works because your data is now more matchable to the bureaus data and friendlier to their matching algorithms. (Is matchable a real word?)

So, the next time you are about to spend a lot of money appending data, whether it’s B2B or B2C, perform some pre-processing to standardise the data, we can assure you it will be worth your while.

If you’re not sure then contact us and we can guide you in the right direction.

Don’t Do These 6 Things When Migrating Data

When that moment comes it’s a nervous feeling, you are about to deploy your new CRM application to the whole organisation.

Will the users like the system? Will there be an influx of issues or will there be an influx of praise?

In the end, if you have put enough effort into acceptance testing then apart from initial niggles, everyone will feel a great sense of achievement for a good result. Over time the users become familiar with new CRM system and the old one becomes a distant memory.

But look how a small error can change a perception - I heard about one CRM deployment where the project roll-out seemed perfect, the software run smoothly, the processes implemented made life easier for everyone, but on the very first screen the very first two company records starting with ‘A’ where duplicates.

All of a sudden, the focus shifted from the magnificent project roll-out to how come we have duplicates and did anyone clean the data. A proliferation of questions was raised and the focus became the data. All those positive feelings gone out of the window!

Taking a generous view, perhaps only a few duplicates remained or a less generous view, perhaps their data quality was just ignored.

Either way, don’t let mistakes like these affect your results, after all you deserve that perfect CRM roll out, so read about 6 things we found most commonly ignored by organisations.

Don’t Forget to Consult End Users

By consulting with users at the start of a CRM deployment you gain clarification, during the project you can validate data and at the end you have complete sign-off.

You will ensure end-user buy-in and more importantly have the data the business really wants.

Don’t Forget to Remove Old Information

Often a large amount of data is so old that no relationship exists with the client. The record is there simply for historical purposes. If the information is old then seriously think about deleting the record, as it is likely the contacts have moved on and/or the company details have changed.

Check with your legal department if you need to keep it for historical reasons.

If you don’t need the data then better to delete and un-clutter the CRM system.

Don’t Forget to Validate the Currency of Your Data

Data degrades daily; everyday contacts are moving, companies going bust, contact details changing, and so on. Knowing that the business has all the current contacts builds confidence in the CRM application – end users don’t differentiate between the CRM application and the data in it.

So having up-to-date data will not only strengthen your marketing ROI and customer experiences, but it enhances your reputation within the organisation.

 

Don’t Forget to Create a Single Customer View

Having all the duplicates removed, contacts linked to the right organisations and a correct view of a client will help your sales organisation to target clients accurately. The CRM application will give a complete picture of the account history so you can decide how to up-sell and/or cross-sell.

Don’t Forget to Standardise Your Data

Reputation is very important to any organisation, so don’t let poor data formatting affect this. For example, incorrect name spellings, incorrect salutations, poor layout of an address, incorrect job titles, badly formatted telephone numbers, and so on.

How often do people get annoyed at their personal details being held incorrectly – too often to mention.

It’s your reputation and this is something that’s easily fixed.

 

Don’t Forget To Measure Your Data

Use a simple metric to start measuring your data. Migration is the perfect time to start this process. Create a simple measure and then track monthly and you will soon see a picture of the quality of your data.

You can only manage what you measure, otherwise it’s gut feel.

How to Position Data Quality

You know your data quality must be fixed, but convincing others isn’t always easy. After all they don’t have the problem of answering queries from account managers when someone’s name has been spelt wrong or that shipment is lost in the ether or the latest newsletter never arrived, and so on.

You know that you need to clean the data, but how are you going to convince management or the steering committee to part with some money, they see it as a nice to have or just not a problem.

But why is there apathy towards data quality in some organisations? Well, we thought we would map out the different types of data from a commercial viewpoint to see why this is the case.

Look at this diagram, we call it The Data Stack, you immediately see that Data Quality is at the bottom, but that’s just reading the diagram in correctly. Data Quality is the foundation for all the other types of data. An error at the data quality layer has a big impact on the layers above it.

We’ve posted articles about data quality ROI using simple email and direct mail campaigns.  When you see how a missing email leads to lost opportunities, which leads to lost revenues, which leads to lost recurring revenues then fixing data quality problems becomes an insignificant cost and an urgent fix.

1. Sales & Marketing ROI – These are the big numbers that directors love to see, what are our sales conversions, what are the returns on the direct selling team, which marketing campaigns are the best, etc.
2. Business Intelligence – This is the inside commercial information that provides the bones to the above layer.  Which products are selling, what are our up-sell and cross-sell conversions rates per product/service/account, how well is the market place engaging with us online?, etc.
3. Data Intelligence – Do we know everything we need to know about our target markets, what is our level of segmentation, do we have all the required intelligence on existing and new accounts, etc.
4. Data Quality – Is the information we have correct, does it have integrity, is it improving with time, have we stopped it from degrading, etc.

Each of the layers affects the layer next to it.  Errors at the data quality layer will mean incorrect figures for management; incorrect data intelligence will affect the results at the business intelligence layer, and so on.

You can access a whitepaper on The Data Stack for a fuller explanation or visit our main website.

At a quick glance at The Data Stack, you can see by fixing the foundations of data quality, accurate and reliable business data is made possible.

We started by saying, in some organisations there is apathy towards data quality, probably through lack of understanding; and when we see that data quality is at the bottom of the stack, no wonder this perception occurs. (Management may not have the concept of The Data Stack, but the subconcious will rate it lower.) Once you look at the ROI and the affect of poor data quality, it couldn’t be more important.

So use The Data Stack as a tool to explain to others how this works and position data quality appropriately.  In the coming weeks we will be looking at each layer in more detail.

The 3 Key Skills To Successful Data Projects

I recently called upon a senior manager to explore their data challenges. I politely introduced myself as a data expert and tried to encourage them to share their data issues with me. Before I could complete my sentence, a resounding 'Data is boring so why should I bother with it?' came out.
 
My instinctive reply was 'It may well be boring, but you still need to deal with it.' Now, this got me thinking about how data is perceived and what skills you really need to have for a successful data project.
 
Maybe the boring perception came from a great deal of manual data work or the benefits weren’t clear, however, a great deal of money continues to be spent on data projects. It seemed getting a real understanding of data projects was key.
 
Many organisations attach great value to their data and for most it's a cumbersome job to resolve all data-related issues. After all, it takes a great deal of effort to get data right and then it only degrades as time goes on.
 
The 3 Key Skills
 
Getting data right is an interesting mix of (i) Commercial Knowledge, (ii) Data Knowledge and (iii) Technical Knowledge. To find someone who knows all three disciplines is difficult.
 
Let’s assume we are deploying a new CRM application, and we come to the stage where the data needs to be consolidated, cleansed and made fit-for-purpose so our sales and marketing can flourish.

During the migration we will need commercial knowledge to work how accounts and contacts should be merged. For example, with duplicate accounts and contacts (a) which is the most recent contact, (b) pick the account that has generated most business, (c) pick the account with the most recent activity, and so on. The list of commercial decisions is specific to each organisation.
 
Cleaning data, removing duplicates, checking emails, creating business hierarchies, enriching data require data knowledge. You need the algorithms, methodologies and reference data to deliver all of the above.
 
Finally, exporting, importing and merging are technical procedures requiring technical knowledge. Depending on the complexity of your systems, a different level of technical skills will be required.
 
No wonder data projects can be difficult and time-consuming to resolve. Finding a natural home in any organisation isn’t easy.

When is Enough, Enough?

What level of data quality should you strive for? This might seem like an obvious answer, namely, 100%, in reality that’s not possible. Your data quality is worse today than it was yesterday, because data degrades, a company no longer exists, a telephone is no longer active, a contact name changes, and so on. Also, unless data is being entered in perfectly, new data will erode the 100% target. There is no way of ensuring all these factors are accounted for every single day.

If we are to measure data quality then aiming for 100% is a target you cannot meet. However, aiming at 90% or 95% or 97% are far more realistic targets and very achievable.

So when is good data quality good enough for your business? Well rather than thinking of an arbitrary target, think it of it as work in progress. Set a target that is achievable, say 90%. Define the metric that makes up the 90% (a simple metric could be a mailable address and a valid email address.) Write the processes to measure and report monthly on this figure. After a few months you will know whether data is getting worse or better or just averaging a particular value.

Once you have a handle on the quality, you can create specific processes, programmes, marketing campaigns, etc. to improve the quality. This is great for management, as they need simple figures, charts, trends and explanations of what is going on. Once management see the trend, then make sure they understand the ROI, justifying expenditure on data quality will then be easy.

There are other articles on data quality ROI in this blog so I won’t cover that right now; but for marketing data, the answer is always a no brainer because each piece of accurate information can lead to sales which lead to recurring revenues.

When is data quality good enough? Well, it’s not the absolute figure that’s important, it’s the measure and trend that is important. Ensure an upward trend by removing data degradation and proactively improving the existing data.

An Unspoken Problem, But Common Problem

“Sometimes there are practices that create problems but go unnoticed simply because it’s part of our everyday experience and we are not conscious of how it creates problems for us. The problems persist and only come to light when an unintended consequence happens that is important to you or someone else. This usually stems from not having a plan that focuses on solving these issues. It could be sometime before we realise the overall impact and how long it’s been going on.”

Now, the above statement could be applied to any aspects of our lives, but in our world of data quality, the act of import/exporting data seems to fall into this category. Let me explain.

We receive a lot of data to review and surprisingly find that import/export errors persist in the data from either the recent task of sending the data to us or a previous import/export that has left the data with problems lingering in their database, sometimes unnoticed for months or even years.

In the distant past I have instigated such problems myself without realising what I had done. At the time, you deal with the issues at hand. However, over time you become wiser and take more care.

But over the years the proliferation of these problems and the subsequent time wasted resolving what is preventable, has got me to ask is there a better way.

These problems undermine data quality, but more importantly waste a great deal of time. The data needs to be re-imported or re-exported, and this time you carefully check for errors. So, why not deploy procedures (manual or automated) that check for these particular errors proactively.

Anyone manipulating data is familiar with these problems, particularly if you are using desktop applications such as MS Excel, MS Access, Notepad, etc.; or relational databases such as MS SQL Server, Oracle, DB2, etc.; or CRM applications such salesforce.com, Saleslogix, Siebel, SAP, Goldmine, etc; and other proprietary applications. Although importing and exporting is made easy (to be honest - exporting is all ways made easy, importing is another matter), not taking into account the nuances in your data that conflict with imports/exports or not using the application properly leads to errors.

Let’s look at some common problems.

1. Text Delimiter Problem – When the character chosen as the text delimiter can be found in the data, import errors will occur. The application will think there are extra fields, incomplete field or even merge fields. How often have you switched from using commas, tabs to pipe characters to avoid errors.


2. Text Qualifier Problem – Often a character is used that should be surrounding a piece of data that is used by a software application to say this is the start and end of the data. If one is missing then this may look familiar:


3. Diacritics – Non-Roman characters become formatted to strange looking combination of text. For example: Számítástechnikai becomes SzßmÝtßstechnikai. Which people often spend a great deal of time to resolve by either (i) using Find/Replace operations or (ii) deleting these characters. The simple answer is to fix the application that caused the problem or if you are lucky then you can use a text editor and ‘Save As’ using a new encoding method such a Unicode to automatically resolve this.

4. Unexpected Characters – Adding new line characters or carriage returns can confuse an application. It can also cause problems with applications that manipulate data – we don’t often expect a new line character to be in a telephone number, but some CRM applications allow this.

To resolve this we have written our own verification product that identifies any issues before we import or export data. This allows us to inform clients of errors on data that they have sent us, and enables us to verify our data before the it is distributed. In addition, simple manual checking such as (i) record counts, (ii) listing distinct values for a field, (iii) identifying any conversion problems, (iv) noticing that a number should be right justified rather than left (v) checking diacritic characters are correctly represented, etc. can help significantly reduce time and effort.

Overall, our automated software and the practice of proactively checking saves a great deal of time.

How to Create Outstanding Marketing Data

Kaizen, Japanese for "improvement", or "change for the better" refers to philosophy or practices that focus upon continuous improvement of processes. Kaizen was original used by Japanese companies to improve manufacturing processes and ultimately the quality of their products. Popularised by the well-known management guru and statistician Edwards W. Deming, who introduced his statistical methodologies to top management of leading Japanese companies in the 1950s, !960s and 1970s. Through this on-going measuring and adjusting, extraordinary results were achieved, not only in manufacturing, but in product quality, service, sales and many other areas. The Japanese commercial invasion of the US and Western Europe in the second half of the last century is well documented. A great deal of credit is attributed to this never-ending process of improvement that transformed Japanese businesses and enabled low-cost, better-quality products and services.

This very same approach is the best way to create outstanding marketing data. You don’t need high-level statistical knowledge or complex time-and-motion studies of all your processes, but the simple act of regularly measuring marketing data and reviewing it will propel you to the top 5% of companies that take time to consistently improve data quality to achieve more sales and significantly improved marketing ROIs.

What is Outstanding Marketing Data?

Outstanding marketing data is data that meets the goals of your marketing strategy which is likely to maximise sales revenue, leads, lead nurturing conversions, cross-selling, up-selling, etc. Because of the importance of these numbers, they are measured and reported to senior management. The numbers are used to assess the contribution of these activities against targets, comparisons against past experiences and are used to forecast revenues. The analysis of these numbers allows you to improve your marketing results, Kaizen is already at work here – measurement and review is already happening here at senior management level.

Marketing results improve as you test, adjust, test, adjust, … , test, adjust all your campaigns. Through this constant process of improvements, your marketing campaigns deliver greater and predictable results.

Outstanding marketing data maximises the results from these campaigns. Let’s look at a simple scenario. You have an email campaign to drive prospects to attend an Exhibition (could be a seminar, a breakfast meeting, a webinar, etc.) Now assuming the email is targeted at your ideal prospects, you would like as many prospects to attend as possible. From the exhibition, you convert 1% of prospects into sales. If your database is missing emails for your ideal prospects, then 1% of these would convert to sales. You can calculate this value exactly – the value of lost opportunities directly attributed to not having the right emails.

In this case, outstanding data is having emails of the ideal prospect data. The more accurate and complete these emails are, the greater the marketing results will be.

What Marketing Data Should I Measure?

This depends on the type of business you are in, but if you send email campaigns then email addresses are important, if you mail direct marketing campaigns then postal addresses are important, if you have a telemarketing team, then phone numbers are important, and so on. Names and/or company names should always be measured and must be of good quality, nothing worse than poor data quality affecting your reputation.

In marketing terms it’s not only the number of companies or contacts, but how much data do you have of your target universe. You may have 100% contact data accuracy, but only 10% of the possible market availability. Measuring against target universes is also important.

When and Where to Measure Marketing Data

So, deciding what to measure is more or less easy to do, just match your marketing ROIs and target universe sizes to your data. The more difficult question is when and where should I measure this data. We need to understand how data behaves to answer this question correctly.

Firstly, data degrades over time. For B2B businesses contacts move, businesses change ownership, businesses terminate for varying reasons, etc. For B2C business, customers change address, change names through marriage, change telephone numbers and have multiple email addresses, and so on. Your data is worse today than it was yesterday simply because changes are occurring daily.

Secondly, data is sometimes entered in poorly. As human beings, we are all guilty of entering poor data, we may only enter some of the data or simply make typing mistakes. Whatever the reasons, the end result is poor data or missing data.

To get started, there are three key measurements projects that should take place:

1. Baseline Measurement – Create a measurement system as the base of what the data quality is right now. This could be simple as the percentage of valid emails present or the percentage of mailable addresses. See Acuate’s Seven Dimension of Data Quality for ideas on measuring data.

2. Data Entry Measurement – Create management reports that highlights the errors in data for individuals who regularly enter data as part of their daily job, such as, call centre staff, salespeople, support staff, etc. The reports should be reviewed once a month (or an interval that suits your business) and corrective action taken.

3. Degradation Measurement – Either through industry reference data or telemarketing campaigns you can determine the level of degradation in your data. Once known, you can remove the data that has degraded; ensure more data is entering into the database to replace the outgoing data. The idea is to increase the size and accuracy of your database over time.

By creating processes, reports and corrective actions for the above three measurements, significant improvement in marketing results will manifest. Applying Kaizen to these processes will accelerate you marketing abilities and give you that outstanding marketing data to fuel your business.

 

Magic of Marketing

A true wonder of effective marketing is the amazing rates of returns one can achieve. When these are compared to risks/rewards that can be achieved on the stock market, you will see effective marketing can outperform most investments. 

Marketing is an investment, an investment of acquiring new customers. The down side can be negligible because we can test marketing campaigns on a small scale. If the testing works and a good returns are made then you can confidently scale up the campaign.

If the testing showed poor results then you have lost the money for the test only rather than the costs incurred for a large scale campaign. So the downside of a marketing campaign can always be managed and costs made negligible. Not so easy to manage the risk in the stock market.

Once you know the return you are achieving with your test, then you can predict how the campaign will work. Also, this gives you great confidence because if the campaign is repeatable then you have a sure method for generating the same return over and over again.

Now, depending on your business, the returns can be extraordinary compared to returns in shares. If shares go up by 5-10% in a month then that can be considered an excellent result. In marketing you can have returns in excess of 50% - 400% regularly and that beats what most investors can achieve.

This is truly magical, seeing marketing as an investment is the key to increasing sales and/or leads.

Big companies and sophisticated small companies know that by diligently measuring and testing marketing, then wonderful returns can be created, but this can only happen if you measure and test.

By the same token, improving the underlying database of contacts in your target market will enable you to improve your marketing results. Measure the quality of data and the quality of your intelligence and you will have the ability truly maximise your marketing.

Finding a measure for data quality and data intelligence isn’t always easy, but we have some ideas that can get you started. Acuate’s Seven Dimension of Data Quality is a framework to help measure your data quality and data intelligence.

By using a consistent measure it gives you the power to manage the data when targeting niches or segments. Coupled with sophisticated marketing techniques of measurement and testing, extraordinary returns can be achieved.