Death by 1,000 Data Points: Why The Wrong Data Is Killing Your Nonprofit

Once upon a time, many nonprofits were clueless about data.

Legacy systems and paper ledgers abound, the average nonprofit was about as data-savvy as your grandmother’s knitting class. But that’s all starting to change.

Now, many shops are starting to use data to guide the ship. They’re relying on numbers and analytics to tell them what’s good, bad, working, not working, and everything in between.

Unfortunately, there’s still a problem lurking.

While many nonprofits embrace data as an idea, many of them still struggle with good data management practices. Their mindset has caught up with the times—their processes and technology have not.

In particular, we often see nonprofits that have tons of data, but just aren’t using it very well. And this problem can often be just as bad or possibly even worse than organizations who don’t have data at all. After all, is data really valuable if it’s the wrong data or being used in the wrong way?

Let’s look at some common scenarios where having too much of the wrong data can really hurt an otherwise successful nonprofit.

The “right” data is the data that matters to your organization

First and foremost, it’s important to know that not all data is created equal.

Unfortunately, as part of the shift toward data-driven nonprofits, many organizations have taken on a mindset of hoarding data just in case it becomes valuable some day.

While disk space to store that information may be cheap, the time and effort it takes to sift through troves of unneeded records are certainly not. The more data you collect just for the sake of collecting it, the harder it becomes to find the data you actually need at the time when you need it.

Your process deciding which data to collect should go something like this:

1. Identify the questions you want to answer
2. Pinpoint the data you need to answer that question
3. Determine how or where you can capture that data
4. Specify the form the data should take (format, organization, etc)
5. Develop a system/process for collecting the data

The idea is to essentially work backward. Start with the end in mind and walk backward through the steps it would take to get you to the answer you’re trying to find. Unfortunately, most organizations take the opposite approach. They collect massive amounts of data—only some of it usable—and then try to extract insights from the mess.

“Wrong” data leads to the wrong insights (and actions)

In the world of data analysis and analytics, there are a number of pitfalls that can trip up unseasoned users of data. You may gather data that tells you that your last direct mail campaign was a failure—and then make decisions based on that data—when in reality, your data was flawed in some way.

It’s like when Apple Maps tells you to turn right, but really you need to turn left. Sometimes the data just points you in the wrong direction.

The key is to be knowledgeable about these pitfalls, understand when and how they occur, and then design experiments in a sound way to control for them.

Here’s a (not-at-all comprehensive) list of pitfalls to watch for in your data:

• Confounding variables – When you change more than one thing in an experiment, it’s impossible to know for certain which change led to the outcome you observed
• False positives/negatives – Experiments run under unstable conditions or with limited number of trials/tests can return positive (or negative) even when they’re not
• Localization – When you analyze a specific portion of your data and find trends that are only present in that portion but don’t hold true for all cases. (e.g., you run an A/B test but analyze the results by looking at just a small percentage of people, which happens to lead you to a conclusion that isn’t true if you were to look at all of the data.)

Then, there’s the case of just plain looking at the wrong data.

Especially for shops that are new to using data to make decisions, it can sometimes be easy to look at data that seems relevant to a specific question, when in reality it’s not.

Returning to the earlier example of your direct mail campaign, you might think to analyze its success by looking at your overall donations following its delivery. But, it may actually make more sense to look more specifically at the response rate of those people who actually received the letter. Or, perhaps you should analyze the demographics that did respond to it and see if it is more successful than email campaigns at reaching and engaging those constituents.

The point here is that data is powerful. And data-driven decision making is a wonderful thing. But with the power of data, we need to be extra careful that we’re using it correctly before making major decisions.

Dirty data is the ultimate timesuck

The other kind of “wrong” data that many nonprofits experience is dirty data.

Dirty data—or, data that isn’t uniform and properly formatted for analysis—takes tons of time and energy to process. Even for full-blown data scientists, cleaning and organizing data still accounts for about 60% of their time.

Nonprofits are often hobbled in their ability to use data because they have inadequate data collections processes and practices, which leaves them with gobs of data they can’t actually use. Not only does it become useless (or extremely time-intensive to use), but it also clouds the usable data and make it more difficult to find meaningful insights.

It’s easy to see how this kind of time commitment can be deadly.

For each hour spent cleaning and organizing data, your team is losing time they can spend on programs, fundraising, or actually using the data. It’s a trap that plagues many nonprofits.

But you can minimize the amount of work—and time—it takes to process data (meaning you can more quickly get value from it) by creating and enforcing better data practices. Create taxonomies and hierarchies for the data that’s put into any database and then enforce them vigorously.

From the day that you start collecting a certain type of data, you should already have a clear vision in mind for how it will be used. Then, work backward to determine how it needs to be logged into your system for later analysis.

Data is worthless if no one is using it to make decisions

Above all else, one of the most common mistakes nonprofits make is that they collect a ton of data—but no one actually uses it effectively.

In many cases, there are meticulous and time-consuming databases that are compiled by staff from all walks of the organization. And that information just sits there on the proverbial shelf, waiting to be used in some actionable way.

Ask yourself: Do we have a strategy and someone responsible for taking the data that we collect and applying it across the organization?

If the answer is no, you’re really hurting yourself in a huge way.

You’re wasting time tracking the data and missing out on opportunities that could be uncovered through better use of data.

There are a number of ways you can begin to use data and integrate it into your shop’s operations:

• Identify trends in donor behavior
• Test and measure different messaging
• Run pilot campaigns and monitor their long-term effects
• Segment your constituents based on which media engages them best

The opportunities are nearly endless. But, the first step is to start. Find some data that you can use and design a system for how and where it will be used.

If you haven’t started using data yet, there’s no better time than the present. Through smart use of the right data (and by sidestepping the wrong data), you can use your information as a nimble decision-making tool instead of a weight that’s holding you back.