The Vanity of Nonprofit Metrics

Sometimes, people latch on to poor metrics when they can’t find a good proxy for what they want to measure. And those poor metrics can lead them away from what they want to accomplish.

In the software world, a controversial management technique is to measure a developer’s productivity by the lines of code they write per week.

The goal of any development team is to produce good software. But “good software” is notoriously difficult to measure. Sometimes, “good” means shipping a less-than-perfect product to time the market. Other times, it means taking longer to build the absolute best alternative to a market leader you’re trying to unseat.

And in the absence of clear and objective metrics for good software, some managers choose to measure the easiest (or only) thing they can think of: lines of code.

The problem with this, of course, is that it’s akin to measuring the productivity of a writer by how many words they produce. Longer does not always equal better.

So in their pursuit of good software, these managers end up incentivizing their teams to produce bloated, overly-complex, inefficient software—by all accounts, bad software.

“What gets measured gets managed,” as the saying goes. But when your measurement is a poor proxy of your goal, it can lead you in the wrong direction.

Nonprofit website metrics are no different.

Lots of organizations are obsessed with metrics such as traffic or stickiness. Presumably, these are a close proxy to your mission, but Buzzfeed gets a lot of both and is doing nothing to improve people’s lives.

I want to be careful to clarify that traffic and stickiness are not bad metrics. That may sound contradictory, but it’s not if you understand what these metrics truly measure.

Traffic measures online awareness. Stickiness measures website engagement. If your goal is to increase online awareness and website engagement directly, these are good metrics.

But suppose your goal is to make someone’s chronic illness more manageable or improve people’s mental health or any other nonprofit-esque mission statement. In that case, traffic and stickiness are poor proxies of how well you’re doing.

I think these metrics have become widespread for a few reasons:

• The products produced by digital behemoths (social media, media/publishing) shape our expectations of how websites and the internet behave, and people tend to copy what they know and like. But digital behemoths optimize their products around traffic and stickiness because they translate to ad revenue—which I’m guessing is not your goal.
• Today’s funders are asking for these metrics as proof of your impact, even if they’re poor proxies.
• Nonprofit technology tends to lag 3-5 years behind the for-profit sector, which has become more sophisticated in its measurements, and our sector has not quite caught up.

If you are convinced traffic is the best way to measure your impact, it can be easy to artificially inflate your numbers with digital advertising or pour all your resources into SEO. But if those people stay on your site for 10 seconds and then leave forever, have you really helped them?

The best digital measurements are closely linked with the outcomes of your mission and can’t be copied from other organizations.

I like to call these your “smoking gun” metrics.

Imagine a funder approached you today and offered to write you a blank check if you could prove beyond a shadow of a doubt that your digital presence was contributing to curing Alzheimer’s or helping people with heart disease increase their lifespan. If you could wave a magic wand and have any data you wanted, what metrics would you need to convince them?

Frequently, these metrics are not obtainable by navigating to a dashboard somewhere or becoming a Google Analytics black belt. They tend to be more qualitative than quantitative.

For example, what if you could say, “84% of our website visitors leave feeling more hopeful about their ability to care for their child with autism”? 

Or, “the average person has lost 21.2 pounds since they began reading our online educational materials”?

How would that change your ability to fundraise? To write annual reports? To recruit staff?

Ironically, once you have these “smoking gun” metrics, other measurements like traffic or stickiness tend to become natural lead measures. But not before then.

Joseph Stiglitz puts it this way: “What you measure affects what you do. If you don’t measure the right thing, you don’t do the right thing.”