Creating a Data-Informed Culture: How to Crawl, Walk, Run, and Fly

Creating a Data-Informed Culture: How Your Organization Can Embrace the Data and Use What It Can Teach You is the title of Chapter 3 of Measuring the Networked Nonprofit, the new book by Beth Kanter and Katie Paine. Below are two excerpts from that chapter: a section introducing the concept of being data informed, and a section that explains the Crawl, Walk, Run, Fly stages of data-informed functioning. (Order the book here.) The cartoon, also from the book, is the work of Rob Cottingham. Click on it to see it larger

“DATA INFORMED” MEANS MORE THAN KPIS AND MEASUREMENT

Data-informed cultures have the conscious use of assessment, revision, and learning built into the way they plan, manage, and operate. From leadership, to strategy, to decision making, to meetings, to job descriptions, a data-informed culture has continuous improvement embedded in the way it functions.

Key performance indicators (KPIs) are the specific quantifiable metrics that an organization agrees are necessary to achieve success. They are the mileposts that tell a data-informed organization whether it is making progress toward its goals. Too often, however, organizations chose KPIs that simply reflect activity.

Measurement is a tool that data-informed cultures use to improve their programs; they observe the results of their programs and then learn from those results to improve and refine their next programs. These cultures design measurement into their projects not just so they have measurable outcomes but so they have the data necessary to guide how to improve them.

Measurement can be used for many things, and some of them are undesirable, like justifying your existence, getting someone fired, or proving a point. But data-informed cultures use measurement to continuously improve. These cultures exist at various levels, from barely there to fully developed.
This chapter discusses a fully functioning data-informed culture and sets out how to change your organization to become data informed.

THE STAGES OF BECOMING DATA INFORMED:
CRAWL, WALK, RUN, FLY

Obviously not all nonprofits are born with the data-informed gene. And it’s not a culture you can acquire along with your analytics software. It’s an evolutionary process that happens in the four stages of crawl, walk, run, and fly.

Crawl
At this stage, the organization does not know where to start. It collects data from time to time but does no formal reporting. What data is collected does not relate to decision making. There are no systems in place, no dashboards, and no collection methods. Staff members are often overwhelmed by the thought of measurement, and the task falls to the bottom of their to-do lists. There is no process for analyzing success or failure. Decisions are all passion driven.

Walk
Organizations in the walking stage are regularly collecting data but not in a consistent manner. For example, different people and departments may be collecting but not sharing data. Or data is focused on the metrics specific to social media channels but not linked to high-level organizational results or mission-driven goals across programs and could even be the wrong data. Discussions on how to improve results are rarely part of staff meetings, and there are no linkages to organizational experience. The organization does not understand the fine distinction between being data driven and the intelligent use of data.

Run

At this stage, the nonprofit has an organization-wide system and dashboard for collecting measurement data that are shared among departments. Decisions are based on multiple sources rather than on a single piece of data or intuition. Managers hold weekly check-ins to evaluate what’s working and what’s not across communications channels, as well as any specific social media feedback received that would help shape future campaigns or social media use.

The nonprofit now monitors feedback from target audiences in real time and supplements that information with trend or survey data. The organization may work with measurement consultants or specialists to improve skills and capacity, and it provides training and professional development for staff to learn how to use measurement tools.

Fly

At this phase, the nonprofit has now established key performance indicators that it uses across programs. It has a staff person responsible for managing the organization’s data, but staff are empowered to check and apply their own data. In addition to providing weekly check-ins, the organizational dashboard includes key performance metrics related to goals. The organizational dashboard is shared across departments, and a process is in place for analyzing, discussing, and applying results. Staff members use data visualization techniques to report the data analysis but also to reflect on best practices culled from the data. There is no shame or blame game because of “failures”; instead, these are embraced as learning opportunities. A regular report to senior leadership provided by staff details high-level successes, challenges, and recommendations for moving forward. Staff performance reviews incorporate how well the organization is doing on its KPIs. Leadership celebrates successes by sharing measurement data across the organization.

###

Excerpted with permission from the publisher, Wiley, from Measuring the Networked Nonprofit: Using Data to Change the World.  Copyright © 2012.