A friend recently sent me an article announcing the new field of impact evaluation for development. Five minutes later, another email pops up in my inbox: Evaluation is a top trend for 2013. Every week there are new magazine series, global conferences, books, and nonprofits devoted to measuring impact. Evaluation it is.
As a Monitoring and Evaluation (M&E) Officer at the Infectious Diseases Institute and general evaluation enthusiast, I am both elated that impact evaluation seems to have made it onto the global development radar, and wary that it will just come and go like any other fad.
Creating elaborate impact evaluations and randomized control trials is important for knowing which ideas work well, but there is an important gap missing that most evaluation cheerleaders don’t talk about—sustainability. Just because an impact evaluation says that a project works well is no guarantee that it will be scaled up or ever used after the research money runs out. There may be plenty of funding to find out if mentoring clinicians improves patient outcomes (as my placement organization, IDI, has discovered), but there is no funding to actually send mentors out to facilities on a regular basis following the study’s end. So another well-done, well-financed study goes into the graveyard of unscaled impact evaluations. Without ensuring that there is funding, political and community support, and a clear plan for implementation, impact evaluations of health or social programs will not achieve the fantastic outcomes they promise.
That’s why I think this new attention on evaluation is great, but misplaced. Instead of focusing on expensive, one-time studies that may or not actually get used, funnel that money into evaluation of a slightly more mundane variety: using simple data in day-to-day management of clinics and community based organizations. As a GHC fellow with IDI, I train health center workers on data management and M&E. We see health centers that do a great job of collecting data and sending it up the chain to district and national governments, where it is analyzed and used to make policy decisions. But though clinicians and records staff spend hours collecting and compiling data at the local level, they rarely use it for themselves. It is always for the ministry, or that university researcher, or this NGO program. But imagine if the people closest to the problems were better able to look at their data and use it to identify what they are doing well and not so well, learn from this, and use it to change their practices for the better. This would be relatively inexpensive, and have the potential to impact hundreds of health facilities and organizations, not just those lucky enough to be in a research study.
So let’s celebrate the newfound respect for metrics and evaluation, but work to really integrate it as a basic part of our work. Let’s talk about how all employees can collect and use good data. Let’s reaffirm that impact evaluation is not just for establishing credibility with funders or publishing articles, but for improving how organizations are managed. Let’s make sure that everyone at our organizations knows why data is important and why it should guide decision making at every level.