Keep on Talking about Impact

I opened my New York Times last Sunday to find an article (“Despite Push, Success at Charter Schools is Mixed”) about charter schools that echoes the rising and falling cycle of expectations about microfinance. The cycle begins with a vexing problem: after so many years and dollars trying, why aren’t our schools better, especially for the poor? Along comes a bold new idea – charter schools – and a parade of prominent and wealthy champions: the Gates Foundation, business-minded Internet-rich people like the founders of Google and Amazon, plus some celebrities. A series of pilot projects become a “movement,” with hype and expectations mounting along the way. Next come the round of formal evaluations, some with random control methods, that are, well, a bit of a drag. Inconsistent results from one study to the next. Arguments over evaluation methods. And, in the end, the conclusion that expectations were exaggerated and that the tenacious barriers to high-quality education for all won’t fall easily.

What is common to both the debates over microfinance and charter schools are these factors: (1)a vexing problem, (2)innovative approaches, (3)outsized expectations, (4)well-meaning donors whose funding and visibility further boost expectations, (5)evaluations that are sobering in their conclusions, and (6)a certain amount of letdown and then recalibration of expectations. In the case of microcredit, does anyone think that all of the enthusiasm, funding and acclaim (even a Nobel Prize!) would have accrued if the initial promise had been that it would “allow…people to make large purchases that they would not have been able to otherwise” – as advocates now cautiously cite as one of microcredit’s main benefits?

On the questions of evidence and proof, I agree that each of us has a responsibility to invest the necessary time, thought and resources to build a strong evidentiary case for our effectiveness. We must do so, of course, because thoughtful donors ask for it. But we must also commit ourselves to careful measurement because it the key to how we can continuously improve our work and manage better. I’ve always tried to manage by this maxim: Find out what works, so that you can do more of it, and what doesn’t work, so that you can fix it or stop doing it. Ultimately, that second purpose – effective management — is more important than the necessary task of being accountable to our funder.

As important as our measurement efforts are, we must not forget Freedom From Hunger’s advice on how much “proof” to demand. I also offer this simple wisdom from Jim Collins, in his 35-page masterpiece, Good to Great and the Social Sectors:

“It doesn’t really matter whether you can quantify your results. What matters is that you rigorously assemble evidence – quantitative or qualitative – to track your progress. If the evidence is primarily qualitative, think like a trial lawyer assembling the combined body of evidence. If the evidence is primarily quantitative, then think of yourself as a laboratory scientist assembling and assessing the data.”
Collins goes on to caution against treating any measurement system as received wisdom or divine truth:

“Al indicators are flawed, whether qualitative or quantitative. Test scores are flawed, mammograms are flawed, crime data are flawed. What matters is not finding the perfect indicator, but settling upon a consistent and intelligent method of assessing our output results, and then tracking your trajectory with rigor.”
Aside from the levels of “knowing” or proof that we should expect of ourselves, and donors expect of us, I would like to add one more question to the discussion of microfinance’s impact. That question is: For whom?
One of the exaggerated expectations about microcredit is that it is equally effective – and available – to all people living in poverty. As we all know – but isn’t as clear to the general public, including most donors – the conditions of poverty and vulnerability are quite different for those living at the $2/day threshold and those at the $1/day line. Roughly a billion people in each category.
Based on both our experience and our data from our work here at Trickle Up, we know that microcredit is often not the appropriate financial strategy for people living below the $1/day (or, if you prefer, the more current shorthand of $1.25/day, as defined by the World Bank) who are trying to build a sustainable livelihood. Our approach combines seed capital in the form of a grant, livelihoods training and savings support. The microfinance analogy for our seed capital grants is that they are much like equity in conventional finance – or, more precisely, the “friends and family” funding that entrepreneurs in this country use to fund new ventures. Savings, which are now widely regarded one of the most effective and scalable forms of microfinance, is an integral part of our three-part model.

My point here is not to launch into a commercial for Trickle Up or other development institutions, many of them MFIs looking for an effective way to help the very poorest (e.g, BRAC, Fonkoze, SKS) that follow a similar approach for this population. I am simply urging that, in the continuing discussion of the effectiveness of microfinance as a means to sustainably alleviate poverty, we ask both “is it effective?” and “for whom?”