If not results, then what? The risks of not having a results agenda

March 17, 2011

Arab uprising update; Wellbeing v GDP in China; IMF analyses cricket; aid reality checks in Bangladesh; India's cyber-city; DFID and India; another aid satire: links I liked

March 17, 2011

So where have we got to on Value for Money, Results etc?

March 17, 2011
empty image
empty image

Great posts, great comments. My head is now spinning as I try and disentangle some of the different threads that havecomplexity sign emerged over the last two days.

First: horses for courses. Some aid work is akin to Ros’ bathroom problem – linear, measurable, and suitable for a logframe + results approach. Other areas are emergent and unpredictable and a results approach would struggle. Say you had a programme in Egypt right now, and were wondering what to spend your money on. You could reassure your donors and supporters by opting for a measurable bathroom problem, say building schools, but that would be to ignore the historic opportunities for change presented by the social and political upheaval in that country. But how could you support that with any likelihood of proving impact or attribution? Tricky, but a clear risk that the results agenda will drive you in the wrong direction. Could senior management, as Jonathan suggests, create a situation where some programmes are assessed on results and others on relationships? In the current climate, it’s easy to imagine that the latter category would end up being starved of funds.

Second: upwards v downwards accountability. Can a results agenda strengthen both –  can countability improve accountability? (thanks for the soundbite, Sceptical Secondo). Claire, supported by Penny Lawrence and Alex Jacobs (with an excellent link to some practical examples) thinks it can.

Third: theory v practice. Claire’s right, I think, that in theory, a results agenda can be built on the perceptions of beneficiaries, improving quality and accountability. But Ros has spent a lot of time looking at what happens once all these ideas are implemented on the ground, and what looks good in the thinktank (and apparently in the NHS) may not survive the collision with reality, where staff are overstretched, working to tight deadlines and have little time for innovation or risk-taking. When I talked to Oxfam’s number crunchers about this exchange, they said they would love to take part, but were simply too busy generating the numbers needed to satisfy our donors!

Fourth: Trust. Ros rightly raises this, but trust between whom? The results agenda aims to build trust between northern publics and aid agencies, which is of course vital if aid spending is to continue to rise. And given that NGOs endlessly tell corporates and governments that we have moved from a ‘trust me to a show me agenda’, it would be pretty hypocritical to say the same shouldn’t apply to us. But what about the trust between aid workers on the ground and the partners they work with? Ros worries that that trust will be eroded by a crude results focus (and Alice Evans’ example from Zambia suggests she’s right), whereas Claire seems more concerned about using measurement to tap directly into the lived experience of poor people (hard, if that group is not as easily identifiable as NHS patients – back to my Egypt example).

Fifth: It all comes back to people, in particular the skills and motivations of the people who work for bilaterals, NGOs and all the other bits of the aid industry. If you have brilliant, motivated staff dripping with a sense of vocation, then they can probably make either a results agenda or a relationship-based approach work just fine. If you have demotivated nine-to-fivers who see this as just another job, then they will find a way to tick boxes and achieve little, whatever approach is adopted. I guess the interesting question is about those in the middle – what works best with normal staff doing the best they can, while coping with all the other pressures in their lives?

Which brings me to my final conclusion. I assume that the value for money people would never dream of asking us to take their ideas on trust. What are the results of the results-based agenda, compared to other approaches? What would be the best way to evaluate the evaluators? Is the balance of evidence different for say, work on women’s empowerment, governance, livelihoods or health and education? Lots of work for researchers over the next few years. [update: Ben Ramalingam came to much the same conclusion a few months ago on the Aid on the Edge of Chaos blog]

So thanks everyone, I’m now better informed, but still on the fence. As are the rest of you, judging by the pretty even split on the poll. It stays up til Monday, so not too late to vote…..

Update: in a similar vein, someone just put this up on twitter [h/t Henry Northover and Ian Thorpe]:

“Can I pay for Nancy Birdsall’s new book on Cash on Delivery aid after I’ve tried it out to see if it really works?”


  1. “Motivational crowding-out” is the concept I miss in this discussion on aid and result-based management.

    “Motivational crowding-out” occurs when people’s “intrinsic motivations” are disturbed by external agents efforts to establish incentives and controls. Psychologists, sociologists and behaviour economists have written on it. Recent field experiments have confirmed that it is a factor one should take seriously. Just one of many recommended readings: http://people.unica.it/vittoriopelligra/files/2008/09/Promoting-Trust-paper.pdf .

    “Motivational crowding-out” seems to characterize much of the aid industry, at present, as a result of uncritical misuse of result-based management tools.

  2. The underlying problem is comparing apples to oranges. “Development” is a broad church. It is possible to compare the results and risks of projects promoting public participation in elections with each other, but not with the results of projects providing vaccination for young children. By considering development as a compound measure, the weighing becomes indeed absurd. However, when looking at the different values or sectors separately, it should be easier to compare within each category. Deciding what sector to fund is more politics than science however.
    Like Easterly says: democracy should be supported in its own right, not because it could lead to economic growth or less child mortality.

  3. Every right-thinking aid advocate would want aid to lead to results. But the problem with the current paradigm of results-based aid is that it manifests largely in the form of numerical targets with the risks of creating perverse incentives to reach them. We know how that panned out in the target-happy era of New Labour – managers taking the wheels off trolleys and reclassifying them as beds. Following up on Goran’s thoughts, we would be better-served to foster a culture of obligation of resources + trust, than a culture of obligation of results + mistrust (with thanks to Patrick Watkins for introducing me to this concept).

Leave a comment

Translate »