‘Stuff happens’: the risks of a results agenda. Guest post from Rosalind Eyben
A few months ago, I blogged about the risks associated with the aid industry’s current overriding obsession with audit/value for money/results (pick your term). Since then, that debate has been swirling around both on this blog and (more importantly), in aid and development circles in many countries. So to help it along a bit I’ve asked two people who think about this a lot more than I do to set out some competing arguments. First up is Ros Eyben, who got a big and largely positive response to her recent challenge to the dumber/more extreme varieties of value-for-moneyism. Tomorrow the ODI’s Claire Melamed responds. Please join in the debate.
“The UK’s development ministry (DFID) has just completed a review of its bilateral aid programme. The Secretary of State for International Development, Andrew Mitchell has ‘set out the results that UK aid will deliver for the world’s poorest people over the next four years’. DFID will be more ‘hard-headed about making every penny count’. Its press release highlights results such as 11 million more children at school and 50,000 fewer women not dying from having babies. Digging into the review’s report, you will find numbers relating to DFID’s other aims, including wealth creation and tackling the root causes of conflict. Here, DFID is more modest: 50 million people with the means to help work their way out of poverty, rather than creating millions more jobs as some enthusiastic DFID country offices apparently offered to achieve. How can a government (let alone a foreign aid agency) deliver jobs? Likewise, DFID is not going to reduce the number of conflicts in the world but instead help citizens hold their governments more accountable.
When we look at the details, DFID’s plans seem pretty sensible. But the press release worries me. Explaining to the British public how UK aid delivers value for money – promising to educate more children than those we educate in the UK, but at 2.5% of the cost – must surely influence how DFID thinks and works. I am in charge of redecorating our bathroom while my partner is away. The paint is peeling and there is mildew on the ceiling above the shower. To demonstrate I got value for our money I will get two quotations for the redecoration. Many donor governments are treating the complex problems of poverty like my bathroom. They contract a Third Party Operator to deliver a result pre-determined by DFID. At the end of three or four years, there is an evaluation to check on results before paying the contractor.
Sometimes DFID’s bounded problem-approach to change (as typified by the logical framework) is going to work. But there are major concerns about the institutional and financial sustainability once the intervention ends, if these have not been addressed as an integral part of the design. By 2006 the global polio vaccination campaign had successfully eradicated polio from all but four countries, yet by 2008 it had reappeared in nineteen additional countries. In the drive for results, insufficient attention had been paid to the national health systems needed to keep polio at bay.
To be able to count exactly how each penny or Euro of aid money gets spent, donor governments are risking not making any difference at all. They can show how many kilometres of roads they have built or numbers of babies vaccinated as compared with before they started the projects. But such facts reveal little about how the change was achieved and what can be learnt for future policy and practice. End-of-project evaluations are no substitute for continuous learning and adaptation of approach. Donors are ignoring lessons long since learnt: without local people empowering themselves to change those less tangible factors that cannot be counted, once donor money stops the roads will crumble away and the next generation of babies will not be vaccinated. These inadequate measures of assessment – and the effect of such measures on the design of aid – risks donor governments wasting, instead of securing ‘value for money’.
Eventual outcomes are often very different from what the logical framework required. Stuff happens. Power, history and culture shape the multiplicity of relationships and actors influencing any aid intervention. It makes more sense to design aid to recognize this. Experienced staff and consultants know it. But they are being forced to misrepresent reality in order to keep things simple for the taxpayer. They have to work with complex problems – such as why maternal mortality rates refuse to go down – as if they were bounded problems like my mildewed bathroom. In a largely unpredictable and dynamic environment, rather than choosing a single ‘best option’, a more value-for-money might be achieved by financing two or more different approaches to solving a complex problem, facilitating variously-positioned actors to implement an intervention according to their different theories of change and diagnoses and consequent purposes.
Aid bureaucracies have never recognised that effective aid depends on people and the quality of their relationships with each other. Sheela Patel of SPARC, an Indian NGO that supports slum dwellers federations has written that when SPARC was founded in 1984
‘ Donors gave money to us because there was a sense of trust. These funders did not set our priorities; communities of poor people did….. we were given all the space we needed. Consequently, SPARC and its partners now operate in nine states of India and help some 750,000 households….. I cannot imagine donors in today’s world granting an organization like SPARC the kind of latitude it required in its early years. Instead,[they] have become more focused on developing portfolios of projects, managing risks, and producing outcomes rather than on listening to communities, healing deep inequities, and supporting innovation’.
The origins of the results agenda lies in a mistrust that eats like a cancer into aid agencies’ capacity to make a difference. I am not convinced the emphasis on results will solve the problem of trust. On the contrary, it risks making things worse. The results rhetoric gets exaggerated by bureaucratic systems and by those middle level managers with little country level experience who are forcing grantees and development partners into straitjackets that constrain them from helping transform the lives of people in poverty.
We aid practitioners must start building trust. Steps in the right direction include paying attention to the inequitable power relations, including our own behaviour, which keep people in poverty; being modest about what any purposeful intervention can achieve; and communicating simply with taxpayers about complex realities.
Rosalind Eyben is a Fellow at the Institute of Development Studies and former Chief Social Development Adviser at DFID.