Who Wants to Farm? Hardly any young people, it seems. Should/Could that change?

May 22, 2014

The case for democracy – a new study on India, South Africa and Brazil (shame it’s not much good – missed opportunity)

May 22, 2014

New research shows aid agencies get better results if they stop trying to control their people on the ground, especially in complex environments (and performance monitoring can make it worse)

May 22, 2014
empty image
empty image

This fascinating excerpt from a recent Owen Barder speech to the little-known-but-huge Children’s Investment Fund Foundation (CIFF) covers two new papers owen barderon the management of development interventions, with big potential implications:

‘[First] a study of the evaluations of 10,000 aid projects over the last ten years from nine different development organizations. In this paper Dan Honig, from Harvard University, looks at whether different kinds of projects have been successful in different contexts, and he looks at the impact of organizational devolution within aid agencies. He takes all this data and does some regression analysis to try to determine the factors that affect the success of aid projects.

Dan finds, as you would expect, that aid projects are much less likely to succeed in complex or fragile environments, such as in post-conflict countries, and that more complex projects are less likely to be successful than simple projects.

So far, you don’t really need a Harvard academic to tell you this. The interesting part of his findings is that aid agencies that allow a large degree of autonomy to their people on the ground and in implementing agencies see a much lower decline in performance for projects in a complex environment than do agencies which exert a higher degree of monitoring and control.

According to Dan’s numbers, USAID projects scores are about 20% worse in fragile countries than in more simple environments. But if USAID had the same amount of organisational autonomy as DFID, Dan’s results suggest their project success would fall by only about 2% when they work in fragile countries. So Dan Honig’s paper confirms what you might expect: more likely it is that things will change in unexpected ways, the more important it is to have power and decision-making sit with the people who can see that change coming and respond to it.

moses as control freakThe second paper I want to tell you about was written by Imran Rasul and Dan Rogger from University College London. They have assembled an extraordinary dataset of 4,700 public sector projects in Nigeria. They have hand coded independent assessments of the projects’ completion rate and quality, and the complexity of the project; and they have conducted a rigorous survey to quantify the management practices of the 63 different organisations responsible for those projects.

Rasul and Rogger also find that more complex projects have lower success rates than simple ones, which is what you would expect. And they also find a strong statistically significant effect from schemes to create incentives for the bureaucrats and to measure performance – but these effects are negative, not positive, and much more negative for complex projects than for simple ones. Rasul and Rogger have some quite detailed information about organizational incentives, and so they are able to provide some evidence about what matters.

In summary, they find that freedom to adapt and respond improves results for most programmes but most especially for complex projects. They find that incentives schemes make very little difference either way – presumably because the organisations carrying out the work are motivated by intrinsic motivations. And they find that performance monitoring has a significant negative effect on results. Rasul and Rogger show that there is a modest complementarity between incentives and autonomy. They get a correlation coefficient of about 24%. One interpretation of this is that where organisations are able to measure results, they are more likely to be willing or able to grant autonomy to the implementing agents.

The important thing in their data, however, is that it is the autonomy, not the results measurement, which is bringing about the improvement. It follows that the results agenda is likely to improve project effectiveness when it is used instead of micromanagement of inputs and processes, but likely to make little or no difference if it is used on top of that micromanagement, as has often been the case with official aid agencies in recent years.’

Sounds plausible – does that resonate with a lot of aid agency staff outside HQ? Over to you


  1. Thanks, Duncan. This analysis fits it well into my experience in the Spanish aid agency. My job is to promote better effectiveness in our aid programs and I have always thought that implies giving the field more autonomy so our offices overseas can carry our a policy dialogue and define country programs based on ownership, using local systems, etc.

    I would be careful, though, because in a quick reading of this blog it might seem as though it is not important to improve our results focus and that more autonomy by itself will make things better. Both are important; the challenge is to find the right balance between giving more autonomy and being more demanding in orienting our work towards development results.

  2. Hi, this analisys fits in the ideas that emerged in the preliminary stage of data analysis from my current research about Portuguese aid in education in Guinea-Bissau. But Chris F. points a important aspect, in the program I’m studing it seems that the people on the ground had too much autonomy in the first years and the projet results and effectiveness was compromised. So it seems that the challenge is to find the right balance.

  3. Duncan, as I look back on my nearly eleven years with USAID, I recall an organizational dynamic that had a bit of everything rather than just a top-down character. When we did not have a strategy, we all did what we thought was good education programming. The reality is that we could not aggregate our efforts to show an overall outcome result. The coming of a strategy brought coherence in purpose and a new challenge (improving reading outcomes).

    So direction is not a bad thing except when the vision is prescriptive about how to achieve the desired result. Add to this a preoccupation with upward accountability that focuses on measuring things that are disconnected from reality and are unlikely to stimulate innovation.

    What I also know from experience is that not everyone desires autonomy. Risk averse people want to do as directed. I know lots of people who wanted to say they did the right thing (not just in USAID) even if it did not make sense. Importantly, autonomy is one of those virtues that is most effective when it negotiated transparently. I recall a positive experience whereby our Washington colleagues trusted us to make the right call so our relationships was not adversarial. Finally, I will make one more pitch for the Ostrom et al. (2001) piece on SIDA because I believe it is truly a valiant effort (the only one that I know of) to understand the mechanics of aid delivery.

  4. What if increasing autonomy makes more effective projects but is much less effective in getting funds from donors and taxpayers , so decreases the overall impact the aid agency can have ? Has anyone done the regression on how autonomy correlates with fundraising effectiveness ?

  5. This fits entirely with my xx years experience in the sector too..from being a CD to managing CDs to being Programmes Director in 2 different agencies for the last 14 years. The correlation between excellent CD and the best programmes is statistically clear. CDs need to feel trusted and confident from a shared understanding of what success looks, like the right frameworks to work within ( not too tight, not too lose)and the right support tailored to meet their needs ….and that’s it. I invest a lot of time in finding and recruiting the right staff …the most worthwhile investment any leader can make in terms of outcomes on poverty and inequality

  6. This is very relevant to the discussion of Result Based financing in the climate space. It suggests that the international climate community should be seeking to suggest indicators for results based finance but also ensure that the implementation process to achieve those results are as autonomous as possible.

  7. This is very interesting. It also fits with an accumulating weight of evidence which shows that all forms of Outcomes-Based Performance Management (Payment by Results etc) distort organisational practice, and undermine effective frontline practice – by making organisations focus on hitting targets rather than meeting the needs of clients as they present. For my research on this, see here: http://www.ncl.ac.uk/nubs/about/news/item/little-heresies-kite-seminar-series-is-launched

  8. In my experience (of managing a country program), despite the lip service paid by international NGOs to working in a more reflexive and adaptive manner, they are still unable to crack the fundamental challenge – getting the right people into the roles. The current (and perhaps next generation) of international development workers are still being schooled in linear project design and management theory that limits their ability to manage in complex systems. All of the grandest theory in the world (and multiple Masters degrees and PhD’s)are poor substitutes for experience, local knowledge, trust and humility. Not to mention a huge dollop of respect for the capabilities people you set out to ‘help’…
    Many of the people have worked with, from outside of the country, have a deep mistrust or ‘organic theory’ i.e the processes and understanding that’indigenous/embedded’ development practitioners use to make decisions on a day to day basis. There is too much trust in simplified(simplistic?) logic models that are unable to respond to the complex, intertwined and interrelated causes an effects that typify human social systems. Rather than treating people as complex subjects, programs are more often that not, designed and managed using ‘logical’ and ‘theoretical’ decision making frameworks to guide implementation. In this highly theorized environment indigenous and ‘grassroots’ development workers, who are unable to articulate their theory in a language that the ‘knowledge economy’finds pleasing, are overlooked.
    If we want to extend the autonomy question to the right point, where it will make a significant difference, we have to aim beyond the bureaucrats of the professional national NGO, INGO and donor community, to the organisations who are involved directly in working with people in respectful manner that recognizes the complexity of their daily lives and does not reduce their issues or priorities in favor of ‘logic’

  9. Great debate. For me a key question is how we help HQ staff (whether policy folk or corporate managers) learn to let go of trying to control front line operations. While centralising programme decisions may make give the illusion of control, the reality is surely very different? Once we admit that, can we start to empower front line teams to use their judgement to adapt to changing realities on the ground and build stronger, better strategic accountability? The subject of much discussion in DFID as we start to realign our operating framework to allow this to happen…

  10. Interesting to know how CIFF received this, if any of their staff are reading this blog. When I heard a presentation on the CIFF approach three years ago, I was struck by the intensiveness of their monitoring — far more demanding than even the most results-driven of government donor agencies — and I wondered aloud whether their data-intensive approach would exclude working in chaotic environments where data at the necessary scale are hard to come by. I don’t know how it has evolved since that time; but does CIFF see itself as requiring a “high degree of monitoring and control,” and if so, any second thoughts occasioned by this research?

  11. This an interesting finding and it is in line with my experience and thinking. I also agree that designing project approaches and implementation strategies sitting at the Head Quarters of international NGOs is not expected to be successful bust this is the actual practice mostly in USAID funded projects. I hope this finding may bring a new thinking and approach in implementation of projects mainly in inserting the degree of flexibility elements in designing project implementation approaches

Leave a comment

Translate »