What next for human rights organizations like Amnesty?

November 12, 2014

What are the big trends on conflict and fragility? Some great presentations at DFID

November 12, 2014

Should NGOs jump on board the Payment by Results bandwagon? New research suggests proceed with caution

November 12, 2014
empty image
empty image

Payment by Results is getting a lot of airtime at the moment, not least due to the indefatigable advocacy of CGD. Should NGOs be jumping on the bandwagon? Michael O’Donnell, MOD photoHead of Effectiveness & Learning at the UK network of Development NGOs, Bond, summarizes the findings of some new research.

Over the summer, DFID launched a new strategy on Payment by Results (PbR), which committed them to considering the use of PbR in all new funding decisions. Did this seal the judgement that working with DFID was a “demoralising dehumanising nightmare” as Robert Chambers memorably claimed? Or was it just a question of finding the right 12 Principles to make it work, as the not-quite-a-wonk-war between Stefan Dercon & Paul Clist and the Centre for Global Development suggested?

We found it hard to tell. A lot of the commentary was abstract and didn’t seem to speak to the realities that NGOs faced. So we asked NGOs with PbR contracts about their experiences, looked at evidence from domestic sectors in the UK where these mechanisms have been used for some years, and had a think. The product of that is a new report on “Payment by Results: What it Means for UK NGOs”. Launched yesterday, it aims to help NGOs think through the implications of PbR for themselves.

At its simplest, PbR involves the implementing organisation paying upfront for some or all of the costs of activities, and the donor only pays them back if they verify that pre-defined results were achieved. The theory is that it incentivises aid agencies to ensure that their activities are actually delivering changes in lives.

If donors allow implementers the flexibility to change and adapt their plans as they learn and progress, another assertion is that it will promote innovation. Ultimately, there should be better results and the donor or taxpayer only pays for success: a very appealing idea for cash-strapped governments keen to assure a suspicious electorate of their firm hand on the financial stewardship tiller.

Bond PbR coverBut needless to say, the reality is a lot messier. On innovation in particular, proponents of PbR appear guilty of some overselling. PbR contracts transfer financial risk onto NGOs whose unrestricted funds and reserves are precious. In extreme cases, the financial risk would be enough to put the NGO out of business if it was realised. NGOs thus talked to us about “playing it safe” in their bids for PbR contracts, going with tried-and-trusted interventions rather than innovating.

Other aspects stifled innovation too. Commissioners new to PbR themselves have been anxious to retain control over outputs, budgeting and reporting, and have been reluctant to liberate PbR implementers from the tyranny of monthly or quarterly auditing. At least one NGO has successfully pushed back on this, however.

The nature of what you’re trying to change affects your ability to innovate and adapt. The number of times that you can try new things, monitor outcomes, learn and change course within a typical three year project cycle differs hugely between, say, a project treating acute malnutrition and one aiming to change social norms about girls’ education.

A second area of concern is whether and how PbR contracts reward efforts to tackle inequality. A model that pays a fixed amount for a defined change, yet doesn’t take account of potential differences in the cost of achieving those outcomes for different people, can be the enemy of inclusion. In response to a PbR-based tender for water projects, for example, where bidders could propose to work in a wide selection of countries, we heard two separate examples of organisations explicitly deciding not to opt for South Sudan because they thought it would be too risky and would make their costs look higher than competitors who proposed to work in more straightforward environments.

Another organisation told us how they debated internally whether to include disabled children in their targeting for an education programme. They knew it would cost more to provide the full range of services needed to get those children into school than the average price the PbR contract would pay. Being a values-based organisation, they went ahead anyway. But could we bank on the same behaviour from all bidders in competitive processes? In the UK, where the phenomena of “creaming” the easy clients and “parking” the complex ones are well known, experience suggests we can’t.

These are all potentially manageable issues. Greater collaboration between donors and potential service providers at the design stage could help inform PbR Vennwhether PbR is appropriate at all, and if so how best to make it work. Flexibility in contract delivery can be negotiated. The percentage of the contract value that is withheld until results are achieved can be adjusted. Differentiated pricing can be agreed to ensure that service providers are rewarded for targeting marginalised groups. And NGOs can learn to become more adept at pricing the risk of failure into their bids.

But this is a complex business. Our guidance points to issues for NGOs to consider before bidding for PbR contracts, including their financial situation, their technical capacity (in bidding, monitoring and evaluation and contract negotiating amongst others) and their risk appetite. It also sets out different programming considerations that affect whether PbR may be appropriate or not, and how to mitigate risks of PbR producing unintended consequences.

Payment by Results isn’t inherently either a good thing or a bad thing. Any new idea needs a lot of testing, learning and refining. And that applies even more so to one that carries such high financial risk to organisations and which can undoubtedly work against its stated aims if handled badly. Our advice? Proceed with caution.

6 comments

  1. A few points:

    A massive amount of projects are implemented by private sector consultancies, especially those funded by DFID and USAID. PbR would seem to suit these guys more than it does NGOs because they’re probably more able to take financial risk. The DAIs and Adam Smith’s of this world can deal with a failed PbR project that they won’t ever be paid for better than Christian Aid or someone can. So PbR approaches would seem to push the development world even further in that direction.

    Thinking about how financial risk works in the private sector, would you not need to increase the payout available for working in high-risk countries like South Sudan to get anyone to work there on a PbR basis? I’m in Afghanistan at the moment and it’s pretty easy for a project to fail here. I mean, if you were implementing something up in Kunduz over the summer you’re going to have had some serious problems with any kind of project, because of a major Taliban offensive. I worked in Bangladesh before, and that kind of environment is so much less risky, so if the rewards are the same I’d put my money there.

    Final one – I’d expect PbR to cause nightmares for M&E people (like myself). If a lot of money is involved there’s going to be pressure on internal M&E people to find good results. For external people coming in to do M&E, ie consultants or review teams, you’re going to have problems with getting staff from the project you’re evaluating to be very open with you, if their NGO stands to lose hundreds of thousands if you find that the project hasn’t work. That would get in the way of learning a lot.

  2. When talking about payment for results or the results agenda in general, I’m always reminded of three gentlemen called Charles Goodhart, William Claude Dukenfield and Donald Thomas Campbell. The three ‘laws’ that are named after them (although Dunkenfield’s was not intended by himself to be called that way I guess) show a pretty clear picture if taken seriously: if a measure becomes a target (as results in PbR do), then there is an incentive for people to either cheat to achieve it (or cheat when measuring it) or to adapt their strategy to achieving this particular result even if the achievement of the overall development goal would require a change in focus or strategy.

    A second point that makes me a big skeptic of PbR is the finding “in virtually all circumstances in which people are doing things in order to get rewards, extrinsic tangible rewards undermine intrinsic motivation” (New Scientist, 12th April 2011, pp 40-43). This would explain why innovation and value-based behaviour is actually stifled rather than fostered by PbR.

    Incidentally, I just wrote about this here: http://wp.me/p1J0sr-8Y

  3. What I would like to know is why every agency needs a different name for for essentially the same thing. As if development speak didn’t have enough acronyms already… DFID calls it PBR, at the Asian Development Bank it is results-based lending (RBL), and the World Bank has a global partnership for output-based aid (OBA). My personal favorite is the Center for Global Development’s cash on delivery aid. COD Aid! Brilliant. I’m all for it.

  4. Experience in the UK domestic sector of NGOs seeking to take on PbR contracts and then passing on some or all of the delivery risk in the PbR contract to external funders has been largely positive. I could cite examples of charities saying that PbR contracts have given them space (and indeed forced them) to reflect on how they seek to achieve their charitable mission; has enabled them set aside their risk aversion and try new things; has let them plan for longer-term projects than they usually get funded; and inspired them to gather more and better data – and then use it for informed self-improvement.

    But to get there, there has been a lot of money floating around to at least part-finance all of the feasibility and investigative work, supported by the Cabinet Office, the Big Lottery Fund, Big Society Capital and others. The funding landscape is different in International Development.

    1. Excellent piece Rita, thanks – I learned a lot about PbR and where it is going wrong (sometimes). Depressing how often potentially good ideas get distorted by the practices and biases of the aid business!

Leave a comment

Translate »