Impressions of North America’s aid and development scene: the good, the bad and the ugly

May 14, 2013

What do we know about the impact of savings groups on poor African women?

May 14, 2013

How to Plan when you don’t know what is going to happen? Redesigning aid for complex systems

May 14, 2013
empty image
empty image

They’re funny things, speaker tours. On the face of it, you go from venue to venue, churning out the same presentation – more wonk-n-roll than rock-n-roll. But you are also testing your arguments, adding slides where there are holes, deleting ones that don’t work. Before long the talk has morphed into something very different.

So where did I end up after my most recent attempt to promote FP2P in the US and Canada? The basic talk is still ‘What’s Hot and What’s Not in Development’ – the title I’ve used in UK, India, South Africa etc. But the content has evolved. In particular, the question of complex systems provoked by far the most discussion.

Complex system US Afghan mindmapI started off with the infamous US military mindmap of Afghanistan. Although ridiculed at the time, the map looks like a genuine and nuanced effort to understand the country and is probably fairly typical of the complexity of power and relationships in any given country. The point is that such a system is complex, not complicated. Complicated means if you study it hard, you can predict what happens when you intervene. In contrast a complex system has so many feedback loops and uncertainties that you can never know how it will react to a stimulus (say $100m in aid, or an invasion….).

The crucial point is that most political, social and economic systems look like the map. Yet the aid business insists on pursuing a linear model of change, either explicitly, or implicitly because a ‘good’ funding application has a clear set of activities, outputs, outcomes and a MEL system that can attribute any change to the project’s activities – a highly linear approach. Other organizations – say forest fire managers, or the military, seem more able to cope with complexity, although I found out from a woman in one seminar who had served in Afghanistan that the power map was actually drawn up by a consultant, who was promptly sacked after showing the slide to General Petraeus, so maybe the soldiers aren’t so comfortable with complexity after all.

In denying complexity is obliged either to seek islands of linearity in a complex system (vaccines, bed nets), which may not always be the most useful or effective places to engage, or to lie – writing up project reports to turn the experience of ‘making it up as you go along’ that epitomises working in complex systems into the magical world of linear project implementation, ‘roll out’, ‘best practice’ and all the rest. That not only wastes a lot of staff time and energy, it also reduces the ability to learn about how to work best in complex systems.

So how should the aid system change? Overall, we need to think though ‘How to plan when you don’t know what is going to happen’ (my best effort at explaining complexity without resorting to jargon). Here are my bullet points, and brief explanations:

Fast feedback: if you don’t know what is going to happen, you have to detect changes in real time, but also have the institutions to respond to thatcomplexity road sign 2information (as was not the case recently in the Sahel).

Focus on problems, not solutions: Drawing on Matt Andrews’ work, the role of outsiders is to identify and amplify problems, but leave the search for solutions to local institutions. (At the World Bank, Shanta Devarajan pointed out the contradiction between this approach and NGOs’ preference for big, simple solutions – end land grabs, no to user fees. Ouch.)

Rules of thumb, not best practice toolkits: I am told that the US marines do not go into combat brandishing Oxfam toolkits and online resources on best practice. They operate on rules of thumb – take the high ground, stay in communications and keep moving. They improvise the rest. Aid workers on the ground operate far more like this than our project reports admit. If we were honest about it, we could have a better discussion on how to improve those rules of thumb.

Some possible approaches that spring to mind (and I would love to hear examples of others)

Work on the ‘enabling environment’ rather than specific projects: things like norms, rights or access to information

Evolutionary/Venture Capitalist approach: run multiple experiments and then zero in on what seems to be working best. Example, the Chukua Hatua project in Tanzania

Convening and Brokering: Get dissimilar local players together to find solutions – the outsiders’ job is to support that search, not do it themselves. Example, the TAJWSS water project in Tajikistan

But any attempt to move in this direction raises some fundamental challenges to the current structures of the aid industry:

Results for grown ups: The current approach to measuring results favours linearity. But rejecting results altogether is the wrong approach – both evidencebecause even those who recognize the central role of complex systems still want to know if they’re doing any good, and because the results people control the cash. No results, no funding. We need to get much better at ‘counting what counts’, and reclaim the idea of ‘rigour’ for qualitative and other methods better suited to complex systems.

Who to employ? Risk-taking, entrepreneurial, maverick searcher types have a hard time in an aid business dominated by bureaucratic procedures and risk aversion. Moreover, working in complex systems requires deep local knowledge of formal and informal power maps, something expats on a two or three year posting are unlikely to acquire. How do we turn the tables to attract and retain searchers, and value locally embedded knowledge?

Short Term v Long Term: Funding and project cycles are short term, change in complex systems is often long term. How can we bridge the gap, for example by combining good, plausible stories about the short term, with more rigorous impact assessment in the long term (how often do we go back and study the effects of an intervention 10 or 20 years after the funding has ended?)

How to keep/build political support given that working in complex systems means acknowledging a lack of control over what takes place and limits to attribution (no you can’t ‘badge’ the Arab Spring as created by Oxfam, USAID or anyone else, sorry). It also means greater tolerance of failure – a venture capitalist approach means accepting 9 failed start-ups for every 1 big success, but imagine what aid critics would do with a 90% failure rate. And how do we communicate and sell this approach to the public after systematically dumbing down the aid and development story for decades? (From buy a goat and save the world, to a post-goat narrative….)

Ben Ramalingam has been thinking about this for years, and writing about it on his Aid on the Edge of Chaos blog. His book of the same name is due out later this year, so let’s hope it can settle a lot of these issues (and doubtless raise many more).

13 comments

  1. Hi Duncan

    Interesting post, I think that last factor (winning political support) is the thing that will scupper any serious efforts to properly tackle complexity, in the short term anyway. Coincidentally, last month the ODI published a guide for planning and strategy in the face of complexity here: http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/8287.pdf

    And if you or your readers are interested in the findings from a 2yr research project on what complexity means for development programming as a whole you can find a summary here: http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/6811.pdf

    Harry

  2. Well, thanks Duncan. Complexity rules the aid discussion nowadays. The political support issue is the most interesting point of discussion if you navigate your organisation through complexity and the way forward would be to focus on the two anchors of your legitimacy: where your heading to (the end of peoples exclusion) and where your coming from (the norms and values of your organisation). If you can explain to your constituency the reason behind your decisions (while navigating your ‘messy’ context) on the basis of these two anchors, you have a start of a new paradigm in result based measurement. Or is this too simple?

  3. Truly excellent blog Duncan. I agree the issues of complexity are attracting much more (and much needed) attention in aid circles these days.
    I wish there were a way (a wiki? or a social bookmarking group?) where some of the disparate resources, and more importantly, conversations could connect. I feel we’d all move forward more quickly then. In any event, I wanted to draw your and your readers’ attention to a recent report on monitoring and evaluating market system projects http://bit.ly/17ZuDIZ. It focuses on the challenges of monitoring and evaluating market system projects rather than providing solutions, and proposes principles to support a complete rethink of the current M&E approach. While they are focusing on market system projects, the observations and insights are relevant for aid projects supporting change in any complex system (which as you so rightly note, is much, if not most, aid).

    1. Oh wow, thanks for this Tjip. Great to see Afghan spaghetti diagram meets TED talk meets complexity theorist with helpful suggestions about how to navigate it. Very useful.

  4. Duncan, how are we defining “complex systems”? You will agree that it covers a broad spectrum ranging from weak governance, to famine, to war. Justifiably, all these situations are “complex” in their own way, but I feel the aid industry also tends to lump everything together and ignore the fine print, as you have discussed. As a result, we have them approaching all these situations, with the same framework, including (horrifyingly enough), humanitarian and post-war reconstruction efforts. I find that in itself to be appalling. And then they wonder why things dont work out they way they had hoped them too. Im not one to overdo the nitty gritty of definitions, but there needs to be a clear delineation in the aid sector (I hate calling it an industry) about why they are doing where, so that they can then figure out the how.

  5. Not every complex situation is the same. The example of military in the war zone operating on rule-of-the thumb is quite different from trying to reduce poverty in a politically rather stable country. The military seem to behave almost as if they expect chaos to break out at any moment, where you need ‘forceful’ action to create control.

    I think to operate in the type of complexity we are generally facing in our work on poverty reduction, anti-corruption, AIDS/HIV, etc. we need a set of capacities of which I think the following are key:
    • Learn the tools. Increasingly the tools are emerging to rapidly get to terms with ‘traction’ and with changes in the environment almost in real time.
    • Develop a structure for process accountability. The decision-making process to initiate action in a dynamically changing environment requires individual and institutional capacities, like appropriate internal structures, cooperating arrangements, legal and operating systems (for instance how data is being released) as well as sound structures that monitor the decisions that are being made.
    • From one-off system assessment to sustained scanning and dialogue. We need to be able to have a sustained scanning of the environment to remain informed of change and emerging threats and opportunities, and to have a sustained dialogue with and between the stakeholders to adapt to change.
    • From inward to outward capacity assessments. In dynamically changing environments open and ‘outward’ capacity assessment methodologies are required, where both known and – initially – unknown stakeholders are incentivized to contribute to the assessment.
    • Engage the outside for solutions. We need to be able to solicit (capacity) development solutions from the environment in which we are working and deploy them productively. This requires opening (information) systems, engaging in dialogue in open manners, and internal decision-making processes that allow appropriate and flexible responses to the opportunities that are emerging from these open processes.
    • Visioning the future, open up and collaborate. Without clear ideas where to go, change action will become haphazard and directionless.
    • Internal systems need to accommodate flexibility. Where changes are fast and action changes accordingly, the internal operating procedures in the public sector need to accommodate this.

  6. Hi Duncan, a very important post.

    Consciousness of complexity raises a lot of question marks, and rarely the Definite Answers the media and politics want (remember the one-handed economist?).

    A question I am struggling with is on how to marry the need for short-term assessment, feedback, with the need for a longer term approach.

    It is a way out to choose for short-term contracts, while guaranteeing a long-term commitment to a problem, not a partner nor an approach (http://t.co/7HHuTjgKDm).

    By setting the goals, objectives, principles, focus areas in a very long-term commitment, short-term projects, with a possibility of renewal might be more stable than the current 5 year (election cycle in the UK) paradigm cycle, with 5 year projects followed by a rupture.

  7. re “the role of outsiders is to identify and amplify problems, but leave the search for solutions to local institutions”
    This does sound worryingly like the old definition of a consultant, as someone who borrows your watch and then tells you the time. :-(
    Won’t the local institutions” often already have pretty strong sense of where the problems are?

  8. Dear Duncan,

    This is an excellent debate, thank you for bringing such a practical approach to it.

    1. Complexity theory is amazing, but it’s math, not development. To plot a complex or emergent system requires all the data and all the variables. The DNA code. Development / social science it ain’t. It’s tough enough modeling complex behavior among bacteria in a petri dish, so forget about predicting the London or Lagos ‘system’. You land up with cool visuals, or at worst, some metaphors.

    2. However, complexity is a great way to remind ourselves that context and preexisting conditions are essential. The flower is already encased the seed’s DNA-data, so you want to work on the seed, not the plant (dammit, those metaphors again…).

    Overall, I’d say complexity makes the strongest possible case for simplicity in development:
    1. Think. Identify and agree on a problem.
    2. Map it out. Get data.
    3. Be simple. Break it down.
    4. Iterate.

    I’d venture we are (sometimes) in tune with 1, starting to pay attention to 2, but still missing on 3 and 4.

    Ps: the TED talk was great, but did you notice how solid scientific method slides into a fuzzy commonplace at the end? Perhaps complexity applied to development can contribute to map and inform, but not predict. That’s already progress.

  9. Eliding military actions with aid work is unwise and unscrutiny, collapses. They have clearly defined objectives, determined externally with little or no consultation, certainly not with local communities. Easy to have a rule of thumb approach in such a scenario. And they have plenty of ‘best practice books’ also.
    For my 10cents, try a closed set/ fuzzy set approach to evaluating change.

Leave a comment