Where have we got to on Theories of Change? Passing fad or paradigm shift?
Theories of change (ToCs) – will the idea stick around and shape future thinking on development, or slide back into
the bubbling morass of aid jargon, forgotten and unlamented? Last week some leading ToCistas at ODI, LSE and The Asia Foundation and a bunch of other organisations spent a whole day taking stock, and the discussion highlighted strengths, weaknesses and some looming decisions. (Summary, agenda + presentations here)
According to an excellent 2011 overview by Comic Relief, ToCs are an ‘on-going process of reflection to explore change and how it happens – and what that means for the part we play’. They locate a programme or project within a wider analysis of how change comes about, draw on external learning about development, articulate our understanding of change and acknowledge wider systems and actors that influence change.
But the concept remains very fuzzy, partly because (according to a useful survey by Isobel Vogel) ToCs originated from two very different kinds of thought: evaluation, (trying to clarify the links between inputs and outcomes) and social action, especially participatory and consciously reflexive approaches.
At the risk of gross generalization, the first group tends to treat ToCs as ‘logframes on steroids’, a useful tool to develop more complete and accurate chains of cause and effect. The second group tend to see the world in terms of complex adaptive systems, and believe the more linear approaches (if we do X then we will achieve Y) are a wild goose chase. These groups (well, actually they’re more of a spectrum) c0-exist within organisations, and even between different individuals in country offices.
One confusing consequence is that the term ‘Theory of Change’ describes both a formal document and a broader approach to thinking about development work. For some Theory of Change is a precise planning tool, most likely an extension of the ‘assumptions’ box in a logframe; for others it may be a less formal, often implicit ‘way of thinking’ about how a project is expected to work.
And when used as a planning tool, ToCs are being implemented in addition to logframes rather than instead, and grant recipients are reporting to donors against the latter, with predictable results: ‘At end of day, we have to prioritise logframes – that’s what you’re being graded against’.
I came away feeling that this is all generating so much confusion that we actually need to find new words to describe the different understandings and approaches to ToCs.
For a start, it might help to distinguish between Theories of Action, which focus much more on whatever intervention is being discussed, and Theories of Change, which unpack how the system is changing (or might change in future) even absent our intervention. In my experience, the organizational imperative to do stuff, raise money, demonstrate impact, or just be active means that people spend far too little time studying and understanding the social, political or economic system before intervening (books like Portfolios of the Poor are a brilliant exception).
Enough ToCs are now being discussed and included in project plans and implementation for us to start seeing how they work out in practice. The leader on this is probably The Asia Foundation, with its collaboration with LSE, (discussed last year in Craig Valters’ excellent paper) but others are joining in – see the Comic Relief review or this 2012 study by CARE of the use of ToCs in 19 peacebuilding projects in Uganda, Nepal and the DRC.
Some of the more interesting benefits of ToCs identified during the seminar included:
- Opening up a conversation with donors about more realistic programmes and aims, allowing both donors and recipients to challenge unrealistic expectations and over-claiming
- Making explicit a lot of the implicit/tactic knowledge and analysis that underpin what aid agencies actually do (especially important given high levels of staff turnover – see below)
- Motivation: ‘reminding everyone why they are doing this’
- More quickly identifying things that aren’t working, so we can stop doing them (and spend the money on something else) – this is an aspiration, but Craig says he has so far seen few signs of it happening.
Some other impressions from the discussion:
1. It all comes down to people: ‘The ones who like ToCs are the development cynic types kicking against standard agendas/normative claims, who like having a language that forces people to think harder. It’s often those with more research/academic backgrounds. The others are not rejecting ToCs, but just not investing so much in them – they have lots to be getting on with.’ At the moment, all the success examples seem to require exceptional ‘development entrepreneurs’, but they are in short supply – what would a ‘where there is no Jaime Faustino’ approach look like?
2. Down with diagrams? OK, I am not a visual person (at school I was always rubbish at art), but I really hate the diagrams that
seem to accompany ToCs (see example). Others disagreed – simple clear diagrams can be a great comms tool, much more accessible than 40 page narratives. Trying to cram everything into a single diagram by just adding more and more boxes and arrows may be helpful for those participating in the exercise, but within months, they will all have left (staff turnover again) and the incomers will stare at the resulting visual spaghetti in terror and incomprehension. Maybe we should develop a special kind of ToC diagram that self destructs after 10 seconds, Mission Impossible style, and has to be endlessly redrawn by each new staff team?
3. How did everything get so top-down? think one cause lies in ToCs’ links with Political Economy Analysis: ‘a brief moment of omniscience when you can do a PEA and understand power relationships’. The ‘you’ in that sentence is always a consultant, a researcher, or an aid agency staffer – never a community or a (growl) ‘beneficiary’. Where are the bottom up, participatory ToCs? Why are IDS and Robert Chambers so absent from this discussion? ToCs talk a lot about power in other settings, but ignore power relations in our own organizations – what if they become ‘just another corporate stick to beat people with’? NGOs like Peace Direct and CAFOD are doing some bottom up work on this, but we need to be much more vocal.
4. Maybe we do need a toolkit after all: toolkits get a really bad press – restrictive, ignoring local context and tradition etc etc. But what’s the alternative? I fear it is the overpaid consultant or other expert flying in to explain all (see previous point) – ToCs could end up excluding local staff, partners and communities even more than the traditional approach. A good toolkit could include and reassure staff who don’t spend their whole lives on this stuff, but the question remains how to design it so that it doesn’t impose outside templates and preclude imagination, innovation and adaptation. Seems to me a toolkit would need to set out the kinds of questions to ask in designing a ToC, and the process for then implementing and adapting it. The Asia Foundation seems to be developing one, involving teams keeping and updating timelines of the evolving ToC, and having regular ‘time outs’ to revise it.
As for where we go from here, I think we have to consciously differentiate between ToCs as a bigger, better blueprint, all designed up front, and ToCs as a compass for helping us find our way through the fog of complex systems, discovering a path as we go along. But that will only happen if we can help people overcome their hunger for the certainty, however illusory, provided by linearity and up front design. At least donors are up for this – one DFID rep said she wanted ‘truly strategic partnerships in which not knowing is seen as a strength’.
Meanwhile, if ToCs are to endure, we need to demonstrate some quick wins from using them, and make it easier to do so. And NGOs and others need to step up and join TAF in offering themselves as guinea pigs.
[With thanks to Craig Valters for voluminous comments on my (much shorter) first draft]
Update: turns out that, as with all the most important questions of the age, South Park has it covered. Here’s the 9 second intro to ToCs [h/t Martin Clark]