Jobs, Justice and Equity: excellent new overview of Africa's progress

May 14, 2012

Famine and Foreigners: Ethiopia since Live Aid

May 14, 2012

Theories of change = logframes on steroids? A discussion with DFID

May 14, 2012
empty image
empty image

‘Theories of Change is just the latest attempt to shine a light on what lies behind, what makes everything work or fail. We constantly reach for new tools, but we keep alighting on small islands and losing the big picture.’ Jake Allen, Christian Aid

I recently spoke at a half-day DFID seminar discussing a draft paper by Isabel Vogel – ‘Review of the Use of Theories of Change in international development’. The draft is here (keep clicking) – Isabel wants comments by this Friday 18 May, either on the blog, or emailed directly to info[at]isabelvogel.co.uk. She is particularly looking for examples of documented theories of change (ToCs) originating in developing countries (as opposed to donor-funded programmes).

The level of interest was impressive – 40 DFIDistas in the room, plus 7 country teams via videocon and sundry NGOs and consultanty types. My overall impression was that Monitoring, Evaluation and Learning (MEL) is driving the ToCs discussion in DFID, and not always in a good way. So in my allotted 5 minutes, I stressed that ToCs should not become a ‘logframe on steroids’ (a phrased nicked from Alfredo Ortiz) and the importance of power analysis and ToCs as a permanent aspect of the planning cycle – and not 280px-Cynefin_framework_Feb_2011just for programmes but for policy and campaigns work.

Plus their usefulness (albeit in different ways) in all 4 quadrants of the Cynefin framework(Simple, Complicated, Complex, Chaotic – see graphic), rather than just in the simple/complicated quadrants preferred by development types. I also said we should throw away those horribly complicated ToC diagrams once we’ve finished them (lest they terrify those that follow).

The discussion confirmed these concerns. Lots of people (including many of the measurers) are fully aware of the risk and want to avoid it, but are struggling against powerful incentive structures that make it happen anyway (principally the results agenda, but also the difficulty of using non-linear ToCs in practice). Hivos, a wonderfully cerebral-but-practical Dutch NGO that has done a lot of thinking on this, talks about a broader range of ‘ToC thinking’ as a useful way to prevent it all being turned into just another toolkit (‘ticking the ToCs box?’). Rick Davies recalled that the logical framework approach was originally a separate exercise to filling in the logframe table, but they collapsed/reduced into the table due to the structure and working practices of the aid business. Might the same fate await ToCs?

What of the benefits? In addition to those discussed in previous posts, Joanna Monaghan of Comic Relief (a funder), sees ToCs as making explicit the hypotheses underlying funding decisions – ‘the rules of thumb we all carry around in our heads’. That allows partners to challenge them, if they think the funder has (gasp!) got it wrong.

People also saw ToCs as making people look at the evidence and identify what is known/unknown (that rather alarmed me – what were they doing before?), but also helping programmes adapt more quickly as new evidence emerges. From the MEL end, an explicit ToC also allows a discussion with beneficiaries on what indicators to measure progress against (rather than the funder just imposing them from outside).

ToC challenges
When it came to the challenges of implementing ToCs, the big headache is how to balance donor accountability (reflected in the pressure for measurement and results, and holding partners to account against pre-agreed plans), and the ability to use ToCs intelligently to learn and adapt to changing environments.

ToCs are about people engaging intelligently with the complexity and nuance of context and process. But how do you rigorously assess the quality of people’s thought? The development community usually focuses on process and outcomes, whereas ToCs may demand then a miracle happenssomething more like academic assessment on how deeply people are thinking about things. ‘Accountability has to be about trying hard enough. We never ask questions about critical thinking, only about delivery on a set of results which 5 years ago we thought we would be able to achieve.’ Stand by for quasi-professorial marks for project proposals (‘beta minus, must try harder’).

The more practical types worried over how you can balance constantly revisiting/revising a ToC with the need to get on and actually, you know, do something. One answer: pre-agree circuit breaker reviews at e.g. one year, two years into the project, when everyone knows the ToC is up for grabs; another – test (and fund) a series of ToCs in a pilot stage before deciding on a final ToC – a bit like the DFID-funded research programme consortia, which include (and finance) an ‘inception phase’ during which the recipient is allowed to test and finalise their research plans for the subsequent years. Perhaps there also needs to be a clear process for designated people to have access to a ‘red button’ change of direction in response to major contextual shifts that require a rapid revision of the plan (‘Mugabe dies’).

If failure is indeed a source of ideas etc, we need to create a safe environment to recognize, communicate and learn from it. That requires a shift in culture and incentives – e.g. circuit breaker reviews must have a convincing discussion on failure and what we’ve learned – if a project can’t demonstrate failure as well as learn from it, it probably isn’t trying hard enough.

Another plea from the practical peeps – can we separate out communities of practice from communities of theory, otherwise the practitioners are cowed into silence by the theory wallahs sounding off (who could they be thinking of?)

One final random thought: Is this (i.e. funding projects with plural ToCs, greater appetite for risk of failure etc) a suitable role for philanthropic foundations who are more able to take risks on failure than publicly funded donors?

9 comments

  1. It seems to me that the kernel in each ToC that does the work is a theory or action (or several theories if there are actors who are fundamentally different from each other). The problem is that this is usually implicit, and most ToCs would be a lot clearer and more useful if they made their theories of action explicit. the useful thing about theories of action (if they are any good) is that they lead to predictions about how different actors respond to events, including the actions of others, and so can be applied in projects and then tested out to see if they work. If they don’t, throw ’em out and try another one.

  2. Thanks for the summary of up to the minute thinking on project planning/monitoring/evaluation. As an MA student studying for an exam in Planning and Managing Development it is interesting and useful.

  3. I realise you are probably being realistic, but I find it worrying that you seen very willing to give up on the prospect of publicly funded donors taking risks on failure. I think that’s a battle worth fighting.

    First, risk goes with reward. Bigger risk of failure also means bigger achievements when things work.

    Second, a donor that won’t accept failure will not find it easy to encourage full and honest reporting and reflection on the part of implementing agencies.

  4. Interesting post. We at Twaweza and Uwezo like TOCs, because it’s a way of explaining our analyses and hypotheses of how change will happen, laying bare our implicit rules of thumb, forcing us to clarify what we mean. That does not mean everything is certain; ambiguity and hunches abound, but that we are more transparent about it.

    My only quibble is with your call to separate the communities of theorizing from the communities of practice. What happened to the good old dialectic? — where theory is informed by empirical experience and evidence, and practice is challenged by sharp questions and rigorous thinking? Surely both communities are better off together than alone.

  5. Thanks for the interesting post.

    Any recommendations on a good ToC guide? I recently read Hivos’ ‘De-mystifying a theory of change’ which I really enjoyed but would like to check out some more to get an idea of what other approaches are out there?

    Also, are you familiar with Caren Levy’s Web of Institutionalisation? Not sure if it would be specifically classified as a ‘theory of change’ but I think it’s one of the best development frameworks out there for analysing how to approach sustainable change in a given context (I wrote my Msc dissertation looking at how the framework could be applied to voice and accountability programmes / the active citizens and effective states dynamic that you write about in your book)

  6. Thanks for this posting, Duncan.

    You raise some really interesting points from the paper and the DFID seminar. Some of the points in your blog which match findings we had through CARE International’s work with theories of change.

    CARE has used theories of change in its strategy and programme design for many years now. We have identified the following seven components of a theory of change to develop a programme:
    1. A statement of the current situation and major underlying causes affecting the program impact group
    2. A desired long-term goal
    3. Domains of change and main stakeholders
    4. Pathways of change, which include breakthroughs and incremental changes
    5. Stakeholders
    6. Indicators
    7. Assumptions

    For more detail on these and how CARE International uses it in its design, visit: http://p-shift.care2share.wikispaces.net/Theory+of+Change+Guidance

    CARE International UK has just completed an EC funded project on design, monitoring and evaluation of programming for peacebuilding and conflict prevention entitled “Strengthening Capacity to Design, Monitor and Evaluate Peacebuilding Programming”.

    Through the project, three research teams in Uganda, Nepal and the Democratic Republic of the Congo were established and piloted methods of using theories of change in DME for peacebuilding.

    Highlights from the findings include:
    • Determining the appropriate actors to work with, and not just the easy-to-reach, enables better programme focus;
    • More explicit links need to be made between local level activities and national peace processes for desired changes to occur;
    • Conflict analysis is critical for determining the relevance of activities but is rarely done;
    • Staff often require support in ensuring their theories of change are sufficiently explicit;
    • Current project planning tools do not help practitioners articulate their theories of change;
    • Gathering evidence to validate a theory of change is challenging, particularly in conditions of conflict and fragility;
    • Critical review of theories of change needs to be undertaken in conjunction with other forms of evaluation to have maximum value;
    • Theories of change can encourage an overly linear approach, when change in conflict contexts can be more organic or systemic.

    Although this project focused on peacebuilding, many of the approaches and lessons could be applied to development projects more widely.

    To read the Peacebuilding with Impact report and its findings, follow here: http://www.careinternational.org.uk/research-centre/conflict-and-peacebuilding/155-peacbuilding-with-impact-defining-theories-of-change

    Finally, CARE International UK will publish Guidance for Designing, Monitoring and Evaluating Peacebuilding Projects: Using Theories of Change at the end of May. Again, this tool was developed from peacebuilding experience, but its application is wider.

  7. It seems to me that one of your conclusions is that the value of a TOC exercise relates to surfacing the beliefs about pathways of change, as well as assumptions, in a way that allows some challenge and discussion.

    My practical question would be whether there’s a real process for, and openness to, review by peers and partners. If only the program designer and maybe a supervisor review it until year 5, it will be a check-the-box exercise. But how to usefully get feedback on what it makes explicit, especially as most of these, for donors, will link directly to subsequent competitive procurements and so subject to restrictions on fair access?

  8. Thanks Duncan for the post and stimulating others’ contributions.

    With the current interest in TOC, I think it is interesting to look/try harder to find the balance between TOC or evaluative thinking and what you call the ‘practical types’. From my perspective, they don’t have to be different – it is more like a meeting in the middle. Some would argue that it is practical to think about how change will be triggered before starting the ‘doing’. Additionally, the ‘practical types’ can contribute to the development of theories (i.e. grounded theories) and therefore further enhance critical thinking.

Leave a comment