Adaptive Management looks like it’s here to stay. Here’s why that matters.

December 9, 2016

Links I Liked

December 9, 2016

How do we choose the most promising theory of change? Building on the context-intervention 2×2

December 9, 2016
empty image
empty image

One of the slides from my standard HCH presentation that resonated most during the many conversations and book launches in the US was the 2×2 on which kinds of interventions are compatible with different contexts. I first blogged about this a year ago, when the 2×2 emerged during a workshop of aid wonks, but the recent discussions have added some nice extra ideas to what this diagram does/doesn’t tell us:

context-intervention-2x2

Top Right: If you are confident about both your understanding of the context, and ability to run the intervention, you may be in the top right quadrant, where traditional linear approaches can work. Get out logframes and project toolkits. But Harvard’s Salimah Samji worries about ‘false quadrant syndrome’, whereby everyone thinks they’re in the top right, heaves a sigh of relief and says ‘great, we can just carry on as usual’. But what about hubris – they don’t know what they don’t know? What are the signals that you’re confidence is unwarranted and you need to look at the other quadrants? One might be if unexpected stuff keeps happening all around you (suggesting your confidence on context was unwarranted), or if your project keeps going wrong (ditto on intervention).

Top Left: This feels like where a lot of the adaptive management crowd are headed (see yesterday’s post). If you’re doing cash transfers in Somalia, and people have to keep fleeing the violence, the key is to have rapid feedback and response, enabling you to move your operations to where they have run to – you probably don’t need to rethink the merits of cash transfers.

What’s noticeable is how much more comfortable and active the aid business is above the line than below it. Below the line corresponds to aid peeps acknowledging uncertainty over their interventions – it seems the call for humility, especially about our own role, is a tough one for a lot of organizations. Adaptive Management is dipping a toe in the water of the tyranny-is-the-absence-of-complexitybottom right, through things like PDIA (Problem Driven Iterative Adaptation). But I see no-one coming to the table on the bottom left (‘you don’t know where you are + you don’t know what you’re doing’). That’s where positive deviance comes into its own, but no-one seems to be doing that, probably because it is all about seeing where the system has come up with its own solutions, so no role for experts, or for spending loads of money, so where’s the incentives? Yet I increasingly think this could be one of the big wins for the aid industry. Anyone doing it?

Oxfam MEL (monitoring, evaluation and learning) guru Mary Sue Smiaroski raises the interesting point that decent MEL should help push people between quadrants. Eg finding that your confidence is baseless, or after a couple of rounds of iterations, deciding you have found a good bet, promoting it to top right and scaling up.

Matthew Spencer, Oxfam’s new Director of Policy and Campaigns, was interesting in locating different campaign styles in different quadrants. He sees Oxfam’s campaign as bottom right – lots of trying things out and testing, complexity-signbefore moving up to top right and rolling out the big guns. Avaaz and change.org are top left – single interventions with fast feedback. He sees bottom left as inhabited by lots of small organizations campaigning away on different issues and occasionally throwing up big wins, as in the UK’s Modern Slavery movement.

The 2×2 is limited in scope – it describes the potential role of outsiders, not what local change looks like, but what I like about it is that it provides a way to discuss the merits of lots of different approaches, traditional, emerging and deeply unconventional, rather than proposing a single magic bullet.

Thoughts?

4 comments

  1. (With the caveat that I still haven’t finished your book…)

    Positive deviance in the bottom left quadrant feels underwhelming, for two reasons. 1) It seems like a fairly narrow intervention, compared to what you have in the other quadrants. Positive deviance may not be a single solution, but it’s definitely an approach and methodology, while the other quadrants are filled with more general principles. And because it’s more narrow, 2) it doesn’t apply to some of the most difficult problems facing that lower left quadrant. Positive deviance only works for certain types of problems, e.g it has to have a broad population of similar actors who end up with variable outcomes, some clearly better than others. That works for nutrition, education, health, etc. But can you use positive deviance on governance, peacebuilding, empowerment, human rights, etc.? (Again, caveat: maybe I just need to finish the book…)

    Even for those problems that are addressable by positive deviance, you have a prioritization problem in the lower left: not understanding the context probably means you don’t know what problems to address. So first you need something like relationship-building, partner engagement, etc to determine what problems to tackle. Then you can either move up to “fast feedback and response” or over to “experiments and iteration”. I would actually put PDIA in the lower left (not lower right) as part of the “muddling through” that Andrews describes is the process of getting to know the context, and moving out of the lower left quadrant (the classic move in any 2×2!).

    If you can’t get out of the lower left, which might be the case in a volatile political context, then the response might be something like building trusting partnerships with those who understand the context better.

    1. Like it Dave, and agree that positive deviance is only part of the lower left story. With comments like these, the 2×2 is maturing nicely!

  2. As Duncan says, this schema could bring clarity to a complex space of possibilities, and motivate wider reflection on them.
    But the contrary also holds true: it might obscure the space, by imposing a quite “static” categorization that does not correspond well with reality.

    Is it really possible to differentiate between “understanding of context” and “understanding of intervention” -as if they were independent- or are both interlinked? If you lack any of the two, your understanding of the other seems questionable too. Can you say that you understand the context very well if you are uncertain about whether the intervention will work or not? And do you really understand the intervention that well, if you are unsure whether or not it will be suited for the context?
    And as Duncan noted when talking about the ‘false quadrant syndrome’, you cannot seriously pretend that you really really know both of them well.

    In practical and fundamental terms, the type of understanding you need is precisely about the “CONNECTION of the intervention to the context”. Not two uncertainties (context vs. intervention) but just one, which multiplies both. A blurred uncertainty which, to some extent, is always uncertain about what it is uncertain about.

    That’s the knowledge that matters. That’s the knowledge that needs to be pursued. And that’s the knowledge that is always incomplete and elusive. No matter how much you know, there are still things you do not know, because the contexts keep evolving: it works today, yes, but will it still work in the future? You cannot be sure.

    And thus you could argue that there are not four objectives, or ways to act, but essentially one: to reduce the uncertainty about how to act.
    I am not talking here about a “silver bullet” that solves everything, but about a fundamental, complex process, which is risk-driven, and aims to reduce uncertainty as a way to maximize impact. A process that, to some extent, incorporates or is composed by elements like the ones included in Duncan’s diagram.

    But rather than a 2×2 space, I would think more like of a dynamic 3D spiral cone. Something like http://pgfplots.net/media/tikz/examples/PDF/spiral-cone.pdf – where you can slowly ascend, by doing iterative circles that move you away from ignorance.
    The higher you go, the more knowledge you have treasured about contexts, interventions and their combination. But in fact, you always stay uncertain and you can/must always keep doing efforts to further reduce uncertainty.

    How? Well, focusing on the uncertainty you can identify, be it about the context or the intervention. Or rather: of the context/intervention interaction. You have to ask yourself: What are the greater uncertainties, and also: which of them have associated the highest risks? Which of my working assumptions are more risky, and will make the intervention fail if proven wrong?

    So, in order to reduce uncertainty and risk, you keep iterating the following process:
    1. Hypothesize / Predict / Make your theories of change and action
    – If possible, these should be linked with the bottom-left in Duncan’s schema, and be inspired by positive deviance: start from the local context, and work with local actors. Only in case you cannot map existing deviance… would you be advised to invent experiments in the wild. In fact, the more you know about the context and intervention, the easier it should be to identify relevant positive deviance, or the local actors and contexts that could inspire your guesses.

    2. Design and run EXPERIMENTS (in parallel, if possible) which help you validate your more risky predictions
    – This is linked with bottom-right quadrant of the schema, “experiment & iteration”. When your uncertainty is big, you should rely on primitive, low-cost and quick to test experiments. Initially you would use thrown away prototypes and experiments, but as ignorance gets reduced, you could start using experiments that are integrated within your sustained intervention.

    3. Validate the predictions, based on the FEEDBACK that you get from the context when executing your experiments.
    – This would be linked to your top-left space, “fast feedback and response”. This is all about LEARNING, validating or discarding your assumptions and thus selecting the best ones to continue iterating and ascending in the cube.

    4. EVOLVE INTERVENTIONs, incorporating the learning that results from the previous validation, or the refutation, of your hypothesis.
    – This would be linked with the top-right sector of the diagram, “linear planning + evaluation”. Acquired knowledge is incorporated into interventions, but this is always a partial knowledge, and you cannot really stop questioning and adapting.

    This for steps learning process need to be iterated constantly. The highest the level of ignorance, the faster these learning cycles should be done.

    Interesting to note. This is very much linked to the reflections that, long time ago, led to the Agile Manifesto.
    Specifically to the classic “spiral model” from Boehm, which was a “risk-driven” iterative model:
    Boehm, B.W. (1988) “A spiral model of software development and enhancement,” Computer 21.5: 61–72, http://csse.usc.edu/TECHRPTS/1988/usccse88-500/usccse88-500.pdf

    Clearly, this is also very much linked to the PDIA, CLA, DDD and in general, adaptive management proposals.
    The way I see them, These approaches are not restricted to be used in one of the quadrants of Duncan’s 2×2 diagram, but aim to inform the transition among them, whatever the combination of uncertainties are.

    1. This is brilliant stuff Pedro, many thanks. I still think 2x2s are useful aids to thought, but how to keep them fluid, stop them from introducing a false sense of static-ness?

Leave a comment

Translate »