What did I learn from a week discussing Adaptive Management and MEL?

Just got back from an extraordinarily intense week in Bologna, running (with Claire Hutchings and Irene Guijt) a

Bologna: Tough gig, but someone had to do it

course on ‘Adaptive Management: Working Effectively in the Complexity of International Development’.

The 30 participants mainly came from NGOs and non-profits, but with a smattering of government officials and consultants. What made the discussion different from previous AM chats is that they were largely involved in the nuts and bolts of Monitoring, Evaluation and Learning (MEL). Broadly, they fell into two camps: MEL people whose organizations find themselves increasingly being told to ‘be more adaptive’, and want to know what on earth that means, and others who have already bought into the importance of systems thinking and adaptive management, but now are wondering how to put it into practice in an aid sector that has lots of countervailing pressures – short project cycles, tangible and attributable results, the dreaded logframe and all the rest.

That meant that the week got much more into the practical side (here’s the outline – please steal Adaptive Management week plan). Students were asked to bring a ‘nut to crack’ from their day jobs, and Irene and Claire, who have decades of MEL experience between them, were able to provide lots of tools and ideas that can help navigate complex systems (while always eschewing a single AM recipe). I contributed my usual systems/ power/ how change happens waffle.

The main output was a set of ‘rules of thumb’ for putting AM into practice, generated by the participants. We’re still refining that – will post for comments once it’s ready. In the meantime, here are some random insights and observations from the week.

Important to think about when not to do AM – the Cynefin framework is really useful – trad linear approaches work in the simple and (with more thought) complicated quadrants. AM comes into its own in the complex one.

AM raises all sorts of problems and challenges for partnership, whether with organizations you fund, host governments that you are trying to work with, or partners in multi-stakeholder initiatives. Who decides when/what/how to adapt? What if they don’t want you to be adaptive, but would rather you just get on and deliver what you initially promised? Lots of host governments just want ‘training and trips’.

AM costs money: all this standing back, reflecting, real time evaluation, piloting, comms with donors and governments to keep everyone onside while the project evolves etc requires time and investment. Are donors and our own bosses ready for that?

The ‘toolkit temptation’ is alive and well: try as they might to stress that the key to AM is rising above the toolkit and ‘dancing with the system’, Irene and Claire were constantly being asked for them. That’s just how aid folk rock. I experienced the same feeling while writing How Change Happens and ended up compromising with a ‘power and systems approach’ that emphasizes ways of working, behaviour and questions, not processes or answers.

Do MEL teams need to be rethought, rebranded or even scrapped altogether and absorbed into project teams? Too often MEL is seen as low status bean-counting, internal police/executioner, or a combination of both. I used to be very rude about them til I realized that people like Claire and Irene are among the smartest people in Oxfam. In AM, the MEL function needs to emphasize the ‘L’ bit – how can MEListas become critical friends, incubators of new ideas and champions of AM, both within and beyond the organization?

Comms with donors, host governments and others are even more important in AM, eg during long inception and experimentation phases with few tangible results. But whose job is it? MEL teams are often happier with number crunching than telling inspirational stories. Should story telling become a recognized and separate function? Come to think of it, I guess it’s part of what I do at Oxfam, though we’ve never described it that way.

Trust is essential to AM. Donors need to trust that AM is not just bad management; host governments and communities need to trust that all this fiddling about is going to help them in the end. So how to build and maintain trust? There could be a role for ‘critical interlocutors’ – people at arm’s length from the project, who understand it and the issue, and can communicate with others with independence and authority.

But what happens if trust is abused? What happens when lazy or incompetent people do AM? It’s a bit like the US constitution – designed not for good governments but for bad ones (and getting a good stress-testing right now, as it happens). Is AM easier to game than traditional aid formats? On the margins of one recent discussion, I was told that unscrupulous management consultants now deliberately bid low to win contracts, then hide behind AM as an excuse to reduce their delivery targets.

AM in fragile/conflict settings. On the surface it looks like a no brainer – these are supremely complex places, so linear approaches are much less likely to work. But in practice, often aid workers can do little more than simply respond to events – riding the tiger/ ‘blaming ethe context’. That misses out on the other essential element of AM – reflecting on and revisiting assumptions and strategy.

How to spread the AM message? The aid sector typically propagates new orthodoxies through toolkits and protocols, but standardising those is anathema to AM. If not toolkits then what? Case studies? Charismatic champions (where’s the AM TED talk?). Exposing and ridiculing alternative approaches that fail? Other ideas?

Overall it felt last week like we have reached a crossroads: There’s now a ‘there’ there, a substantial case and growing body of evidence for when and how AM works, but numerous barriers and uncertainties remain. Maybe AM now needs its own more deliberate theory of influence – as one participant said: ‘what works best is waiting until we are 2 or 3 years into a project, when it hits a crisis and those in charge suddenly become interested in new approaches.’ Applying AM to the adoption of AM is a bit meta/turtles all the way down, but it may be what is needed.

Thoughts?

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our Privacy Policy.

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.

Comments

9 Responses to “What did I learn from a week discussing Adaptive Management and MEL?”
  1. I’ve been wrapping up some work with USAID on CLA (“collaborating, learning, adapting”—which isn’t quite AM but is close enough for our purposes) where one of the key takeaways is that there are many ways to do it, but that doesn’t mean any which thing you do it counts. And that leads to distressed practitioners: we won’t tell you exactly what to do, but some things are out of bounds, so good luck!

    What I think we’re missing are some archetypes* for distinguishing different ways to do AM. One archetype would be the very data-driven, fast iteration, likely tech-enabled approach to AM, which often accompanies high-volume service delivery or other large-scale work. Another archetype is more partnership-focused, slowly building relationships over time and facilitating community decisions. A third might be more like a portfolio investor, funding lots of small bets and seeing which ones take root.

    Our AM/CLA frameworks talk about these as if they were different *components* of AM—like they add up to AM together—but really they’re different *types* of AM. Trying to do them all together would be a mess. Each one might be more or less appropriate depending on a combination of contextual factors and the nature of the problem you’re tackling. And each one calls for quite different capacities from the implementing team, different roles for MEL, different potential for toolkits, different partnership approaches, etc. Discussing them all under the broad label “adaptive management” inevitably leads to confusion.

    (*Credit where it’s due: Manmeet Mehta of GKI really crystallized this idea of “archetypes” during a recent workshop.)

    • Varja

      Hi Dave – I like the archetypes suggestion, are you working on these? Could be a good complement to the standard Toolkits – which tend to cover everything and confuse people (and are boring), and could be more guiding / directive than a smattering of “case studies” which end up being too specific to transfer. Let me know if this is something to discuss, contribute to, review.

      • Hi Varja – The idea is pretty fresh so it hasn’t gotten much beyond the comment above! I’ll be putting a bit more thinking into it for a blog post in the coming few weeks. Would love your feedback once I get that together.

  2. Masood Ul Mulk

    Our experience with Adaptive Management in fragile and complex environments is that you needs two M & Es. The traditional M and E which caters for the logframe, and accountability requirement to do donors needs to be focussed on it and deal with the burgeoning information requirement from them. The second M and E is part of daily management and is closely linked to how the management responds to stimuli from the field. The traditional M an E remains restricted to the public transcript in programmes and is protected against the daily pressures which the field throws up in dealing with context, culture and power, while the M and E which is integrated into management records and learns and helps the management build responses to situation which the linear planning process never saw. It always surprises me when northern organisation expect the traditional M and E to lead the implementation process and give it a prominent role in it. Very little learning is taking place here. The learning is in the private transcripts which are part of the management and about which the donors based in a different cultural context has little interest in learning and undestanding.

    • Duncan Green

      Brilliant as always Masood, but also tragic – donors are kidding themselves that all that traditional M&E is somehow of benefit to the programme they are funding, but in fact it is an entirely artificial imposition, and you need to do real MEL alongside it.

  3. Outcome Mapping is a good case to look at for supporting AM practice – where the community of practice is the primary strategy for promoting the approach.

    IDRC developed OM in the late nineties, early 2000s, and through applying it along side their partners they developed a manual about the approach, the tools and how to facilitate them. They then made three important decisions.

    First, they chose not to mandate their new approach in any way. It was offered to their partners an an option and support was given to those that chose to adopt it.

    Second, they put all their materials in the public domain (which being a crown organisation is probably required of them anyway) so OM became free for all.

    Third, they recognised that the early adopters of OM were starting to take the approach and make it their own, and really keen to learn from each other. They provided support, for nearly eight years, to the Outcome Mapping Learning Community, which over time became the steward of OM. It still exists today, as an independent organisation (www.outcomemapping.ca).

    The original manual still exists and is useful as a reference to the original OM and for learning OM concepts, but the practice of OM has developed well beyond it. The OMLC website, with it’s resource library, practitioner guide, discussion forums and database of OM cases, is the platform for promoting OM and supporting learning and further development of the approach.

  4. Amy Gray

    I strongly disagree with the comment “MEL teams are often happier with number crunching than telling inspirational stories”. My experience as a MEL nerd is that I and my colleagues would much rather be emphasizing the “L”, working with others to generate meaningful information that is useful – less of the abstract number crunching and focus on “rigor” for the sake of it. Unfortunately my experience with MEL as a scarce resource is that donor and management accountability is emphasized at the expense of more innovative MEL that could support adaptive management. I think MEL teams could be strong allies, but seem to spend a disproportionate amount of time trying to explain to others inside and outside organizations why counting things will not magically lead to learning, and being “creative” with very small budgets to try and support useful “evidence” generation and participatory learning processes.

  5. Dear Duncan,

    I am just catching up on your blogs and came across this fascinating read. May I suggest a few more lessons and related blogs which may interest you.

    All of these comments relate to the fact that we are, like Manchester United or Aldershot Town FC, in the results business.

    Lesson one: Adaptive Management only works if it is applicable right from the Donor, to the Service Provider to the Beneficiary and then back again. If one entity is adaptive but the others are static we will not achieve results. https://www.linkedin.com/pulse/lets-get-real-results-based-management-owen-edwards/

    Lesson two: MEL, or RBM, has to better understand how adaptive management resources can respond to M, E or L. If the data says deliver in the rainy season but the procurement plan or staffing can not respond to that, then it is no good for an MEL expert to stand around and say told you so. The recommendations presented need to take into account how management resources can react. https://www.linkedin.com/pulse/dont-forget-bm-management-based-upon-results-owen-edwards/

    Lesson three: We must not forget the simple, disciplinary skills required for ensuring we achieve results. Focus, routine and quality data. Constantly. Only this way will adaptive management lead to results. This case study from Ethiopia demonstrates that. https://www.linkedin.com/pulse/delivery-1-million-land-certificates-approved-2-months-owen-edwards/

Leave a Reply

Your e-mail address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.