When throwing evidence and facts is not enough. How Change Happens in the Humanitarian System

Here’s a sentence you don’t often hear. I just read a really interesting conference report. Transforming Change: How Change Really Happens and What we can do about it, by Paul Knox Clarke, summarizes a big 2017 discussion on the drivers of change in the humanitarian system, as well as the blockers. I reported on it at the time, but went back to it this week, as I am mugging up on the humanitarian system for some future work (details to follow).

One bit that jumped out was the box on the link between evidence and change – this one’s for all you ‘research into impact’ people.

‘Opinion was divided, both on whether we have enough evidence to support change in critical areas, and on the importance of this evidence in processes of change. While some participants felt strongly that the lack of evidence prevents change, others were clear that ‘we have mountains of evidence’; or that (with specific relation to cash) ‘we had 15 years of evidence collected. It wasn’t an evidence problem why it wasn’t being taken forward’.

we had 15 years of evidence collected. It wasn’t an evidence problem

When it came to the power of evidence to drive change, participants were able to give numerous examples where change had not occurred despite strong evidence. Although there were several credible investigations into protection failures in Sri Lanka, there seemed to be no change ‘in our consciousness or decision-making’. Similarly, while there is ‘very strong evidence of the value of core funding’ to local and national NGOs, many donors seem to have decided to go the other way, to more project-based funding.

In some cases, evidence is not being used because it is not easily accessible to those who most need it – particularly those making decisions on the ground. But a more common – and probably more intractable – problem lies with the way decisions are made. ‘Often

is only a small part of the decision-making calculus for policy makers’, or, as one participant suggested candidly:

‘as the Chief of Policy Analysis [for a large agency] I never read a journal, I never looked at a lit review’.

The panels supported research by ALNAP and others, which has strongly suggested that the use of evidence in humanitarian decision-making is limited, either because consideration of evidence is not an explicit part of the decision-making process, or because decisions are affected by a number of other considerations: politics, resource availability, or security. And, if we think back to the models of change presented in this paper, we should not be surprised if decision-makers are not entirely rational, or if decisions are strongly influenced by politics.

This is not to say that evidence is never used to support change. The meeting also considered a number of examples where the provision of evidence was an important part of the change process and some participants went as far as to suggest that evidence is a necessary, if not a sufficient, condition of change.

Change  seldom seemed to happen in a ‘straight line’, with people considering evidence, and then deciding to follow the evidence in their actions

Whether or not this is the case (do we need evidence for change, or can change happen without it?) it is interesting to consider how evidence was used to support change – because it seldom seemed to happen in a ‘straight line’, with people considering evidence, and then deciding to follow the evidence in their actions. Instead, participants explained how evidence was often used late in the change process to provide a sense of certainty, and so emotional security, around decisions that had already been made. Evidence was also used to bring people together and provide a common platform for discussion: ‘when there are disagreements, you can have that conversation at a very different level. You have it around technical evidence…and it allows you to surface what might be a tension’. It is interesting to see, in these cases, how the value of evidence lies in its ability to address some of the social and emotional challenges to change, rather than purely to address technical questions.

It was also interesting to see the circumstances under which evidence became important to a change process. It seems that evidence is more likely to be used where it is specifically wanted or commissioned, or where it is answering a specific question that decision-makers are already asking. It is also more powerful at certain times – in situations of uncertainty or doubt, people may be more likely to turn to evidence. Evidence may also become more important in decision-making where the organisation, or system, ‘raises its sights’ and focuses more on outcomes than on conducting activities according to ‘industry tradition’. And finally – and importantly – the importance of evidence in making decisions may differ from one organisation, and even one individual, to another. As one presenter said: ‘I threw facts at him and I threw argument at him, and he wouldn’t budge … Sometimes it’s only the visual, it’s only the human story that’s going to move people there in their heart, and not up there in their head’.’

Great stuff.

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our Privacy Policy.

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.


3 Responses to “When throwing evidence and facts is not enough. How Change Happens in the Humanitarian System”
  1. Masood Ul Mulk

    This is so true for the entire development sectors. In over thirty years at the working with communities we did endless sophisticated studies on institutional development among the communities. Donors wanted the evidence to become planning inputs for the programmes. But the frontline workers who were part of the studies were always however always hesitant to use them because the intuition part which they used frequently in their work was missing from it. I think the issue is not just evidence but what it misses out in programme implementation. During the earthquake 2005, conflic 2008 to 2018 and floods in Pakistan some of the local institutions had a very large role in implementation when the internationals left the field because of security situation but i have yet to see that reflected in any of the evidence

  2. Evidence can be a skewed element, in that project/programme developers, survey composers, researchers, compilers, publishers and end-users all have their respective agendas. This is especially true in regions of volatile governance. I have come across “evidence” too hastily amassed and tabulated because of unrealistic deadlines; falsified information to favour an anticipated or required perspective to be shown; as well as accurate and meaningful material that –however–an incoming new institutional or governmental administration finds encumbering for its own intentions.
    Therefore essential components of any evidence gathering and prospective positive usage must include a parallel effort to be well-networked with the final decision-makers and implementers; to present and discuss evidence with the people in the evidence area; and to work out ahead of time any realistic actions that may need to be taken in light of particular findings. Too many communities have been disappointed in having their circumstances continue despite proof of severe weaknesses.

  3. Nice article and, having worked on that ALNAP conference on change back in 2017, its lovely to see Paul’s paper continuing to have ripples today. I must confess though, your blog reminded me of the nagging doubt I always have when listening to the debate on evidence in humanitarian decision-making. It sometimes sounds a bit like attacking a straw-man: would anyone seriously argue that generating evidence is sufficient for change? And if they did, should we really be listening anyway? The idea that “evidence drives change” seems rather like the idea that, to use the crude analogy, “hammers drive nails”. No one thinks that, surely! We’re definitely missing the point if we take the observation that [decisions get made in quite rich and complex ways] to conclude that [evidence isn’t as useful as we thought it would be]. The interesting bit comes when we ask how evidence can best feed into the complex-swirling pot of intuitions, options, perspectives, needs and, yes, biases, that all decision-makers bring to their decisions. Maybe that means changing the shape and nature of the evidence. But it could also mean changing the shape and nature of the decision-making structures and processes themselves. Probably means both. Isn’t that our starting point?

Leave a Reply

Your e-mail address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.