Another interesting feedback loop in the aid system: a new report, The Marketplace of Ideas for Policy Change summarizes a
survey of 6,750 policymakers and practitioners in 126 low- and middle-income countries to find out which of the innumerable bits of advice and analysis churned out by aid agencies, international organizations and NGOs actually influence their work.
What’s most alarming is how original this is – I am still looking for a similar exercise on the MDGs, which might have made the whole post-2015 process less of a donor-driven gabfest. Right now the SDG wallahs should be reading this paper and asking – what kind of reporting structure might actually influence government behaviour?
A lot of the conclusions had me scribbling ‘NSS’ (No Shit, Sherlock, since you ask) in the margin: influence is to some extent a function of how much the organization spend on marketing? Wow, who would have thought it! But some are worth noting, and a few are genuinely surprising. Here’s some that jumped out for me:
Advocating on family/gender policy has the best chances of success, anti-corruption the worst: ‘The family and gender policy domain is one characterized by relatively high levels of external assessment influence, relatively low levels of (net) domestic opposition to reform, and reasonably good odds of success in reform implementation. This finding suggests that external efforts to encourage and support family and gender reforms may be particularly fruitful. Anti-corruption stands apart as the policy domain with highest level of (net) domestic opposition to reform and the worst track record of reform implementation.’
‘External assessment influence is strongest at the agenda-setting stage of the policymaking process.’ – i.e. get in early in the policy funnel, help define problems etc. Compatible with the whole PDIA approach.
‘Paying attention to “nuts and bolts of government” may result in greater assessment influence.’ Please note, all campaigners – think about advocating on boring but important stuff like data collection, staff training, info management.
‘Country-specific diagnostics generally exert greater influence than cross-country benchmarking exercises.’ OK that’s interesting – maybe publishing regional league tables, naming and shaming etc is not the best way to influence government (but perhaps tables comparing regions or cities is?)
‘External assessments that rely on host government data are more influential.’ Use government data, rather than your own, or some international body’s, and you are half way to getting buy in.
‘The longer an assessment’s track record of publication, the more influential it becomes vis-à-vis others.’ International organizations tend to have much more staying power than INGOs, producing annual reports on this or that, which slowly accumulate brand awareness and impact. By hopping from issue to issue, INGOs may keep the media interested, but they sacrifice impact.
‘Neither incentives nor penalties seem to easily explain assessment influence.’ Oh good, we don’t have to pay people to read our reports.
‘Prescriptive assessments appear to be slightly more influential than descriptive assessments, and decision-makers in the developing world seem to want more, not less, specific policy guidance.’ OK, that’s definitely a challenge to all the complexity wallahs and Doing Development Differently crowd who say that outsiders should focus on highlighting problems, not suggesting solutions (which need to be designed by local actors).
‘Assessments were influential because they promoted reforms that aligned with the priorities of national leadership.’ A ‘working with the grain’ argument that it’s best to try and influence ongoing processes rather than start new ones.
‘Senior government leaders and their deputies engage with external assessments in different ways. One potential interpretation of this finding is that leaders, mindful of their domestic audiences, project strength in the face of external pressure, while their deputies work behind the scenes to secure material rewards from donor agencies and international organizations.’ Love it!
Some important findings on the limits of external influence:
‘The picture that emerges is not one of governments being cajoled or coerced into pursuing reforms that align with donor priorities, but rather that governments pick and choose assessments based on whether they advance domestic priorities.’
‘External sources of analysis and advice rarely help to neutralize opposition to reform or build coalitions in support of policy change.’
And on the range of country types:
‘Some of the most successful reformers “go-it-alone” and shield the domestic policy formulation and execution from external pressure (e.g., Rwanda and Ethiopia), while others rely more heavily on external sources of analysis and advice (e.g., Liberia and Georgia).’
It’s just a survey – so no in depth interviews, focus groups etc to dig deeper, which I am sure would have produced further insight.
Massive blind spot on critical junctures – policy makers everywhere ignore advice until they need it, which is often after a scandal, crisis or obvious failure in previous policy. Building detailed timelines with decision makers would have revealed much more about how they take up policy advice and analysis at such moments.
Also nothing on the role of people and institutions outside government in persuading the state to adopt particular pieces of analysis – coalitions, insider-outsider alliances etc. This is a world where civil servants and pols read or don’t read/listen to reports – in real life, things are a bit more complicated than that.
The good news is that the AidData lab that conducted the survey plans to repeat it (and so accumulate influence, presumably). Now can someone apply this approach to the SDGs, please?