The Power of Numbers: Why the MDGs were flawed (and post2015 goals look set to go the same way)

August 15, 2014

At last, some evidence on the national impact of the MDGs. In Zambia, rivalry with other governments and measurable indicators have made a difference.

August 15, 2014
empty image
empty image

Yesterday’s post covered some new work on the MDGs’ limitations, so in the interests of balance (ahem) today Alice Evans from the LSE discusses her Alice Evans, LSEslightly more positive findings from Zambia. I would love to hear about other comparable research in other countries.

Over the last few years, I’ve been trying to fill in the evidential vacuum on whether/how the MDGs have had an impact at national level, by doing some qualitative research on maternal health care (MDG5) in Zambia, so I thought I’d offer some findings in response to Duncan’s piece.

My qualitative research began by examining what drives attention to maternal health care at district level. It turns out that supportive supervision is particularly significant. Top-down accountability appears to act both as carrot and stick for healthcare workers, making them keener to get results, but also feel more valued, as part of an important, collective endeavour.

In contrast, in clinics and districts where improved performance was neither recognised nor rewarded, I found that awareness-raising activities and in-service training only stimulated momentary attention. It’s accountability that really matters. As one senior hospital midwife explained,

Dr Mwila, [our previous supervisor] just used to come… and be sure the district was shaken a bit and do some supervision and support.  Personally, I was motivated.  He had that heart and concern for us in the rural area, so we felt a sense of belonging.  But now we feel we’re just working on our own…  They’ve got no time to see all our records.  They just depend on what we tell them, so you can lie.  But in those days they would just pick on one of your files and ask what went wrong. 

mother and childHow does that link with the MDG discussion? Top-down scrutiny and attention to maternal health care appears to have increased in recent years – both according to what the participants told me and the stats on resource allocation.

The belief in the unacceptability of maternal mortality appears to have been strengthened by the MDGs. However, merely setting these global targets in New York did not automatically increase attention to maternal health care in Zambia. Focus only sharpened more recently with the realisation that Zambia was lagging behind other similar African countries. These comments in interviews with senior managers in the Ministry of Health capture the embarrassment of being bottom of the league table:

MDGs – we have to be part of the world… We found that we are not on track. The commitment has been there but it was enhanced by the MDGs.

We are a stable country; to be put in that place is a shame [points to sharing rankings with conflict-afflicted states].

The MDGs may have been influential in other ways, such as by financing workshops to raise awareness of maternal mortality rates and amplifying bottom-up accountability by supporting local civil society organizations. To be honest, I didn’t see much evidence of the effectiveness of either. The MDGs may also have been significant by catalysing overseas development assistance. But this only seemed to occur after 2006, when donors started to realize just how slowly global maternal mortality rates were improving. Popular dissent and high-level political attention have also been catalysed by private media, exposing instances of inadequate maternal health care.

The paper that Duncan reviewed yesterday talked about the unintended consequences of the MDGs in terms of crowding out other kinds of spending, as governments game the system to improve their indicators. This is probably inevitable. For instance, a focus on the proportion of births attended by skilled personel (MDG Target 5.2) overlooks the quality of care provided: whether women are giving birth on the floor – as has been recently reported by Zambia Reports (a private media institution).

While these findings are admittedly limited (in so far as they only consider one MDG in one country) and need to be considered alongside other maternal health MDGcomparable studies, they do at least indicate the importance of attention to results, as well as how such attention emerges.

Maternal health care indicators appear to be prioritised when they are benchmarked, at district and national levels. Regional benchmarking and peer review has also been suggested by the High-Level Panel on the Post-2015 Development Agenda, to complement global monitoring.

Such benchmarking will be impossible, however, if the Post-2015 goals and targets are not measurable. As Charles Kenny (at the Centre for Global Development) notes, some items on the SDG shopping list are not measurable. E.g. Target 5.1: ‘end all forms of discrimination against all women and girls everywhere’. While I – as a card-carrying feminist – fully support that objective, it would not seem to make best use of the agenda-setting power of the SDGs to include it here.

Instead we need to think strategically, learning from past experience of what works in influencing development priorities at national and district level. My money is on measurable targets.

As for how we might do this in the case of trickier rights based issues, which Fukuda Parr’s paper said were largely missing from the MDGs, I think we can do better next time. On women’s empowerment and social transformation (for instance), I thoroughly recommend the comprehensive framework suggested by Caroline Harper and colleagues at the ODI. Instead of ‘ending discrimination’, they propose a number of measurable indicators for ‘women’s enhanced participation in political and civic life’. These include quotas; the percentage of women in government; and the proportion of voters who endorse the claim that men make better political leaders than women.

The latter idea – of incorporating survey data on social norms – is particularly novel. Such data could be compared over time, across countries and also within them – thereby not obsfucating sub-national differences. It would also reflect the degree of active citizenship in the general populace, not just that of elite women.

In short, I suggest that the purpose and power of the post-2015 process is to mobilise sustained commitment to important developmental objectives. This, it seems, can be achieved through carefully crafted, measurable indicators of progress, in conjunction with subsequent regional benchmarking and peer review.

Dr. Alice Evans is a Fellow in Human Geography at the London School of Economics. Follow her on Twitter at @_alice_evans

 

4 comments

  1. How does the country know that they are not on track regarding maternal mortality indicators? For the headline goal, most countries do not have good statistics on present maternal mortality, and certainly do not have the baseline indicators against which to make the determination. Did data collection improve as a result of the MDGs? How is the baseline assessed? Is the target deemed achievable?

  2. Easy spoken than done.
    there is no proof from this document of any improvement rather a kind of lip services. The investigator was told what she/he wanted to hear and that is it.
    Come up with more facts next time.

  3. Dr Hillary, in terms of evidence of increasing attention to maternal health care:

    (1) When health care workers, district administrators and senior managers were asked about contemporary health care priorities they all volunteered there had been increasing attention to maternal health care. Historically, they claimed there was more attention to HIV/AIDs, TB and malaria. (I did not prefix this open-ended question by conveying a particular interest in maternal health care. I grant interview responses have weaknesses; a longitudinal ethnography of health workers and managers – spanning twenty years – would have been preferable).

    (2) Additional indicators of increased government prioritisation include institutionalisation of MDG Target 5.2 as the Ministry of Health’s own Performance Assessment Indicator; a national programme to strengthen Emergency Obstetric and Neonatal Care (launched in 2007); a National Reproductive Health Policy (2008); Maternal Death Reviews in all districts (2009); a separate budget line for reproductive health and commodities (2009); direct funding to institutions training health professionals (2009); an annual ‘Safe Motherhood’ week and obligatory inclusion of MNCH activities in district action plans (2010); a Maternal and Newborn Health Road Map (2013); as well as increased government expenditure on family planning commodities.

    (3) There has also been improvement in health outcomes. Between 2000 and 2010, estimated maternal mortality reduced from 540 to 440 deaths per 100,000 live births, thereby reversing the trend of worsening maternal mortality in the previous decade (MMEIG 2012:45). This rate of improvement is not phenomenal: Zambia is not a leader in reducing maternal deaths, but there has been some progress and I have sought to learn from that.

    Scott, the Zambian government typically draws on Demographic and Health Survey data about the maternal mortality rate. Additional data is produced by the WHO. For births attended by skilled personnel the Zambian government relies on data gathered at clinic level. And yes, maternal mortality rates are notoriously difficult to estimate.

  4. Those of us looking in at “development work” from the public galleries want (and perhaps need) to believe that a holistic, multi-faceted approach to eliminating extreme poverty such as the MDG/SDG approach can be and will be, at the very least, helpful. Your post is helpful.

Leave a comment