Grammar fascism; 287m tonnes; development impact bonds;; Germany 4 FTT; latest on aid, inequality; protectionism v regulation; 'I will never be cut': links I liked

June 26, 2012

How should our influencing strategy vary with the kind of state we're working in?

June 26, 2012

Measuring well-being: what can international development learn from the health sector?

June 26, 2012
empty image
empty image

In this guest post, ODI’s Claire Melamed, Emma  Samman and Laura Kiku Rodriguez-Takeuchi pitch for partners for some work on developing new wellbeing metrics in development. Any takers?

What do we think we’re doing when we do ‘development’?  Surely, it has to be about making lives better for people as they themselves experience them. But we know surprisingly little about how poor people actually define ‘making lives better’.  And when it comes to aid allocation, poor people are rarely asked about their priorities. Several current initiatives are seeking to tackle this gap, with Oxfam Scotland’s work on the HumanKind Index providing one recent example.

The drive for results and value for money provides an opportunity to focus on developing measures of development outcomes that help nurse_patient6-6-10to identify which – of the many possible outcomes that aid spending can achieve – poor people value the most.

To be effective, any new metric needs to be easy to use for decision makers, so that it becomes an effective tool for allocating resources and planning interventions, and in their monitoring and evaluation. The challenge, and therefore the research question, is to establish whether we can devise a way of measuring outcomes, based on poor people’s own values, which strikes the right balance between a reasonable representation of reality and usability by policy makers.

ODI is currently developing a project that seeks to address this question, and exploring possibilities to work with a local partner to develop and conduct an in-depth pilot study.

Using health as a point of departure
Outside the development bubble, notably in health care, researchers have designed methods and measures that have gained substantial traction with policy makers, and which are widely applied. The measures sport deeply off-putting acronyms such as PROMs and QALYs, but a recent paper by Claire Melamed, Nancy Devlin and John Appleby reviews what can be learned from these methods and argues that importing and adapting health metrics could transform how we think about development outcomes.

In particular, approaches used in the UK health sector manage to reconcile the twin demands of complexity and simplicity.  Instruments have been developed that allow the impact of interventions to be measured according to how much benefit they produce using a common outcome measure, where ‘benefit’ is defined on the basis of the values of the general public.

health survey metricsThese values are calculated from surveys, in which many thousands of people have been asked questions designed to find out how they rank different health outcomes – such as freedom from pain, ability to move about normally and so on [see pic]. Their answers have allowed researchers to assign “weights” to each combination of different outcomes – numbers that can guide policy. These are now being used to measure outcomes and even, in some cases, to allocate resources.

Potential value of the approach
The key benefit of this approach is that it provides a standard metric to measure and compare the effectiveness of very different types of interventions: for an aid equivalent, the common denominator would be the impact on the lives of aid recipients as they perceive it.

With such data, decision-makers could readily compare the unit cost of achieving an equivalent gain in wellbeing across very different types of interventions, both across and within sectors. It would also provide an invaluable complement to more traditional methods used in the monitoring and evaluation of particular policies and programmes (e.g. Randomized Control Trials). By the conclusion of this project, we aim to produce an instrument that can measure wellbeing in general and within sectors such as health and agriculture, and that recommends investment priorities to policymakers on the basis of this accumulated information.

The pilot
malawi-census2-oct2010At present we are planning a pilot that will aim to determine to what extent and how we could adapt the methodology used in the health sector to establish the preferences and values of deprived people in a credible and useful manner. Key methodological challenges include developing the relevant instruments, the comparability of the preferences of different people and how preferences change over time, and how best to elicit the relative values that individuals and communities place on aspects of their wellbeing. Finally we will address whether and how we might use the information we generate on preferences and weighting to devise a standard metric that can be compared within and across dimensions of wellbeing.

We are looking for partners for this pilot phase, and would love to hear from any groups who might potentially be interested in collaborating with us as we plan for a pilot study (please email us at e.samman[at]


  1. important, timely but challenging considerations. At Re-Action UK we’re exploring the ideas of Man Max-Neef – Human Scale Development – as a way to understand/frame wellbeing. As a ‘recently departed’ employee of the NHS, I am reminded of the experience based design concept of Touch Points, whereby users as experiential experts inform the redesign/redevelopment/enhancement of health services. We’re interested in joining in the conversation!

  2. I would recommend taking a look at what Bath Uni’s Wellbeing Studies folks are doing:

    They have created a weighted index following a process very similar to what you’ve described. It’s being piloted in Zambia and India currently I believe. Very interesting stuff.

  3. I have worked with national and international efforts to measure outcomes of development. One thing that I have learned is that multiple metrics measured together often have a different meaning than individual metrics. Your highlights from the UK health sector are a good example of this. Taking just one item from state A and B would lead to a different conclusion about outcome than perhaps another item. However taking all metrics together creates a more holistic context in order to judge outcomes in more complex environments.
    Once you consider this effect, it is natural to ask what underlying context or trajectory is a better outcome for the whole of the metrics being considered? Another thing I have learned about the developing world is that more acceptable outcomes for its peoples often involve an underlying context of greater opportunities for individuals, families or communities. I think your interest in borrowing metrics from the healthcare field will probably help to underscore this.

  4. Hi Duncan,
    I have recently been appointed the QLD Global Education Adviser for Caritas Australia.A very different role for me after spending the past 30 years in the secondary education sector! I wanted to convey to you my thanks for your in-depth, challenging and knowledgeable blogging contributions-I read each one feverishly and have learnt so much from you. Thank you!! I hope to integrate those learnings in my new role as I venture into educating young people and the broader community about aide, development,advocacy
    and peace literacy.

  5. Very interesting approach and exciting to hear people talking about it. Obviously is speaks to Sen’s capability approach, certainly in terms of supporting people to define well-being for themselves.
    I’m interested to hear your views on ‘adaptive preference’ (i.e. in situations where a person’s choices are limited, the preferences of that individual will be shaped by their environment) and how you plan to deal with this. I find the idea quite uncomfortable but Nussbaum defends it by saying ‘recognizing the phenomenon of adaptive preference does not entail an unacceptable type of paternalism, if this recognition is combined with a version of political liberalism and a focus on capabilities (not actual functions) as political goals’.

  6. Thanks for your comment, belatedly. One of the aims of this research will be to investigate whether there is evidence of adaptive preference, and if so, how this affects stated preferences and relative valuation. This topic that not received a great deal of attention as yet with respect to the EQ-5D nor is there much empirical evidence from developing countries. However, should we uncover systematic biases associated with certain characteristics (e.g., age, or gender or poverty status) then it will be important to probe further into the underlying reasons, and whether certain types of questions can evoke less biased responses. Ultimately the aim of this work is not to replace other types of evaluation that measure outcomes according to other criteria, but rather to provide a way of integrating the views and perspectives of deprived people in debates over evaluation and resource allocation.

Leave a comment