What have we learned about crisis/fragile states? Findings of a 5 year research programme.

February 5, 2013

Theory’s fine, but what about practice? Oxfam’s MEL chief on the evidence agenda

February 5, 2013

Why do some (better) alternatives to GDP get picked up, while others sink without trace? Useful new study on political economy of indicators.

February 5, 2013
empty image
empty image

Took me a while to overcome my reluctance to read a document subtitled ‘Deliverable 1.1’ on the front page (yuk), but I’m glad I did so.brainpool cover The paper ‘Review report on Beyond GDP indicators: categorisation, intentions and impacts’ (cracking title too…..) is published by BRAINPOoL – ‘Bringing Alternative Indicators into Policy’ (is that the sound of teeth grinding?).  The authors come from Eurothinktanks CUEC (Czech Republic) and the new economics foundation (UK).

But enough of the snark, because the paper is really good. Rather than add yet another indicator to the smorgasbord of measures that go ‘beyond GDP’, it tries to understand why some of these indicators get picked up, while others sink without trace. Asking the right question, as Einstein said, is a lot more than half the battle.

The researchers sought to answer that question with a mix of desk research, interviews with indicator nerds, and media analysis. First they tried to categorize the ever-growing list of alternative indicators. They came up with:

  • Level of impact – international, national, local
  • Indicator domains – environmental, social, economic
  • Indicator approaches – subjective, objective
  • Indicator types – single indicator, set/dashboard, aggregated, composite
  • Envisaged users – politicians/policy makers, public, experts
  • Link to GDP – adjusting GDP, “replacing” GDP, supplementing GDP

Media analysis showed that single figure indicators get far more pick up than complex dashboards, as do those that enshrine ‘simple and meaningful concepts’ like the Human Development Index or Ecological Footprint, which lead the pack by some way (see chart).

beyond GDP indicators media coverageBut a media splash, while useful, is not the main purpose of the indicators studied, which was overwhelmingly to influence policy. When have they achieved it and why?

  • The most basic influences noted were transmission to or reference by a decision maker.
  • The next step up from this is use of Beyond GDP indicators in assessment. Examples of this include GPI, ISEW and QUARS being integrated into regional or local official assessment frameworks in the US, UK and Italy. Also noteworthy is the Ecological Footprint being used to set environmental impact targets in several national governments.

And what lessons did the authors draw from all this?

“Indicators were successful when they had real relevance for policy makers. Crucially they need to measure something that policy makers believe they can influence. Subjective well-being indicators when used incorrectly can appear to fail this test, which explains efforts to build the evidence base for how policy can influence wellbeing. Another factor here is cost. In the current climate, indicators that provide clues for low cost policies, or indeed those that can help save money, are of particular interest.

Salience for a broader audience is also crucial and entails the elements of simplicity, understandability and good communication. Initiatives are effective when they allow one to produce a simple and attractive message that relates a meaningful concept. Using communications experts and avoiding taboo words were also identified as being important.

Indicators need credibility and legitimacy. Aside from the requirement of quality data, the appearance of neutrality was seen as the best route to achieve this with some interviewees comparing advocacy organisations’ data unfavourably with that of National Statistical Offices.

Developing the indicators with the audiences at whom they are targeted and/or encouraging participation (in the way that the Jacksonville Community Council Indicators initiative has done) is also seen as a key success factor. The importance of relationship-building also applies to policy makers with most initiatives that had achieved policy success citing direct face-to-face channels as vital.

happiness v researchersSeveral barriers to the success of Beyond GDP indicators have also been identified. The economic crisis was viewed by many interviewees as a challenge for this agenda as it has moved the policy focus. Ideology and vested interests are also noted barriers with subjective well-being and composite indicators receiving strong resistance from those with libertarian or right of centre political views.”

Not sure I buy that last point – isn’t a crisis just as likely to create opportunities for adoption of new indicators, especially if the failings of existing indicators are seen as partly responsible for the crisis? I remember being struck by a presentation from the South Korean statistics office (I really do have all the fun), arguing that Korea was adopting a new quality of life index, based on over 100 objective and subjective indicators of wellbeing, because GDP was incapable of explaining why an ‘economic miracle’ like Korea had the highest suicide rate and the lowest birth rate in the OECD. Failure is a great source of innovation.

The Brainpool project continues and is worth following (especially if they ditch the naff titles). Oxfam is getting involved, working with nef in Brazil and India to explore likely reactions by politicians and voters to possible indicators. It will build on Oxfam Scotland’s Humankind Index, which was an earlier foray into this field. I hope the research includes applying a more serious power analysis to the findings in this report: what fractions of the state, private sector, political class or popular movements adopt new indicators, either individually, or in coalition with others? When and why? That could get very interesting.


  1. Duncan – I think that the question of crisis as challenge/opportunity isn’t well addressed by either their comment or your Korea counter-example. In Korea, the government understood a crisis of understanding national well-being to exist; this is of course a ripe moment for going beyond GDP. While implicit, the quote you selected about the crisis seems to reference new GDP measures that cost money to address “other goods” such as ecology that libertarians/conservatives may be particularly likely not to value to begin with, and notes that this is even harder when each euro is “worth more”.

    One, in other words, is a crisis about the question at hand; the other is a crisis that constrains the decision space about a separate advocacy agenda.

    That’s not to say that the financial crisis couldn’t also be an opportunity for the right kind of new GDP (one that can promise to reduce debt or increase growth over time for example). But the financial crisis in Europe and the Korean question of suicide each make certain mental frames more salient, and depending on which ones, they will generally facilitate or obstruct efforts to change a given policy.

  2. BRAINPOoL seems quite adept at highlighting the selective hearing on which progress is measured. I remember my local currency having a rally against the dollar on one week and losing its gains the next week for the same reasons that the economists credited/discredited. I dont know if ecomomic development, impact and prospects will ever be calculated through algorithms and fancy equations, ever.

Leave a comment

Translate »