Links I Liked

April 21, 2015

Scale, Failure, Replication and Gardening: Continuing the Discussion on the future of Big Aid.

April 21, 2015

Is a Data Revolution under way, and if so, who will benefit?

April 21, 2015
empty image
empty image

Guest post from the beach big Data Festival in Cartagena, Colombia, by Oxfam’s Head of Research and paid up member of the numerati, Ricardo Fuentes-Nieva (@rivefuentes)RicardoFuentesNieva cropped

A spectre is haunting the hallways of the international bureaucracy and national statistical offices – the spectre of the data revolution.  Now, that might suggest a contradiction in terms or the butt of a joke – it’s hard to imagine a platoon of bespectacled statisticians with laptops and GIS devices toppling governments. But something important is indeed happening – let me try and convince you.

A new research report by ODI  “Data Revolution – Finding The Missing Million” (LINK) (launched yesterday in Cartagena during a Data Festival) tries to make sense of the coming data revolution, and what it means for international development. According to the authors: The data revolution is “an explosion in the volume of data, the speed with which data are produced, the number of producers of data, the dissemination of data, and the range of things on which there are data, coming from new technologies such as mobile phones and the internet of things 

data revo 2

and from other sources, such as qualitative data, citizen-generated data and perceptions data.”

For the numerically minded (I proudly include myself in this group) this is a rather welcome transformation. Data, data everywhere – but then why haven’t we, number geeks, solved all of the world’s problems yet?

This is where things get interesting in the report. There are two (for simplicity) main sources of statistics: official and alternative. They both present advantages and particular challenges.

Official development statistics are, for instance, rather expensive, infrequent and often miss the extremes of the distribution. The report indicates that, globally, as many as 350 million people are not covered by official household surveys – most of them either very rich or very poor.

This gap creates massive problems for the most basic of global statistics. Take global income poverty, for instance. According to Laurence Chandy from the Brookings Institution, the ‘fact’ that 25% of the people living in extreme poverty ($1.25 a day) are in sub-Saharan Africa – some 414 million people-  is derived by extrapolating from household surveys dating from 2005 or earlier.

There are a lot of expectations about the potential of alternative sources, such as mobile phone generated data, but they are not without difficulties. The databases generated from these alternative sources are messy, often lack methodological consistency and require a lot of pruning and computing power to make basic sense of them.

But revolutions are supposed to be messy. One of the main challenges, the report argues, is the fight for space between official statistics and these alternative sources. This is, in a way, to be expected, as technological advances change the control of who generates statistics and how they’re used. There is some news that indicates that in Tanzania, for instance, the use of non-official statistics could even be

data revo


In short, the data revolution does have a political economy element. And its success will depend on whether official statisticians see the benefit of working together with the outside data scientists to learn better about the condition of a given country.

What are the opportunities to make use of these changes for the benefit of the poorest? There are two that I identify: how to use the increase availability of data for accountability and how to close the digital divide.

It won’t be easy though. There is hope that more information will automatically make governments at all levels more accountable but this seems naive. The report quotes Rakesh Rajani, formerly at Twaweza, an East African organization focusing on citizen accountability:  “There are problems of power and agency – they are the largest challenges for use of data-feedback. Just having new data or ways of analysing doesn’t trump those constraints. If the government was non-responsive before, technology and data won’t solve the problem or suddenly turn it into more responsive. Data doesn’t assure you that voice will count”

Similarly, on the digital divide, the problem will not go away solely by improving data collection. The authors give an example in New Zealand relating to the Maori population: “Many Maori do not perceive themselves has having benefitted much from the data collection and use of data. They perceive a real and immediate risk of greater data availability being used for ethnic profiling to their detriment”

social data revolutionAll this suggests that data availability and measurement innovation will not be enough. There is the need to for more data-driven active citizenship – or citizen engagement that make use of all this new information to promote inclusive policies and projects and ensure effective and appropriate use of resources. The report provides several examples of where this is happening already: Citizen-led poverty lines in Asia (much higher than the accepted $1.25 a day); Dwelling surveys used to negotiate resettlements in Mumbai; community organizations questioning the Ugandan government about the failure to meet commitments on health expenditure and increase health allocation, among others.

The report contains a lot more information and it’s hard to do justice to it on a blog (measuring poverty using roof materials as proxy, collected by satellite data? check; the rise of the Silicon Savannah in Kenya? check). It has a series of recommendations that seem obvious given the problems described – I found some of them, particularly the quick fixes, lacking in imagination. But overall the report is a very welcome piece – an easy, rather enjoyable read despite the seemingly esoteric topic.

So does this constitute a revolution or am I getting nerdily over-excited? I think new sources and, more importantly, effective use of this avalanche of data will turn many aspects of conventional government upside down, with huge potential to transform power and politics – if not ‘revolution’, what else would you call it?’


  1. Thanks for this article. Analyzing data from different sources can be a really exciting challenge. But, while there are many advantages of examining both official and unofficial statistical data, there are also a whole range of pitfalls and traps. These can lead to conflicting findings and misinterpretations. For example, let us take the issue of definitions. If I understand what is meant by poverty using a different definition than yours,then the number of poor people in our estimates is likely to vary. Who should we believe? Also, data collection methods can be problematic. For example, a straw poll does not necessarily reflect the perceptions of a given population. Questionnaire design can also influence the answers received, as can logistics, training, instructions on questionnaire completion, response rate, etc. The actual quality of each data source needs to be carefully examined in relation to standards, if we are going to use various data sources, and come to conclusions about what they are telling us. Newspapers articles are sometimes good examples of such distortions. A revolution needs some rules!

  2. Thank you Ricardo for these reflections. This revolution sounds both exciting and scary at the same time. The idea of a data-driven active citizenship that uses alternative evidence to question and hold into account power holders sounds extremely appealing. If information is power, is this a new form of democratization? New opportunities to triangulate data… All great stuff. I celebrate this data revolution, but not without some concerns. How to ensure that data becomes information and that we don’t get lost in a jungle of figures? How will this active citizenship distinguish the “good” data from the poor data? Can this offer a new form of social manipulation? Who will set the standards for ethical data?
    Exciting to see this party begin, anyways.

  3. As a fellow data lover, I share some of your enthusiasm and appreciate your and Ros’s statements that the power to define and act on problems is a huge issue. A city analyst recently told me that improved access and technology has let him give the higher-ups increasingly better data for the past 20 years, but he hasn’t seen any change in their desire to act on that data. Hopefully, when data is accessible by anyone, not just hired data folks, it will become a better tool for democracy. One issue I see missing here is that we often conflate having better data on problems/contributors to problems with having better data for solutions to those problems. In my experience, data for solutions is much harder to come by and can greatly lag data on problems.

Leave a comment

Translate »