What can Islam teach secular NGOs about conflict resolution? (and human development, climate change, gender rights…..)

July 23, 2014

How can politics change to serve future generations (on climate change, but lots of other stuff too)?

July 23, 2014

Big Data and Development: Upsides, downsides and a lot of questions

July 23, 2014
empty image
empty image

One of the more scary but enjoyable things I do is be interviewed on stuff I know absolutely nothing about (yeah, yeah, I know – no change there then). You get to grasshopper around multiple issues and

Big Data

disciplines, cobbling together ideas and arguments from scattered fragments, making connections and learning new stuff. Great fun. This week, I’ll blog about a couple of these BS (blue sky, of course) sessions to give you a flavour.

First up, half an hour discussing ‘big data’ with a friend/researcher who shall be nameless (don’t want to destroy their reputation). Here’s some of the points that arose:

First, massive confusion on definition: depending on who you talk to, ‘Big Data’ means scraping massive amounts of existing data from sources like Facebook and Twitter (the UN’s Global Pulse has done great work on this); generating large volumes of new data; computer modelling of the existing data or using it for particular purposes like transparency and accountability, or targeting humanitarian relief.

Big Data is great when it throws up new questions and correlations, and stimulates thinking and discussion (The Economist had a fascinating piece this week on how Big Data is a natural partner of iterative, experimental approaches to change). But I’m dubious about it providing a short cut to political change, empowerment etc. There are lots of inspiring examples of using data to promote social change, but plenty of caveats and warnings against magic bulletism too, as the recent contributions to this blog from 3 transparency and accountability gurus showed.

How will Big Data evolve? It may follow the path of governance work – starting off with lots of supply (people building crowdsource websites that no-one uses), when that doesn’t work, move on to demand (citizens’ movements demanding data from baffled/incompetent/hostile governments) and then end up looking for combos of the two – hybrid institutions for data that combine old and new systems in new, context-specific ways; getting lots of unusual suspects in a room to find tailored solutions, including data-based ones, to agreed problems.

And what about the downsides? What are the risks of Big Data?

Big_data_cartoon risksData tribalism: mass media gives way to tribal media, as everyone splits off into their own online echo chamber, and increasingly has no idea what the rest of the world is thinking.

Big Brother Data: around the world,
governments are trying to close down space for civil society. CSOs routinely use a lot of IT, which provides a perfect channel for snooping and repression.

Don’t assume libertarianism will persist. Ok the internet still reflects its libertarian origins, but what if it is taken over by bad guys, whether governments or corporate?

What’s the link to inequality? Does the top 1% of the digitally connected have access to x times more data than the poorest 10% and is that digital divide growing or shrinking, between and within countries? What are the knock-on effects in terms of power and wealth?

Which all leads to a broader question. Is there something inherently individualist about the acquisition and use of data, as currently conceived? There are signs that it undermines collectivism, for example by allowing what were once pooled risks (eg National Health Service) to become customised, and eventually fragmented (her risk is bigger than mine, so why should I cross subsidise her with my taxes). If so, is a collectivist alternative approach– i.e. collective acquisition and access, data even conceivable?

Possible implications for today’s developing countries:

Big Data could of course allow them to leapfrog the painfully slow business of building solid national statistical capacity. A bit like mobiles v landlines.

Does building their data capacity in a world ruled by outside multinationals require a data equivalent of industrial policy? Perhaps countries should protect and nurture their infant data-related industries, only opening data ‘borders’ when national capacity and competitiveness has been created: a data equivalent of the East Asian tigers. But that would seem to go against any push for data comparability.

It feels like the governance of data is going to become ever-more important as a global issue. Who owns it? When can it be bought and sold? Do we need a UN Convention on

Self explanatory, really

Self explanatory, really

Access and Use of Information to try and lock in some positive norms around its usage?

I think I can guarantee that most, if not all, of this is complete nonsense, but I’d be interested to hear if anything resonates with data people

Next up: how could political institutions emerge that govern for future generations?

Update: Alan Hudson recommends ‘The rise of data and the death of politics‘, an excellent example of big data as dystopia, by Evgeny Morozov


  1. The trouble with all of the big data type efforts and the quest for comparability is the crowding out effect that this causes in terms of resource allocation for data that actually makes a difference. For example, in education, investments in ‘assessing of learning’ (benchmarking/testing) takes away from ‘assessing for learning’ (formative). I would say the same for spending on impact evaluations versus strengthening day to day monitoring. Certainly it is not zero sum and the two serve different purposes. But it does come down to controlling resources. This power is up there and they will say why it is a good idea to spend the money even if the information does not make a difference.

  2. Data has become really important in the context of the SDGs and many talk of a data revolution. For some countries, the revolution will be the strengthening of national statistical capability so that even the most basic data can be collected. Others see the revolution as the opportunities provided by data science, the manipulation of massive data sets to provide new products and services to deliver goals and address global challenges.

    The two views are at different ends of a spectrum and I agree there is much confusion and little knowledge on what countries actually want from data. Open data for all is a much called for mantra but ‘what next?’ from a development perspective? A great discussion and we’ve been talking to a number of organisations to understand who is working on ‘data and development’ (that in itself is a very confused picture) and on ‘what next?’ but don’t yet have an answer – any ideas? Get in touch if we’ve not talked to you yet!

  3. Useful piece, thanks. I particularly like your one para summary of the evolution of work on governance :) For those interested in the sorts of issues Duncan raises, I’d strongly suggest a couple of things.

    On the “yay for open data” side, have a look at what came out of the Open Knowledge Festival last week. http://2014.okfestival.org/ including things about the Open Development Toolkit http://opendevtoolkit.net/en-US/ You should come along next time Duncan – more talk about politics and power (and open data for what?) would be welcome!

    On the “not so yay for open data side:”, have a look at Evgeny Morozov’s piece from Sunday’s Observer newspaper in the UK – on “the rise of data and the death of politics”. http://www.theguardian.com/technology/2014/jul/20/rise-of-data-death-of-politics-evgeny-morozov-algorithmic-regulation

  4. Interesting post. I particularly like the idea of inequality in big data access.

    I have two concerns/considerations for big data as it relates to development. First, we need to keep a focus on root causes of issues (such as power dynamics) and not simply predictors of issues (which are often more proximal). In big data analysis, a focus on inference of causal factors (preferred by stats folks) is being rapidly replaced by a focus on prediction (preferred by machine learning folks). Although wonderful for business and services that care more about predicting outcomes then understanding the academic “why”s, it can make us focus too downstream. The military operations in Iraq show this well. Big data lets the military know well where violence and uprisings are likely to happen, thus enabling the military to respond rapidly; but it takes the focus of the big picture systemic and structural changes necessary for stabilizing an area (similarly, in education in the US the focus is on proximal indicators that can boost test scores rather than systemic inequalities). We can (and probably should) do both, but developmental change may need more of a focus upstream.

    The second related consideration is that we need to keep a focus on how to create change rather than just predict. Most public health issues have a wide variety of risk and protective factors that are well known. Big data can help us uncover ones we are not expecting. Yet, the question for development is often not whether something predicts bad/good things but how to change it. We all know of widely known risks/causes that get mostly ignored because of a political climate. Big data may help uncover important leverage points, but doesn’t guarantee that information will be used to create change (back to your often mentioned point about power dynamics).

    So, yay data and prediction, but it seems the hard work of change still falls on the same folks in much the same way.

  5. ‘Big Data’ is one of those terms that genuinely sets back human progress. It blurs and merges so many different things – most of them important, useful concepts – that the only decent thing to do is to refuse to engage in discussions where people seriously bandy the term around.

    Damn – I’ve just failed on my own terms.

    1. Thanks Tom, I think writing this post brought me to much the same conclusion. Too late for the readers, alas!

    2. Hey Tom, Reminds me of a quote “Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it…

Leave a comment

Translate »