What are the implications of systems thinking for the way we design research?

If you stick around in your job long enough, you end up getting consulted a lot. Every week I seem to spend a couplebrian-savage-thank-god-a-panel-of-experts-new-yorker-cartoon of hours on skype banging on to assorted academics, NGOs consultants etc about NGOs, aid, development, life, the universe etc. The only upside (apart from a bit of human contact and an escape from reading/writing boring development documents) is getting the odd idea for a blog out of the conversations.

This week it was Open University’s Jude Fransman, who coordinates a seminar series on the politics of evidence in academic-NGO research partnerships. The topic was a bit of a chestnut – how NGOs understand, carry out and use research. I’ve written about it quite a bit in the past, but the new angle was taking the conclusions of How Change Happens (did I mention I’ve got a book coming out?….) on how systems thinking means activists should be rethinking their roles and applying them to the way NGOs (and others) do research. See what you think.

Getting beyond supply or demand to convening/brokering

Supply-driven is the norm in development research – ‘experts’ churning out policy papers, briefings, books, blogs etc etc. Being truly demand driven is hard even to imagine – an NGO or university department submitting themselves to a public poll on what should be researched? But increasingly in areas such as governance or value chains, we try and move beyond both supply and demand to a convening/brokering role, bringing together different ‘unusual suspects’ – what would that look like in research? Action research, with an agenda that emerges from an interaction between communities and researchers? V little of that going on at the moment. Natural science seems a bit ahead on this one – Irene Guijt points out that when the Dutch National Research Agenda ran a nationwide citizen survey of research questions they wanted science to look at, 12,000 questions were submitted and clustered into 140 questions, under 7 or 8 themes. To the organizers’ surprise, many citizens asked quite deep questions.

Diversity and Unusual suspects

A linked point. The research world is a very determined set of monoculture sub-cults/sub-systems, with discipline-centric accepted ways of doing everything, siloed sets of standards that inhibit interdisciplinarity etc. There are some good reasons for that (e.g. quality assurance), but there are some more disreputable ones to, in terms of paradigm maintenance in various disciplines. What would research agendas and processes look like that were decided elsewhere – by indigenous groups, women’s groups or faith organizations?

But what if something happens?......
But what if something happens?……

Critical Junctures

Social and political change is seldom smooth, and often crystallises around critical junctures – shocks like Brexit or the Turkish coup throw existing power relations and norms into the air, and when they come back down, new opportunities are briefly possible (not all of them good). Currently thinktanks are pretty good at this kind of rapid response, at least in terms of commentary (eg CGD and ODI on Brexit), whereas academic institutions are much more sluggish. Could/should NGOs do more, especially in going beyond commentary to using research to actively influence policy or attitudes in the wake of shocks?

Norms not just policies

The central role in development of long-term shifts in underlying social norms has often been underestimated in an aid business focused on project cycles, policies and decision-making, rather than big, slow attitudinal shifts. Gender is one exception – an area where activists working on environment or migration could learn a lot about shifting norms, yet is very under-researched (with some notable exceptions).

Precedents: History and Positive Deviance

OK, I have ranted about this a lot, but it applies in spades to thinking about research. We don’t spend nearly enough time thinking about what already works, either historically or today. Research could really help fill in historical gaps, whether on campaigns or redistribution. We should also pay much more attention to seeing where good stuff is already happening in the system (i.e. in real life, not just in our projects, for example identifying and studying villages with lower than average rates of maternal mortality and then going and trying to find out why).

Feedback and course correction – living documents

In systems, your initial intervention is likely to have to be tweaked or totally overhauled in light of feedback from Living documentexperience or events. Yet we still portray our research papers as tablets of stone – the last word on tax reform, women’s rights etc etc. Digital allows us to make them all ‘living documents’, subject to periodic revision, maybe even encourage wiki-policy papers (with moderation to keep things sensible). After all, I regularly get it wrong on this blog, and readers’ comments put me straight – what’s stopping us doing it across the board?

Then of course there’s Brexit – what is the role of research in an era of post-truth politics and ‘we’ve heard enough from experts’? Think that might need another post, though.

Any other thoughts?

And as it’s been a while since I last vlogged, and it’s summer. Here’s me apparently sitting in a bush in my garden

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our Privacy Policy.

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.

Comments

12 Responses to “What are the implications of systems thinking for the way we design research?”
  1. Kate

    I’m not sure being demand driven is that hard to imagine? What about the Sheffield 100 questions initiative that they did around international development research? https://www.sheffield.ac.uk/faculty/social-sciences/news/id100-1.461504. And there are some nice examples of other big research programmes that invest a lot of time in building relationships with government partners to find out what questions are most important to policy makers (e.g. http://comdis-hsd.leeds.ac.uk/wp-content/uploads/2013/03/COMDIS-HSD-Research-Booklet.pdf). I think it does need the interaction you suggest rather than being purely demand driven, for example, the ‘demanders’/intended audience might not know what research has already been conducted that could answer their questions (although from what I’ve seen, researchers aren’t always that great at really reviewing the literature to see if their proposed research is just duplicating existing work either…)

    • Heather Marquette

      I really agree with this. Very few researchers, regardless of where they sit, develop research in a bubble. It involves lots of interaction at various points in time. There are other issues that could be tackled: risk averse (and hugely competitive) research funding that rewards applications from people incrementally building on their previous research; incentives that discourage collaboration and encourage competition, leaving too many people swimming in the same pool but not linking arms for some synchronisation; demands for ‘problem-driven’ research that funds research focused on today’s problems using a ‘tomorrow’s problems’ time frame; etc.

    • Duncan Green

      And how many University departments (or NGOs for that matter) have made those 100 questions the basis of their future research agenda? I agree that this should not be about slavishly following public demand, but I think that danger is pretty minimal right now. As for being demand driven by funders (aka ‘government partners’ – well that is a different situation, wouldn’t you say?

      • Kate

        Just to clarify, government partners for the COMDIS example are governments of eg Bangladesh, Nepal, Pakistan. Agree they may not have the same priorities as civil society or people living in poverty in those countries, difficult questions around whose demand you listen to given so many stakeholders with different interests.

      • Ruth Kelly

        What about the SDGs? They push the research agenda in a certain direction and arguably they are based on a pretty good consultation, not about research needs necessarily, but about important problems – by combining the results of surveys about what the SDGs should be with the articulation of those points into research questions in the 100 questions, you might get at a research agenda that is probably reasonably responsive to public demand. Funders are likely to push academics and NGOs to do research that is at least in some way responsive to the SDGs. Not quite the same as listening to the public, but perhaps not too bad in terms of setting the scene for a bit more demand driven research?

        For example, one of the reasons the recent ActionAid report on industrial policy (https://www.actionaid.org.uk/sites/default/files/publications/what_a_way_to_make_a_living_pdf.pdf) is strongly focused on job creation and job quality is because these issues are consistently top priority for the young people ActionAid works with in our Activista network. Re. the SDGs, in the UN MyWorld survey (9 million respondents), the need for better job opportunities was one of the top three issues, along with education and healthcare. With a group of about 20 academics and a few INGOs we’re now developing a research agenda looking at job creation, which responds to a number of the 100 questions Kate mentioned.

        Still, the process for deciding the 100 questions was imperfect, even the authors recognise that the consultation missed a lot of perspectives (e.g. from the South, from the general public). So ActionAid, Uni of York, and Beautiful Solutions, with some other academics too, are hoping to do a series of workshops with activists, academics and artists (also with some consultation with a limited number of communities) to identify ‘alternatives’ that merit further research.

        Doesn’t really answer the need for more reactive work though.

      • Ruth Kelly

        (Although to be honest, there’s probably a bit of confirmation bias going on as the job creation angle fit nicely into an issue we were already interested in – I’m not sure what would happen if public consultations were to prioritise research on something that was completely outside our interest or existing priorities…)

  2. Heather Marquette

    There’s an anecdotal story about an academic at Harvard who drove his colleagues nuts for almost a decade – wandering the halls, never publishing anything, not being a ‘good citizen’. 9 years of this, seemingly making a mockery of the whole tenure thing. And then he published a book called ‘On Justice’. Arguably the most important political philosopher of the 20th century didn’t do focus groups or hackathons to decide his research agenda, and he changed several fields as a result, including philosophies on poverty and inequality.
    I’ve heard this story in discussions about the negative, presumably unintended consequences of the REF here in the UK, which tends to drive people to short-term agendas, journal articles instead of monographs, incremental additions to knowledge as opposed to transformative research that changes the way we see ourselves and the world around us. While I don’t disagree with some of the things you say here, there’s a need to differentiate the research ‘ecosystem’. NGOs aren’t think tanks, which aren’t universities. This isn’t a question of quality, because there can be fabulous research coming out of NGOs and think tanks, and crap research coming out of universities, but there are ‘typical’ audiences and ‘typical’ purposes for research for each that are very different (commentary isn’t research). And a healthy ecosystem should be able to cherish (and connect) all of it.
    So the big question for me isn’t how do you ask the ‘right’ questions, because predicting the right questions is an art form, not a science; instead it’s how do you best support and connect each part of the ecosystem, particularly in an resource-constrained environment where public trust in ‘experts’ is declining. Certainly I know my head is buzzing with all sorts of ideas right now, but it will take time to digest how this relates to all of the changes we’re seeing, and lots of cups of tea, before it begins to make up a coherent future research agenda. And I’m not going to blog any of it and give away potentially good ideas!

    • Duncan Green

      Hmm, a planet-sized selection bias in there Heather. How many academics have wandered the halls for nine years, and then produced precisely zip?! Yes it’s an ecosystem, and diversity is good, but I still think much of academia’s lack of systems thinking/diversity is a major problem in terms of its usefulness (and I would extend that criticism to NGOs too, albeit with different manifestations)

  3. Allan Moolman

    The challenge, as I see it is the continued emphasis the grand problems. Systems thinking should be providing the impetus to the idea that sometimes it’s the small, highly localised shifts in the environment that create significant change in the ecosystem.
    For most small organisations research support is absent when it is most needed – during the design and implementation of projects. Instead, hordes of researchers arrive when the organisation has developed some ‘good practice’ and then only to document the outcomes, to retrofit some causality that supports the researchers bias and then to move on to the next big thing.
    Questions are being generated every day by these kinds of organisations – researchers need to open up mechanisms to work with them. The opportunity for researchers to play roles as participant observers is always there. It’s just too hard – and not lucrative enough – to actually do the hard work of partnership. So inevitably most research institutions tend to focus on the big levers, the visible and well-trod avenues.
    Maybe also, we should reframe our research agendas as the pursuit of understanding and knowledge rather than scale?

  4. Linking to your point about historical data and Allan’s point about systems thinking and a need for research support during the design and implementation of projects. At this stage I think it can be useful to try to understand a system using the research, data and expert knowledge that already exists. This can help (0) ensure all stakeholders can develop and share in an understanding of the bigger picture (2) generate new hypotheses (3) help identify where there are gaps in knowledge/data and (4) identify which gaps in knowledge might actually affect the outcome one is interested in and so warrant more research, compared to other gaps which might be of interest to a particular group of researchers but don’t actually have a big impact on the final outcome.

    It can be time consuming and challenging to draw all this information together in a meaningful fashion but I think it is helpful for determining future research. It does challenge people to see their work in context. In fact I once observed a discussion where researchers were unwilling to engage in such an exercise because they feared that their corner of research won’t be seen as being sufficiently relevant or useful for the problem in question and so they would be excluded from being involved in the main project proposal!

  5. Arundhita Bhnajdeo

    Hi Duncan !
    1.When you mention that supply-driven is the norm in development research and distinguish it with demand-driven, Can i imply supply-driven as goal seeking and demand-driven as goal setting, as per Geoffrey Vickers distinction in his’ Art of Judgement’ ?(I am reading Vickers to understand Systems approach better) So, having said that, what could be going beyond goal-setting and goal-seeking? What would be a convergence there is what i was wondering.
    2. When you talk of positive deviance(also in ‘book review:The power of positive deviance’ ), though time taking, but is undoubtedly a paradigm shift in behavioral studies when people ‘learn by doing’. But when you mention that the role of experts should be limited to facilitators, wouldn’t those facilitators will also be in the long run fulfilling a goal of any particular project. Say, in a project of World Bank, in a particular block in India, where the project seeks to look for reasons for rising infant mortality rate and to further ‘solve’ the problem at hand, a positive deviance approach would be letting the villagers/communities decide the baseline, what they think and come out with the reasons for the problem and then finding out a positive outlier. But, in the process how is the role of the facilitator defined? Does s/he only facilitate the engagement process or also influence their baseline (by the very fact of being an expert from outside and their presence in that village) Does s/he interfere when the communities start moving out of the track in the process. And by out of the track, I mean outside of World Banks’s objectives. So, basically in a project of say any INGO, inside that projects periphery, you are advocating for a positive deviance approach, and not the approach taken up as a whole.
    3. Which brings me to my third point, if we talk about systems and a behavioral change, a transformational change in the system, shouldn’t all the participants(multiple stakeholders)in a project undergo that change, which will change the system per se too, by adapting and evolving?

    • Duncan Green

      Thanks Arundhita
      On 1) supply driven = researchers telling people what they think they should know. Demand driven = trying to find out what knowledge they need and want and then trying to help find it. Can’t see the link between goal setting and seeking, sorry!
      On 2) there is a weak and strong positive deviance approach. The weak approach is ‘find stuff that already works in the system, and see what you can learn, then turn it into a project’. PD advocates would, however, say that is only half the task – they advocate ‘social learning’, where the target community does the learning and responding for itself and the role of ‘experts’ is just to facilitate the process.
      On 3) Yes!

Leave a Reply

Your e-mail address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.