International Women's Day – what to celebrate, what to condemn?

March 9, 2012

The only interesting question on Kony 2012 – why did it get 60 million hits?

March 9, 2012

Do research funders have to chose between rigour, innovation and impact?

March 9, 2012
empty image
empty image

Another day, another committee discussing funding for development research (again, no details, sorry). This one innovationsproduced a really interesting conundrum, perhaps even a trilemma, that suggests it may be impossible to simultaneously achieve three ‘Good Things’ sought by research funders: innovation, rigour and policy impact.

First, rigour: Western research funders do their level best to judge the rival merits of research proposals based on the importance to poverty reduction of the research, and its methodological rigour. They require lots of detail on the methodologies to be employed, rely on anonymous peer review and deliberately ignore or minimise the importance of reputation – whether of the institution or the lead researcher. Only fair, surely? The last thing we want is a nepotistic old boys (and girls) network of decent chaps who all went to college together bunging each other research grants with no questions asked, right?

Up to a point, Lord Copper (sorry, Scoop reference – feel free to ignore). Suppose you want to fund innovation: as Tim Harford argues in Adapt, it may be better to find and back innovative people, even if they don’t have a nicely worked out project. Harford points to the outstanding record of the Howard Hughes Medical Institute, which explicitly urges (and finances) ‘researchers to take risks, explore unproven avenues, to embrace the unknown’. And the best innovators may well be at the start of their careers (Einstein and Newton did their best work in their 20s). In the UK, the Wellcome Trust uses this approach with its ‘New Investigator Awards’ that “support outstanding researchers who are no more than five years from appointment to their first academic position, but who can already show that they have the ability to innovate and drive advances in their field of study.”

who ya gonna fund?

who ya gonna fund?

young einsteinBut what about policy impact, increasingly sought by funders of research? Evaluations of why some research has more impact on policy and practice add another personalistic element. The distinguishing factor between research that sinks without trace, and research that has impact is often the relationships of the researcher with decision-makers, and that favours well established researchers, who know everyone (and taught half of them as students). There’s a link to the idea that research has maximum influence after a shock (scandal, collapse of a previous policy, financial crisis, natural disaster, conflict etc). In such circumstances, decision makers are winging it, desperately looking for new ideas, and they are much more likely to turn to people they already know and trust (like their old professor) for those ideas.

That goes for institutions too: in many countries, if you want to influence the state, who does the research matters at least as much as how good the research actually is. Will officials trust the source? Working with parastatal thinktanks in countries like China and Vietnam is often the best way to ensure research finds a fast-track to policy influence.

If a funder wants to pursue three such contradictory goals, what can they do? They could decide to fund larger projects with separate components, but I’m not sure such different approaches and personalities could work together. Or they could create different funding pots. The difficulties include being more willing to accept higher failure rates or being less insistent on methodological rigour (in the case of backing innovators).

As for the risks, what can be done to prevent this just becoming a pretext for some lazy nepotism? All thoughts welcome.


  1. it is hard for donors to find things to fund that meet these three criteria, simply cause they are terribly hard to achieve.. Trying to influence policy, carrying out rigourous research and impacting poverty are in three different time zones. Not to mention that as a researcher trying to bring these three into the same time zone you are also bound by the funding cycle. This is quite a conundrum!

    Also on another point, Who you are really does make a difference to get to the right ears – so to speak! I think this fact also affects donor choices – whether acknowledged or not. Hypothetically speaking if Einstein and John Doe presented proposals meeting the same quality and rigour standards, who would you give the funding to?

  2. There needs to be some criteria for what is defined as ‘rigor’ , ‘innovation’ and ‘policy impact’ : what would be a rigorous methodology? By the methods it uses or by the strong evidence-based findings or something else? and what would be innovative? something which nobody has done in the past? or something complex that has not yet been approached the same way before, for example? and what about policy impact? to what extent can you really say your research had a policy impact? a presentation of your research findings with policy makers would certainly not be enough, but we also know that research is not the only parameter that influences policy making , so some criteria of what would mean policy impact (a big word) would clarify things. I think if we clarify our terms, it would be possible to find research projects that meet many levels of these criteria.

  3. Some of your previous posts on complexity suggest that it’s more or less impossible to know what will work in advance.

    So I’d go for diversity – give the widest possible range of people and ideas their head – and then select what looks promising.

    And to hell with reputation – example Oxford University which for centuries has produced many of our leaders – and just look what a failure that has become – a cloned political class whose members have more in common with themselves than the real world.

  4. It sounds to me as if the committee needs to clarify its objectives (something committees often are not very good at).

    If, for example, changing the world is what matters most, the third “good thing”–policy impact–should take precedence.

  5. Great post. A couple of thoughts:
    If you are working in a development organization rather than a research organization or academic institution then it would make sense to me to emphasize potential policy impact (or else why are you doing it).
    But I also think that policy impact is not only about choosing research from those who have good connections with policy makers (which admittedly does matter) but is also about choosing research topics that are designed to address important policy questions (rather than just interesting ones), and to pick researchers who are able to write well and communicate their findings in a way that makes them more likely to be used.
    I’d also agree with P Baker that if you consider your research grants as an investment portfolio for knowledge then choosing diverse investments (in terms of both questions and approaches) is also a sensible way to ensure that you will pick some winners.

  6. There are practical problems created by insisting on rigour. First, rigour implies setting up hoops for proposers to jump through – and the people who know the shape of the hoops are the researchers who have jumped throuh them before. Newcomers, for example from developing countries, are at a big disadvantage. Second, research funding administrators often only say yes or no to an application – they haven’t the time to suggest modifications to the proposal. Canada’s IDRC are much better in this respect – they look for promising ideas and can work with the candidate researcher to improve the proposal. But that requires the funding organisation to staff itself approporiately.

Leave a comment

Translate »