How is India’s iconic NREGA social protection scheme doing? Interesting research from Tamil Nadu.

February 28, 2014

What’s missing from the ‘Active Citizens + Effective States’ formula in From Poverty to Power?

February 28, 2014

What do White House Policy Makers want from Researchers? Important survey findings.

February 28, 2014
empty image
empty image

Interesting survey of US policymakers in December’s International Studies Quarterly journal. I’m not linking to it because it’s gated, thereby excluding more or less everyone outside a traditional academic institution (open data anyone?) but here’s a draft of What Do Policymakers Want From Us?, by Paul Avey and Michael Desch. The results are as relevant to NGO advocacy people trying to influence governments as they are to scholars. Maybe more so. I’ve added my own running translation in italics.

The authors surveyed all senior White House officials involved in national security under both George Bushes and Bill Clinton. 234 out of 915 responded (pretty good response rate for people this senior).

Conclusions?

‘The gap between the scientific aspirations of contemporary international relations scholarship and the needs of policymakers is source of policymakers skillsgreatest the higher one reaches in the policy world. More surprisingly, this gap tends to be greater the more educated the policymaker. This is consistent with the argument that familiarity with advanced techniques instills greater appreciation for both their promise and limits.

Translation: the more pols know about a subject, the less they believe ‘experts’

Another conclusion we draw from this survey is that a scholar’s broader visibility – both in government and among the public whether through previous government service or publication in broader venues –– enhances influence among policymakers more than his or her academic standing.

Translation: get blogging, people

The primary constraint policymakers face in digesting scholarly, or any other writings, is lack of time. As one respondent put it, “any research papers that exceed 10-15 pages” are not useful to policymakers. Another noted that “I do not have the time to read much so cannot cite” many examples of useful social science scholarship.

Translation: work on those elevator pitches

We were surprised by two other findings from our survey about how policymakers get their information: First, unclassified newspaper articles were as important to policymakers as the classified information generated inside the government. This fact opens up an important avenue for scholarly influence upon policy if scholars can condense and convey their findings via this route.

Second, the Internet has not yet become an important source of information for policymakers, despite its ease of accessibility and the generally succinct nature of the presentation of its content. It could be a just a matter of time until a more web-oriented generation reaches the pinnacle of national security decision-making authority but we also ought to consider whether the internet suffers from weaknesses vis-à-vis traditional print media that dilute its influence. The plethora of internet news and opinion outlets, many of questionable reliability, combined with the lack of an authoritative source among them, may mean that the internet will continue to lag behind the elite print media because it exacerbates the signals to noise problem for policymakers.

Translation: good old fashioned press work beats social media

But our most important findings concern what role policymakers think scholars ought to play in the policy process. Most recommended that scholars serve as “informal advisers” and as “creators of new knowledge.” There were two surprises for us here: First, policymakers ranked the educational and training role of scholars for future policymakers third behind these other two roles. They also confessed that they derived relatively little of their professional skills from their formal educations. (see pie chart) The main contribution of scholars, in their view, was research. Second, and again somewhat surprisingly, they expressed a preference for

Obviously not a policy maker

Obviously not a policy maker

scholars to produce “arguments” (what we would call theories) over the generation of specific “evidence” (what we think of as facts). In other words, despite their jaundiced view of cutting-edge tools and rarefied theory, the thing policymakers most want from scholars are frameworks for making sense of the world they have to operate in.’

Translation: the best narrative (not the best evidence) wins

And the recommendations:

‘The most important roles for scholars to play are as both teachers and researchers, but our results suggest that both areas need careful rethinking. On the former, the findings of our survey should lead to some introspection about how we train students for careers in government service. We suspect that the focus on social science techniques and methods that dominates so much graduate, and increasingly undergraduate, training in political science is not useful across the board to policymakers. On the other hand, a purely descriptive, fact-based approach is not what policymakers seem to want from scholars either.

Three aspects of scholarship appear to be most important: First, policymakers appear to want mid-range theory. Policymakers do not reject methodologically sophisticated scholarship across the board but do seem to find much of it not useful. They prefer that scholars generate simple and straightforward frameworks that help them make sense of a complex world. They seem not so much to be looking for direct policy advice as for background knowledge to help them put particular events within a more general context. We interpret policymakers’ preference for theories over facts to the fact that like most busy people, they are cognitive economizers who need ways to make good decisions quickly and under great uncertainty. Along these lines, Henry Kissinger reportedly demanded of his subordinates: “Don’t tell me facts, tell me what they mean.”

Second, brevity is key for policymakers. We suspect that the reason that Op/Eds are so influential among policymakers is only partly due to where they are published; another important aspect of their influence is their short length. We are by no means suggesting that scholars only write in that format, but we strongly believe that research findings that cannot be presented in that format are unlikely to shape policy. Therefore, our recommended model is one in which a scholar publishes his or her findings in traditional scholar outlets such as books or journals but also writes shorter and more accessible pieces reporting the same findings and telegraphing their policy implications in policy journals, opinion pieces, or even on blogs.

Finally, a related issue is accessibility: Policymakers find much current scholarly work – from across the methodological spectrum – inaccessible. Policymakers don’t want scholars to write in Greek or French, but rather just plain English.’

Translation: tell better, clearer, shorter stories and you may actually be listened to

4 comments

  1. Thanks for sharing this gated research. For anyone who’s a fan of the ‘West Wing’ I don’t think the “surprising” findings are that new, but it’s good to see them coming from research rather than drama. For example, the Politics and Ideas thinknet have used a ‘West Wing’ story to illustrate one of the problems researchers experience in influencing policymaking. In short, that research can make a strong rational case, but it can still be trumped by emotion. http://www.politicsandideas.org/?p=450 And there are countless examples in the show of a good narrative opening the door for facts to follow.

    I like the translation approach to reporting on this research by the way: does it highlight some contradictions in the findings?

    Translation: get blogging, people
    Translation: good old fashioned press work beats social media

  2. Fascinating Duncan thanks for sharing. It is always helpful to get more ammunition to use in the argument for more blogging and elevator pitches from academics. What is even more useful is the reminder of the continuing importance of traditional media.

  3. This is wrong in so many ways that I’m putting together a comprehensive response, and will publish it in an eminent journal, 2-3 years from now. Your days are numbered (in the thousands).

  4. Thanks Duncan for this interesting post. Many of these findings will sound familiar to those working globally in the research uptake/evidence-informed policymaking arenas but still lots of food for thought.

    However, like Cheryl, I also picked up on the apparent contradiction between the ‘Get blogging’ and ‘Good old-fashioned press work beats social media’ translations. Both are essential and I think that those who fail to capitalise upon the potential of social media really miss a trick! Yes, maybe a lot of senior policymakers may not use this channel (although even this is changing), but you can bet your bottom dollar that many of their special advisers and other ‘internal influencers’ do.

    As a former civil servant working for the UK government, for me, social media, including twitter was a great tool to put out some ‘soft feelers’ for immediate feedback on impressions of research, relevant links etc which was really useful to help keep your finger on the pulse. It’s an essential weapon in the arsenal for any researcher seeking policy influence, although like any weapon, it has its strengths and limitations.

    I’m not surprised that issues of time constraints, brevity, contextualisation and accessibility came up, but was interested that the word ‘trust’ didn’t, although you hint at this in saying a) that the more policymakers know about a subject, the less they are willing to believe the views of experts and that b) visibility (familiarity) matters.

    Interesting finding that unclassified newspaper articles were as important to policymakers as internal classified information, but then again, public opinion matters!

    Finally, don’t get me started on my open access/open data hobbyhorses. Open data in particular is such an important objective, not only from the perspective of ‘freedom of information’ but also for researchers. Allowing intermediaries access to your data can help create far more interesting data visualisations to make your research come to life for time-pressed policymakers who won’t have time to plough through hundreds of peer-reviewed journal articles, even if they do have a particular interest in the subject!

Leave a comment