How can we get better results from working with consultants?

OK this is a bit internal, but I thought it was interesting. We had a great 3 day session in Oxford last week with our global-research-team-March2012 #2rapidly expanding Global Research Team (see right – the prominent fella in the front is Martin Walsh, who’s our Global Research Adviser and your best point of contact if you want to talk to them). They’re a bunch of bright sparks from every corner of the globe with a big challenge: to make sure Oxfam’s thinking, programming and advocacy is informed by the best possible evidence. That means training and supporting other staff (lots of people in Oxfam commission or conduct research), working with partners who commission research, sometimes with Oxfam money, plus (increasingly) seeking research funding [note to all you academics out there – need a partner for your next research proposal with people and contacts on the ground and a strong global communications and advocacy capacity? Please form an orderly queue…..]

So what (other than the general excitement at seeing the team all together) stood out from the conversation (or at least the bit of the meeting which I managed to join)? Above all, just how hard it is to get good results from consultants:  the standard model for development research is to outsource it to specialist researchers. Some of them are local, some are expats, and all of them present a tricky management task. Often they are really expensive, with fees inflated by big aid donors all desperate for quality analysis. Many are unreliable, reporting late and often not sticking to the terms of reference. And then they disappear with all that acquired knowledge and all you have to show for your money is a bit of paper and a Word document.

It doesn’t have to be like that – I know and have worked with some great consultants. We’ve though a lot about this and even published a book on it, but it doesn’t seem to get any easier. So what makes sure you get value for money?First, be clear why you are putting everyone through this. Commission research to answer clear questions, not to avoid decisions – it should never be a substitute for direction, or a means of prevarication.

research doodleSecond, take time to find the right consultant; ask around; have preliminary chats with possible candidates; don’t just call the person you always use. Insist on competitive bids (although sometimes you have to skip that if a local guru is clearly the right person for the job and will tell you to get lost if you demand lots of CVs and form-filling). As with any appointment, hiring the right person is 90% of the battle.

Third, take time on the briefing: write very specific Terms of Reference, avoiding broad questions like ‘what is the role of agriculture in country X’ that allow them to regurgitate the literature and don’t help you take decisions. Don’t ask them to do your job for you (‘design an advocacy campaign on Y’).

Finally, don’t give them a contract and say ‘come back when you have a draft report’ – you need regular contact to avoid nasty surprises. Here are some top tips from my colleague Kate Raworth:

 As research manager, your attitude should be “eyes on, but hands off” – that is, engaged in the process but not trying to influence the actual findings.

 Is there a possibility for any staff or interns to work alongside the researcher for part of the work, to learn from them, and to get involved in the issues?

 Ask the consultant to present an early draft of the findings to a group of colleagues. Discuss the results and help the researcher keep focused on the key questions (but don’t be tempted to add new questions now – it is not fair to add to the TOR unless you extend the contract).

 When commenting on the first draft of the report, make sure you don’t simply ask a list of new questions. Look back to the TOR: have these original questions been answered? What other specific information is needed to complete the answer?

Do all that, and you’ll often get good results, but not always. We should also explore the alternatives, like hiring staff on short term contracts as a (perhaps) more manageable and cheaper alternative. We need to get better at working with PhDs and other forms of slave labour students – their language and timescale is often just so different from our own that we don’t manage to sort anything out. Has anyone worked out how to collaborate to both sides’ benefit? What about commissioning high-end local journalists instead, given that sometimes everything we commission seems to need a rewrite into more accessible language?

And often we are not even the ones commissioning the research. Instead, we are funding partner NGOs who in turn hire consultants, and there the challenge is even greater, especially with partners who are understandably prickly about funders trying to backseat drive and micromanage.  How to ensure quality control without Oxfam control? Lessons there include being proactive – offer to go and talk to partners, take part in meetings etc. Building relationships with both partners and their consultants is more likely to work than trying to order people about. Accompanying partners or consultants on a field trip can be a great way to build relationships and see what is going on.

And just in case you think I’m laying all the blame on them, a shout out to all those consultants out there: what advice would you give NGOs and agencies  about how to be better research contractors?

And here’s a Dilbert strip about management consultants – not the same thing at all, but who cares?

dilbert consultants

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our Privacy Policy.

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.

Comments

22 Responses to “How can we get better results from working with consultants?”
  1. Rhonda

    As one of those pesky consultants, I would also like to know the answer to that question. In my experience the four points suggested by Kate are also ones that I appreciate and think make for a better consultancy. Many of us come from years of working in the sector we now consult to and maintain strong attachments so are very interested in doing a good job. Constantly shifting/expanding ToRs are probably the worst offender.
    The other thing that I struggle with is not knowing what happens once the ‘dreaded’ report is submitted – many times I have included in ToRs follow-up, review etc to see how the work is being used, if it needs something else to make it useful but often this falls victim to the ‘too busy’ disease and I don’t hear from the client until the next job.
    I am currently experimenting with submitting a non-typical report for a client- – with their agreement – and with the idea that it might actually get used rather than shelved – I will let you know how it goes.

  2. P Baker

    I did a consultancy once and half way through I committed a cardinal sin – I asked to see the director and then questioned him about the whole raison d’etre of the consultancy and the project.

    This fellow was an ex-McKinsey man and I thought ‘it will be interesting to see how he puts me in my place, I’ll learn something.’

    In fact all he did was stone-wall me with ‘I see where you are coming from but I don’t agree.’

    In a zen-like flash, I received enlightenment – if you are a consultant, your job is to find out what your employers want to hear, if you ascertain this, you will be highly sought after.

    Which maybe explains why I’ve done so little consulting.

  3. Lauren

    I’m one of those cheap-as-chips students! I collaborated on a report and found the experience great, but I did wonder how useful I was to the team. Us interns need guidance which can be time-consuming, and I couldn’t stay on the project until its completion. I agree with Rhonda about the shifting ToRs- and the complexities that arise when different people interpret them in different ways!
    I’m now a PhD student and would love to do some long-term research with an NGO, but wouldn’t know a/what the NGO wants and expects and b/who to contact. Perhaps Oxfam could put together something on that?

  4. JM

    …you said it right: “Commission research to answer clear questions, not to avoid decisions”. As a consultant, I have seen sufficient examples a. where the contracting organization did not clearly know it intended to use the external input and b. that external inputs cannot compensate for internal leadership and management weakness or unwillingness.

  5. I’d echo what previous commenters have said about consultants wanting to do a good job, and am always frustrated when I hear consultants being demonised in some circles as money-hungry monsters who jump in and out of countries with no real interest in what they are doing.
    I also agree with what has been said about clarity of terms of reference and clarity on the purpose of the job. Most of us would rather do something objective than just write to a pre-defined conclusion. But it is often hard to know until you’re really stuck into a project what the client is after. This is why I try, these days, to spend quite a bit of time working with the client on clarifying terms of reference and the methods before even starting the work. Obviously for a consultant there’s a bit of an opportunity cost to spending 2-3 days doing this sort of thing before signing a contract. It’s probably in everyone’s interest if the client agrees to take on some of this cost – at least it gives them the opportunity to hand the main job to someone else if it turns out, based on this initial reflection, that consultant #1 is not right for the job.
    Another thing that NGOs (or donors) who contract consultants could do is engage with consultants in a more ongoing way. Most of us are genuinely interested in the follow up to what we propose but hardly ever get any feedback. If the recommendations didn’t work, tell us why. Ask us to contribute to or participate in your strategy discussions. Just like you, we need to earn a living, but don’t assume that we’re going to want to charge you for every conversation or meeting we have with you!
    And a final recommendation: if a consultant ever turns down work, don’t try and talk them into it. It’s happened more than once that a client has come back to me despite numerous refusals on the grounds that I did not have the skills to do what they were doing. In their view the skills were less important than the fact that I was the only person they knew who spoke the language and who had visited the country under discussion.
    By the way, although I’ve been a consultant for 6 years, I was guilty of all the mistakes described here, and more, during the several years I worked in NGOs commissioning and managing consultants myself…

  6. Silke

    I can only echo much of what has been said, but would like to emphasize the point: “engage with consultants in a more ongoing way” and the point about missing follow-up. In my view, that also means real criticism of unacceptable work. And I have seen it happening in the UN world that consultants hand in unacceptable (in my view) work but still get paid without any comment, and get even called again. Of course that also leaves one wondering if anybody ever looked at their report, and if anybody will in fact notice the difference between one’s own work and that of others…

  7. For most NGOs, research jobs will involve a local ‘partner’ in some way. They will either be expected to facilitate in some way or may even be the subject of the work. Ensure that there is consent that if full, free and informed. When that is not the case, it can undermine the work. Happens regularly.

    Clients of all stripes should be able to provide basic relevant information promptly. Not always the case.

    Clients should require copies of all research materials: interview notes, survey data, seminar notes, whatever. That will keep your consultant on his toes.

  8. Very good guidelines, thanks, I’m sharing them with other consultant listserves.
    Another benefit is that we’re outside the organizations and can illuminate excellence that others can’t see as clearly or champion, or highlight shortcomings that others are too timid to state (elephant in living room :)).

    I’ve had my own consulting firm for 10 years now. The very best consultancies is when it’s an ongoing learning partnership with the TOR being revised based on learning and feedback as the consultancy proceeds (e.g. why as the same questions throughout when we’re getting the same answers, instead, what new ones have emerged that are key). I love Matt’s comment on followup – we’d love to know how our recommendations worked, or even do more work on the project’s learning as it evolves.
    The very worst consultancies have been when I have been asked to either replace many of their team and work as if I was 5 different specialties without field support/ input or when the report was a ‘check the box’ for the donor, and the client had little intention to learn from my/ my staff’s findings.
    Cheers! Jindra

  9. Robert

    On what happens after the delivery, I wish organisations realised that their research has a wider audience than the person commissioning it, and that it could add to the sum of knowledge outside the commissioning organisation. Once delivered it doesn’t have to sit in a C drive or a desk drawer. It’s good if the question “What will we do about communicating this research?” is asked as the work is commissioned and ToRs confirmed.

  10. Irene

    Other thoughts from yet another consultant who has also hired other consultants of varying output quality: (1) ditto Rhonda on shifting/expanding ToRs – aaargh!; (2) absolutely do not pay for c**p; (3) PLEASE do not force collaboration between two or three people that you happen to like/think are good – they might not work as a team (speaking from painful experiences); (4) be graceful about timing of outputs as late reports are also influenced by contracting delays, unavailability of data and people, other organizational priorities; (5) make ToRs interesting (not sameoldsameold) and feasible (not 10 days for ‘priorities for global biodiversity’); and (6) allow researchers to co-publish the work.

  11. CB

    I’ve had a lot of luck working with secondments rather than consultants. It always seems like somewhere in the sea of implementing partners/ network affiliates/ supporters/ etc. we can find the skills we need. Usually paying an organisation for staff time is cheaper than hiring a consultant, the organisation appreciates having staff development opportunities, and we get far better results. It also has the added benefit of strengthening the relationship between our organizations, and increases the chances of collaboration in follow up. It’s probably not the solution for every project, but can really work well!

  12. Martin

    It’s great that this post has elicited so much passion and so many good suggestions. Reading between the lines of the blog and comments, and injecting something of my own experience as a consultant, I’d say that there are two distinct ways of characterising consultancy:

    The default is to see it as a commercial relationship, a contractual means to a practical end: you have a problem and you pay for the additional skills that will help you fix it. This is consultancy as a rational problem-solving enterprise, the flawed practice of which is lampooned in the Dilbert cartoon and critiqued by R.L. Stirrat in the paper referred to in my earlier comment. It’s the formal relationship between client and consultant, spelt out in the blueprint of written TOR and the legal document that binds them together. And as so many of the above observations make clear, it’s a relationship that can frustrate on both sides, and at worst dissolve into mistrust and mutual (albeit often unspoken) recrimination.

    The alternative is to think of consultancy in more personal and participatory terms, as a collaboration in which both parties hold common assumptions and are working towards shared understandings of the issue at hand. This is consultancy with a human face, in which time is indeed taken to refine the TOR together, when the consultant frequently works for more days than s/he is contractually remunerated for, when completion of the contract doesn’t mean the end of the relationship, and a job well done often leads to another. It’s also the kind of consultancy in which an employer is prepared to invest in relatively untried talent, and willing to spend time together with the consultant(s), when both (all) recognise the need for support (likewise if secondment is an option).

    This is the way I like to approach consultancy, and it’s the way that worked best for me, often working for the same friends again and again, enjoying the challenges they set me (if not their deadlines!), and knowing that they sometimes scraped funds together for jobs that wouldn’t have happened if I’d always charged full commercial rates. I was paid, of course, but the only real wealth I came away with was the kind of experience that you can’t buy. This may seem like an idealisation, but something of the same is clearly present in many of the responses to Duncan’s question. Building consultancy relationships like this takes some effort on both sides, as well as a degree of trust that transcends the purely transactional. Competitive bidding processes don’t always encourage this attitude, but they needn’t leave us with closed minds that treat consultancy as just another cog in the machine.

    It’s no accident that many of our recommendations for Oxfam staff involve taking extra time to make consultancy relationships work. That includes taking time to develop clear TOR and to identify the right consultants. As an organisation that works in many countries around the world it’s in our interest to help develop the capacity of all of our partners to undertake and/or commission effective research for development. That includes helping to develop the capacity of southern researchers and consultants, beginning by working effectively with them.

    Btw our Global Research Team talked about a lot more than consultancy in last week’s workshop, but those are perhaps subjects for another day…

  13. There is lot of consultant bashing in your post. Let me react as a consultant who had undertaken evaluations in the past for both Oxfam International and a few of your affiliates.

    The problem as it sounds to me is not so much with your external consultants recruited but your internal consultants who could adopt many of the recommendations your post highlighted.

    Lets take the for example Oxfam’s ‘Growing Better Future’ Report published last year. Oxfam’s claimed that rice yields decline for every 1°C rise in temp is amazingly based on a single study in the Philippines. A cursory reading of this study and scrutiny of the cited source suggests the following:

    – Leave alone representative of global trends, it turns out that the study is not even representative of the Philippines rice production itself! Data is from a single 100 sq metres experimental plot maintained by the International Rice Research Institute (IRRI) and not from the farmer’s fields.

    – Irrigated farms started using high input inorganic techniques since the 70s. What the data shows is that these farms 40 years later have not gone bust as Oxfam’s CEO claims but actually still increasing in yields though productivity rates are now plateauing.

    – More importantly, the scientific understanding of minimum temperatures (Tmin) in relation to agricultural productivity is at present too rudimentary. Besides plants respond to the impact of Tmin and Tmax (maximum temperature) collectively and not to their individual impacts separately. And yet, this study hypes the Tmin factor.

    – The time line of the study is 1979-2003 – cherry picked to coincide with the last global warming cycle. Since 2002 global temperatures have been trending flat and since 2010, found marginally declining.

    – If Tmin reduces yields, how does Oxfam all time record rice harvest in India, two years in a row? Last year, Andhra farmers dumped their rice into the Krishna River to protest low prices due to the rice glut in India. This year, preliminary estimates show rice production increased 3.5%. The final estimates should be higher.

    Barbara Stocking, Oxfam’s chief executive claimed “The food system is pretty well bust.” 2011-12 both global and Indian agricultural will record all time bumper harvests!

    This kind of cherry picking of data and outright spin by internal research consultants of Oxfam suggests that either they are guilty of shoddy research or devoid of conscience. Within this context, this is to request you to please do not paint all your consultants with the same brush.

    As for Kate Raworth, I did a critique of her paper “Introducing ‘The Doughnut’ of social and planetary boundaries for development”. She acknowledged my letter but since a month has not got back to me!

  14. John Magrath

    I’m sorry this comment is so late but I’ve only just seen Rajhan’s contribution, and I can’t let one of his assertions go without correcting it – especially as he accuses Oxfam of “cherry picking and outright spin”. He repeats a tired canard of climate change denialists that “since 2002 global temperatures have been trending flat and since 2010, marginally declining”. He also talks about “warming cycles”. The reality is that temperatures have continued to trend upwards, like a flight of steps (so some years or combinations of years can be flat but the trend remains upwards). Furthermore, the latest assessment of measurements from the UK Met Office, using more data from the Arctic, is clear – between 1998 and 2010 temperatures rose by 0.11C, 0.04C more than previously estimated. Moreover, 2010 was actually the world’s warmest year on record, followed by (in order) 2005, 1998,2003, 2006, 2009, 2002, 2007, 2004 and 2001….in fact the 10 warmest years on record have all occurred in the last 14. See http://www.metoffice.gov.uk/news/releases/archive/2012/hadcrut-updates

  15. I have raised a number of issues in my comment, you found objections to only global temperatures. So I think I still made my point.

    About global temperatures, it depends which dataset you choose to depend upon.

    There are two kinds of data, namely surface measurements and satellite measurements and only the former claim a warming ie satellite data shows a flat trend, with a marginal decline. I use the AMSU-NASA dataset.

    It is widely accepted that data of surface measurements are widely unreliable. The Climategate hacked files establish this as a fact.
    refer http://wattsupwiththat.com/2012/02/07/how-reliable-are-global-temperature-anomalies/

    Secondly, even otherwise, examination of raw data do not throw up a warming trend. Sceptics have analysed raw data of station after station in various parts of the world and compared it to that of Hadcrut data to show huge variance. There is only a warming trend if a “fudge factor” is factored to the raw data.

    Such manipulation we see again in Hadcrut’s claim of the warmest years. In HadCRUT3 the warmest year was 1998 with a temperature anomaly of 0.52 dec C, followed by 2010 with 0.50 and 2005 with 0.47. The revised this dataset in their recent vesion to claim 2010 as the warmest.

    Even so, HadCRUT4 actually shows is that 2010 ties with 2005 for the hottest year and a trivial 0.01 deg C above 1998. Given the errors (0.1 deg C or ten times the difference being quoted) the only scientifically respectable way to describe the warmest years would be to say that 1998, 2005 and 2010 all tied, but that would have perhaps been a little to inconvenient. [0.1 deg C is statistically insignificant] So much of your claim “the reality is that temperatures have continued to trend upwards, like a flight of steps “.

    The credibility of CRU East Anglia has been irreversibly impaired after Climategate. By revising versions, contradicting their early versions, to show increasing warming, makes the public even more sceptical of their data. The partner, Met Office is a joke. 3 winters in a row, they claim a “mild winter” only to prove wrong. So they gave up seasonal forecasting altogether that is why 2011-12 winter they did not give a seasonal forecast.

    Now your Met Office forecasted an above monsoon for India. They are on the way to be proved wrong again.

Leave a Reply

Your e-mail address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.