Twaweza, one of the world’s cutting edge accountability NGOs

October 9, 2013

The war for Twaweza’s soul: the hunger for clarity and certainty v the demands of complexity

October 9, 2013

So what should Twaweza do differently? How accountability work is evolving

October 9, 2013
empty image
empty image

Yesterday I sketched out the theory of change and initial findings on the first four years of work by an extraordinary East African NGO, Twaweza. Today I’ll move on to what some NGO people (but thankfully no-one in Dar es Salaam last week) insist on calling ‘the learnings’ about the flaws and gaps in its original theory of change (described in yesterday’s post).

First, there’s a big ‘black box’ containing Twaweza’s rather large assumption that giving people information (eg about failing education systems), would lead to them taking action to change things. What issues in the black box determine whether this is true or not?

Evan Lieberman (one of Twaweza’s many evaluators, from Princeton University) called this the ’secret sauce’ – the miracle (see much-used cartoon) then a miracle happensthat links information to action. His team had come up with a smart attempt to identify some of the sauce’s ingredients – conditions for a →b:

Do I understand the info? →Is it new info? →Do I care? →Do I think that it is my responsibility to do something about it? →Do I have the skills to make a difference? →Do I have the sense of efficacy to think that my efforts will have an impact? →Are the kinds of actions I am inspired to take different from what I am already doing? →Do I believe my own individual action will have an impact? →Do I expect fellow community members to join me in taking action? Evan argued that only if the answer to all of these is yes, will the black box indeed turn information into action.

Actually it’s worse than that – they missed some pretty big ones (‘do I have the time to do this, on top of everything else?’ ‘Will I run any personal risks if I do this?’) It’s a hell of an intimidating set of conditions and, as was pointed out, the danger is that accountability proponents will just latch onto one of the steps, then wonder why nothing is popping out at the outcome end.

Second, thinking has moved on from Twaweza’s rather Manichean division of politics into the left and right hand sides of informal v formal power (see table), with change to be achieved by the left hand side coming together and demanding action from a nefarious and/or indifferent right hand side. These days, research by groups like the Africa Power and Politics Programme and Matt Andrews argues that both demand side (build the citizens) and supply side (build the state) have failed. What works, they think, is collective problem solving, bringing together citizens, states and anyone else with skin in the game, to build trust and find solutions. People on the ground, like Goreti Nakabugo, Twaweza’s Uganda coordinator, get this: ‘we know we need buy-in from the government, officials, local politicians. We are brokering relationships with them on a daily basis’.

Twaweza two worldsNot only that, but in practice, even differentiating between citizen and state can be problematic – neither category is a monolith, and in some cases, the most active citizens are themselves state employees, members of public trade unions etc.

Third, Twaweza needs to decide whether all citizen action is equally desirable and if not, what kind it wants to promote. Convince a parent that their child’s school is failing and unless they think their protests are likely to get a hearing, they are probably more likely to try and move their kid to a private school than join a social movement to ‘change the system’. Is that a desirable exercise in citizen agency or a systemically disastrous rush for the exit?

Another axis is that of individual v collective action – does Twaweza see them as equal in value or is collective action in some way preferable in terms of building the voice of marginalized groups? Again, not clear from the theory of change. Maybe Twaweza should map the results of its current work on a 2×2 diagram (individual → collective; public → private) and see if it’s happy with the result.

Fourth, envisaging change as taking place in an information ecosystem sounds great, but is it just a metaphor or a real description? According to Rakesh, Twaweza hoped to ‘bend’ the ecosystem towards becoming a driver of citizen action, but ‘to seriously achieve what we had imagined would have required such deep density and high level of coordination and sequencing it would have been akin to playing God.’ The goal is now to try to understand the ecosystem better, and find some less hubristic way of influencing it. A command and control approach was never going to work, so why not twaweza-logopull in some actual ecologists to suggest how to influence real-life ecosystems, and see what could carry across into work on accountability?

Fifth, the evaluators came back with some serious questions about irrational optimism over the magic powers of ICT. Eg broadcasting messages to the public via SMS doesn’t work because people only trust the content if they receive it from someone they know. Otherwise they just think ‘spam’ and hit the delete button.

Sixth, an overarching lesson of all this, as so often, is that the system exhibits much more inertia than we think. Things just don’t shift that quickly or easily, despite the best efforts of smart and determined campaigners. But they do (eventually) shift. And presumably some things shift more quickly than others – my guess would be that norms and attitudes (eg over whether it’s worth taking public action to improve your kid’s schooling) evolve much more slowly than public policy. So is it any wonder that the evaluations showed more impact on the latter?

Finally, it is notable that among the partners in the five channels that Twaweza identified as of mass relevance to poor people (mobile phones, media, consumer goods, teachers and religion), only Twaweza’s media partnerships have really worked out. Is that because the media and NGOs are more aligned in terms of language, interests, politics? Would Twaweza need different approaches (change of language, longer timescale) before it can work successfully in the other channels? This is very important – I was a bit disappointed to see all the fine words boil down to doing more and better media work rather than, say, forging real alliances with faith communities.

Tomorrow, I get onto an underlying tension in the Twaweza discussion – the war for Rakesh’s soul (and Twaweza’s future) between the hunger for certainty and the realities of complexity.

5 comments

  1. Great to see a really critical evaluation of projects like this…Twaweza don’t come across as really a ‘local’ NGO- they are one of those aid inspired creations which abound in cities like Dar. I agree their models of change are far too unrealistic.

    The education system in Tanzania is chronically broken- I work in it and see family members going through it. Its going to take more than a few isolated cases of parents complaining at school level to change it. When only 25% of your secondary school intake pass their final exams….is that not enough of a national disgrace to inspire fundamental change? Everybody knows that the system is broken but nobody believes that the government is capable of doing anything about (despite rhetorical talk of ‘Big Results Now’).

  2. Interesting post. I particularly like the efforts to unpack the host of ‘ingredients’ in what looks superficially like a really simple causal assumption. We don’t do enough of that in M&E work and are often so fixated on high level impact that we dont ask basic questions about output level change (as commonly defined in DFID log frames).
    – Did anyone at the meeting suggest a critical review of DFID programmes that include an assumption that access to information will play a key role in social change? It is at the heart of DFIDs empowerment and accountability TOC and there seems to be quite a bit of ‘evidence’ that it is problematic. Perhaps because it is underpinned by individualistic rational actor assumptions that tend to ignore the effects of structural power relations you draw attention to, e.g. perceived risks associated with action? LSE has been doing some work on TOCs with the Asia Foundation and one case study presented at a recent LSE gig argued that a TOC/theory based approach to evaluation had made the Asia Foundation aware of weaknesses in the info assumption in some of their governance programmes.
    – Related – it will be interesting to see whether the Nike Foundation’s Girl Hub approach to social comms which is innovative in as much as it is testing brand identity as a means to ‘impart’ information and change norms, attitudes has any different effects. Could be apples and oranges, but the assumptions around brand identity are slightly different and potentially interesting though loaded with ethical issues…
    – I’m surprised to still see reference to supply and demand binaries. Haven’t we moved on? As you say many local government officials are (powerless citizens and significantly less powerful than some of their NGO counterparts!). SIDA was supporting some interesting work in Cambodia a few years back that seemed to suggest supporting collective action by poorly paid commune officials trying to respond to demands of their constituents might be quite an effective means to challenge power structures within very politicised states.
    – BTW does NGO work on power analysis look enough at the contingency of power relations and how power relations are maintained? We seem quite good at analysing static power relations in the context of the conceptualisation of a particular problem and set of relationships, but sometimes seem to miss how contingent those power relation are and the subtle ways in which they are sustained through discourse.

  3. Thanks for sharing this interesting example which gave me some food for thought reading over breakfast.

    The assumption that’s being questioned – if you give people information then they will do something with it – is a really familiar one in social marketing or rather, that information or education or public awareness programs are almost never enough to stimulate some kind of behaviour that isn’t already taking place (or cause someone to end a behaviour such as smoking “Smoking is bad for my health? Now I know, I shall cease immediately.”).

    Readers might find some of the behaviour change and decision-making theory used by social marketers useful if they are working in a program that has access to information as its primary route to change. E.g. social marketers will ask:
    “For my target audience, do the benefits of doing what I would like them to do outweigh the costs or barriers to doing it?

    Am I using a combination of activities in order to
    encourage people to achieve the desired action?” (NSMC)

    But…information can lead to action if you are sure it is the missing piece of the puzzle, and delivered through the right medium. The NSMC’s Big Pocket Guide to Social Marketing http://www.nsmcentre.org.uk/sites/default/files/Big_pocket_guide_2011.pdf is a useful intro and includes the Truth Campaign case study – which is an example of where access to information did lead to change, however it was based on thorough understanding of what motivated young people i.e. making them aware of the tactics used by Big Tobacco to manipulate them into smoking (when transparency and honesty was very important to that target audience) and it prevented 450,000 students from taking up smoking.

    I think more research needs to take place at the planning part of information programmes; a TOC or logframe tends to have assumptions listed in it. Should we not be researching these assumptions before we start any action?

  4. Thanks for this.
    What is not referenced is how informed citizens are up against vested interests for a change not to happen. This is important for Theorys of Change when power dynamics are very unbalanced.

  5. Yo Duncan, you hit me again this time !
    Nice post. It is written in a depth reflection i guess.

    In the marketing word i often heard that you are not asking people to choose A to B. Rather you led them to choose A or B. In the social sector, i believe that we still do it sometime tough. Thus, if i may add on your last point, it is as well a competing norms or attitudes between you and the people that you currently working with. If we believe that this is a common reality than alignment of both supply and demand side actually is an alignment of norms or attitudes.

    If i could share. My experience in promoting Social Audit in Indonesia after becoming Exfam five years ago has led me to understand another important element of accuracy and timely respond. So if i may add; yes public policy is “easier” to change rather than norms or attitudes. However, it would require an accurate and timely respond public policy to avoid it decay or no longer relevance to its demands.

Leave a comment