Wonks make peace not war?

keep-calm-and-make-peace-not-war-11A follow up post on the “wonkwar” between 3ie and the Big Push Forward about measuring the results of aid.

Duncan Green said it was the most read ever debate on his blog. As ever, his summary was balanced and readable – great attributes in this war of words!

Last week, I went to 3ie’s public lecture in London on evidence-based development (watch it on video). And today I popped in to the Big Push Forward conference in Brighton.

3ie

In London, Chris Whitty (DFID) and Howard White (3ie) seemed to be saying “if you want to influence policy makers, then randomised control trials (RCTs) – and other forms of evidence – tend to be useful”.

Bob Picciotto gave a masterful critique of the limitations of RCTs on this blog. But they remain influential with policy makers. So, at least for the moment, Whitty & White’s approach seems very practical. We certainly want policy makers to consider the best evidence available, among other factors.

They weren’t saying that RCTs are the answer to all the problems of measurement and accountability in the sector. (Anyone would be mad to think they are!) And we should continue to try to find better ways of providing policy makers with the materials they need to make good policy.

Pushing Forwards

I was only at the Big Push Forward conference briefly, so may not have got the full picture.

They’re rightly concerned about who sets the agenda for measuring social change; and the effect that aid agencies’ measurement has on the people involved. They’re also keen to preserve space for participatory measurement and responsive programmes.

Well that all seems eminently sensible too. And not necessarily contradictory to 3ie. I’m not yet clear what approaches the Big Push Forward is promoting. Maybe they aim to open up the debate about what gets measured and why, before moving to conclusions.

Peace not war?

I was struck by how the conflict between these groups – and all the jargon it involves – feels a little stale.

Maybe we’re moving on now. The two groups are asking different questions (allbeit related), and there’s clearly no single answer to all the problems of accountability and measurement in aid.

At Plan, for my day job, we don’t really need more critiques of the shortcomings of current approaches to reporting performance. A lot of them seem pretty well documented already (e.g. logical frameworks).

We need practical approaches we can use to strengthen our accountability upwards (to senior managers and donors) and downwards (to partners and the people we aim to help). We need more data on our actual performance and we need to encourage a strong participatory approach.

Ideally, given how over-worked staff are, we need one approach that actively meets both requirements.

So, at the Big Push Forward conference, I was delighted by the launch of Who Counts? – a new book about the power of participatory statistics. The approach seems to have real potential as the basis for feedback systems, so we systematically hear what our ‘beneficiaries’ think of our performance.

Outcome Mapping also offers practical tools that organisations can use. What else can we offer senior bureacrats to help them run systems, and fund projects, that work better for all involved?

6 Responses

  1. Hi Alex, I just wanted to point out that the wonkwars were between DfID and the Big Push Forward, something that I think was pretty clear from the various blogs on Duncan’s website.

    Glad you could make it to the conference drinks and sorry you were not able to attend any of the actual discussions. As you will see when you look at the BPF website, the BPF is not advocating any particular approach. The conference was explicitly not about methods, as there are plenty of (sometimes quite stale) discussions about a range of tools and their pros/cons. Rather we are encouraging more explicit discussion of the values, principles, and world views that underpin our choices – .i.e. ‘the politics of evidence’.

    Irene

  2. I think the heart of the remaining “war” is around your statement: We need practical approaches we can use to strengthen our accountability upwards (to senior managers and donors) and downwards (to partners and the people we aim to help). We need more data on our actual performance and we need to encourage a strong participatory approach.

    Much of the hype/concern over the RCTs is the extent to which they seem to privilege the upwards accountability over the downwards accountability. To the extent that we can all make a stronger push for more data, and ensure that this values data useful for downward accountability as highly as it does data useful for upward accountability , I think we can have a lasting peace.

  3. Hi Alex. Re: > What else can we offer senior bureaucrats to help them run systems, and fund projects, that work better for all involved?

    Would be interested to hear your views on ‘People First Impact Method’ (http://www.p-fim.org/) ?

    cheers

  4. Hi Cynan,

    Thanks for prompting me to get in touch with Paul O’Hagan again, an old friend who’s one of the people behin the People First Impact Method. He mentioned that the most up to date material on their work is here on LinkedIn: http://www.linkedin.com/pub/people-first-impact-method-p-fim/53/339/841

    I enjoyed reading his recently published account of putting the method into action in Northern Kenya: http://www.disasterriskreduction.net/fileadmin/user_upload/drought/docs/REGLAP%20Newsletter%20Edition%203%2022%20Feb%202013.pdf

    It sounds like a terrific participatory method for hearing directly from local communities about the priority issues they face, and how much they value different agencies’ support. It could have a wide application and generate a lot of very valuable data for programme managers and their partners.

    Paul emphasises that the method depends on training appropriate facilitators. This feels like a common constraint in taking this kind of approach to scale. Any solutions to this challenge?

    • Hi Alex, yes I think there would be challenges in getting it to scale. But on the other side of the ledger, I think one of the things I really like about it is that because it is centred on the community it can/must be a multiagency approach. Joint Needs Assessments are more and more becoming the norm on the hum response side of affairs, and while not really the same this would be in the same direction of travel. So there’s an inherent multiplier there towards scalability there if any PFIM assessment is co-contributed and used widely.

      But otherwise, I feel the constraints are really around whether the tool can get on the track to wide acceptance (and REGLAP is certainly a good start. Myself I was introduced to it by Gerry McCarthy in Nairobi last year); the resource constraints to doing so are pretty small in the grand scheme of things.

      cheerio
      c

    • Scale is not a problem and never a difficulty getting committed suitable local people who are sons and daughters of an area to do a good professional job. Experience 2010-2013: 573 national front line staff from 224 organisations have been trained in P-FiM and engaged with 5,316 disaster affected people in multiple inter-agency exercises as part of major evaluation and assessment exercises convened by the European Commission, National Drought Management Authority (Kenya), FAO, Action of Churches Together (ACT Alliance), UNHCR, War Child Canada, War Child Holland, Norwegian Church Aid, Trocaire, Children in Crossfire, UNICEF, CARE – Save the Children – Emergency Capacity Building Project (ECBII) and Global Communities (formerly CHF) in Kenya, South Sudan, Haiti, Sudan (Darfur), Burundi, Gambia and Liberia.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: