A follow up post on the “wonkwar” between 3ie and the Big Push Forward about measuring the results of aid.
In London, Chris Whitty (DFID) and Howard White (3ie) seemed to be saying “if you want to influence policy makers, then randomised control trials (RCTs) – and other forms of evidence – tend to be useful”.
Bob Picciotto gave a masterful critique of the limitations of RCTs on this blog. But they remain influential with policy makers. So, at least for the moment, Whitty & White’s approach seems very practical. We certainly want policy makers to consider the best evidence available, among other factors.
They weren’t saying that RCTs are the answer to all the problems of measurement and accountability in the sector. (Anyone would be mad to think they are!) And we should continue to try to find better ways of providing policy makers with the materials they need to make good policy.
I was only at the Big Push Forward conference briefly, so may not have got the full picture.
They’re rightly concerned about who sets the agenda for measuring social change; and the effect that aid agencies’ measurement has on the people involved. They’re also keen to preserve space for participatory measurement and responsive programmes.
Well that all seems eminently sensible too. And not necessarily contradictory to 3ie. I’m not yet clear what approaches the Big Push Forward is promoting. Maybe they aim to open up the debate about what gets measured and why, before moving to conclusions.
Peace not war?
I was struck by how the conflict between these groups – and all the jargon it involves – feels a little stale.
Maybe we’re moving on now. The two groups are asking different questions (allbeit related), and there’s clearly no single answer to all the problems of accountability and measurement in aid.
At Plan, for my day job, we don’t really need more critiques of the shortcomings of current approaches to reporting performance. A lot of them seem pretty well documented already (e.g. logical frameworks).
We need practical approaches we can use to strengthen our accountability upwards (to senior managers and donors) and downwards (to partners and the people we aim to help). We need more data on our actual performance and we need to encourage a strong participatory approach.
Ideally, given how over-worked staff are, we need one approach that actively meets both requirements.
So, at the Big Push Forward conference, I was delighted by the launch of Who Counts? – a new book about the power of participatory statistics. The approach seems to have real potential as the basis for feedback systems, so we systematically hear what our ‘beneficiaries’ think of our performance.
Outcome Mapping also offers practical tools that organisations can use. What else can we offer senior bureacrats to help them run systems, and fund projects, that work better for all involved?