Real Transparency: IFAD’s Independent Evaluation Ratings Database

Repost from Rick Davies on Monitoring and Evaluation News. I agree, this is news!

IFAD follows ICAI by publishing summary ratings of evaluations of all its programmes. Oxfam went some way down the same route with its Project Effectiveness Reviews. This is a great trend towards real transparency. Let’s hope other agencies will follow suit. Continue reading

Hats off to Oxfam. But are they asking the right question?

Oxfam GB has just published their first project effectiveness reviews.

Impressively, they’re available on line, telling an unvarnished story of what Oxfam achieved in 26 projects, along with the problems they’ve faced. This is great transparency. It’s also great for other NGOs to learn from their experience. So hats off to Oxfam! Continue reading

Misusing randomised control trials

Building schools in Burkina Faso

I just reviewed two impact evaluations of education projects in West Africa. Both were required by the same major donor. Both were carried out by the same high-end US consultancy. Both used what they call the ‘gold standard’ of randomised control trials (RCTs).

Both seem to have been a shocking waste of time and money – because they used the wrong tools for the job.  Continue reading

Tools not indicators

How to measure a man?

I’m losing count of the number of times I’ve met people who aim to change how NGOs manage their work by ‘defining the right indicators’.

It’s a powerful line of thought. First, identify indicators that define what you want to achieve – like changes in average incomes or exam results. Then have all your programmes use the same menu of indicators to establish their objectives and measure their performance. Finally, compare performance between programmes to see which ones work best; do more of the good ones and less of the bad ones. Continue reading

Advice for the new aid watchdog

A few weeks ago, the Independent Commission for Aid Impact (ICAI) launched an open consultation “to understand which areas of UK overseas aid … the Commission should report on in its first three years”. Continue reading

Getting beyond impact

Some interesting developments in the debate about measuring impact, in particular around Social Return On Investment (SROI) – and some lessons from microfinance.

A new report by the Third Sector Research Centre, comments that SROI “leaves ample room for judgement”, that it is not a good basis for comparing different organisations and that it “provides a weak basis for understanding how and why impacts occur”. Oh dear. Continue reading