Some interesting developments in the debate about measuring impact, in particular around Social Return On Investment (SROI) – and some lessons from microfinance.
A new report by the Third Sector Research Centre, comments that SROI “leaves ample room for judgement”, that it is not a good basis for comparing different organisations and that it “provides a weak basis for understanding how and why impacts occur”. Oh dear.
It’s not all bad news. Properly used, the method can help organisations map and value the effect they have on different people. But it doesn’t provide a reliable way of comparing performance between organisations. It’s also pricey.
Pretty much the same concerns are raised about impact evaluations in international aid. Critics say that randomised control trials don’t explain how change happens; and it happens differently in different contexts; so these trials don’t provide a guide for planning work in the future. What worked in Malawi last year may not work in Madagascar next year.
Critics also point out that both control trials and SROI can easily sideline the views of the most poor and marginalised people.
We shouldn’t be too quick to dismiss SROI. It’s a serious effort to find a better way of measuring results and allocating resources. I hope it continues to develop and evolve. But it runs into some deep seated problems about impact.
Microfinance has faced the same issues. The sector came up with a great way to measure impact, developed by Progress out of Poverty. Their easy-to-use and reliable scorecards help measure whether microfinance clients are poor and if they stay poor. It’s an impressive approach, that deserves to be widely known. But, as Progress out of Poverty comments, “poverty scoring does not indicate what caused change”.
Something more is needed to provide helpful, direct data that managers can use to contribute better to social change – what Lawrence Haddad, Director of the Institute of Development Studies, calls the “post-impact agenda”.
What has microfinance come up with? Feedback. You might have heard it somewhere before!
‘Social performance management‘ includes structured feedback from clients, among other measures. It’s not the only tool, but it is central. Simple, cheap to collect and powerful, it provides managers with the information they need to improve their services. It also provides summary data that policy makers can use to review performance.
It’s a fascinating example of how sector-wide collaboration can generate practical management tools to strengthen performance across the field. Here’s hoping that similar efforts drive the “post-impact agenda” elsewhere.
Filed under: Accountability, Evaluation, Management, Monitoring, NGO, Performance | Tagged: feedback, impact evaluation, microfinance, Progress out of poverty, randomised control trials, Social Return On Investment, SROI |