A re-post from the Social Impact Analysts Association. The excellent and experienced Caroline Fiennes from Giving Evidence describes simple ways for funders to measure their performance. The ideas are widely applicable to NGOs.
The approach is based on exactly the same principle as this site: focus on your performance, not their impact. I also loved some of Caroline’s other blogs, like Most Charities Shouldn’t Evaluate Their Work: Part One Why not?
Some skiers are better than others. Some singers are better than others. The same for teaching, nursing and curling. So it seems reasonable to suppose that some people are better at supporting charities than others.
But how do you tell? Curlers can easily see if they beat their opponents, and surgeons see if patients live or die, but success in philanthropy is less evident. Whereas businesses get feedback immediately and constantly – unpopular or over-priced products don’t sell – donors don’t. They can’t rely on charities telling them since they’re daren’t bite the hand that feeds them. Steve Jobs cited the difficulty of knowing if you’re giving well or badly as deterring him from giving much at all.
Happily, it is possible – and not terribly hard. Giving Evidence, a consultancy and campaign which helps donors to give well by using sound evidence, has found various tools which help almost any donor to understand their performance. They’re collated in a new white paper, and are simple: they may even seem rather obvious, but have proven useful to individuals, companies, and foundations who give. They are:
1. Monitoring the ‘success rate’: the proportion of your gifts which do well, do alright and which fail. Though clearly the definition of success varies between grants, presumably each one is made with some purpose: this tool simply asks how many succeed in their own terms. It’s unashamedly a basic measure, but then it’s hard to argue that a funder is succeeding if barely any of its grants succeed. We’re not saying that every grant should succeed: many funders sensibly support experimental or exploratory work, but, like venture capitalists, donors should expect some failures, though should have some system for noticing which grants those are to enable learning from the patterns. The Shell Foundation (attached to the energy company) used this measure to triple its success.
2. Tracking whether ‘the patient is getting better’: whether biodiversity is increasing around the lake, or whether malaria is becoming less prevalent. This of course indicates nothing about why anything is changing nor the donor’s contribution. Nonetheless, it’s imperative to know if the problem is worsening – in which case, we might re-double our efforts or invite other funders in – or if it’s gone away. Often data from public or commercial sources shows progress on a funder’s goals.
3. Measure the costs created for charities (and others) by the funder’s application and reporting processes. These can be huge: as a charity CEO myself, I had some grants where the donor’s processes consumed 90% of the grant given. It can be even worse: a physicist at Colombia University calculates that some grants leave his lab worse off, and we’ve heard stories of application processes which cost twice the amount eventually given. Grantees may make great progress despite a meddlesome funder. The avoidable costs from application and reporting processes in the UK alone are estimated at about £400m every single year. BBC Children in Need has examined its process and found ways to make savings, and other large donors can too.
4. Hearing what your grantees think. When I ran a charity, I often saw ways that donors could be more helpful but never told them because the stakes are too high: charities can’t risk offending people whose help they may need in future. So the learning is lost. Yet listening to grantees and beneficiaries has brought great benefits in medicine and social services – and to many philanthropic donors.
5. Clarify what you’re learning, and tell others. Funders do publish, but mainly about their successes. ‘Publication bias’ in medicine in which positive stories are disproportionately likely to be shared means that ‘the true effects of loads of prescribed medicines are essentially unknown’, according to epidemiologist Dr Ben Goldacre. Philanthropy can avoid the same fate. We’re currently working with a foundation to clarify and publish the ‘whole truth’ about how an innovative programme fared. Tales of failure and challenges, however inglorious, teach us a great deal.
Perhaps ‘measuring impact’ is too hard and too off-putting, and we should all instead talk about ‘understanding performance’. The tools in this white paper help with that. Giving Evidence is working with donors on several of them, and will happily talk to anybody about them.