Listening to those who matter most: two inspiring new publications

More and more people are writing about how feedback can improve the quality and accountability of aid. And we’re seeing more serious pilots about how we can use feedback systems at scale.

For example, DFID’s major piloting exercise on using feedback systems to improve aid is getting into gear. Just last week Mark Maxson blogged on a new collaboration called Feedback Labs, working on the same issues.

Two major new publications provide more inspiration to all of us. Both point to similar obstacles and ways forward. And they both lay down a challenge for everyone involved in the debate.

Who Counts? Edited by J. Holland

pub_who_counts_smallWho Counts? brings together global experience for the first time of how participatory methods can generate reliable statistics. It’s full of examples that show how:

“Local people can generate their own numbers – and the statistics that result are powerful for them and can influence policy.”

The argument is that participatory techniques can empower local people to analyse and tackle their own issues while also generating data on social issues to influence senior decision makers. Jeremy Holland describes this as a ‘win-win’ and rightly so.

The book expands on themes in Robert Chambers’s 2007 paper of the same name. He described the approach as a ‘methodological breakthrough’. The quantification is necessary to make local views and issues visible to senior decision makers at scale. It’s brilliant.

The book lays out a series of detailed examples. For instance, one chapter describes how a participatory process in all 15,000 villages in Rwanda has created a national census and data set for the entire country. The process uses large scale social maps and wealth ranking – all created, owned and updated within villages.

Local and national government have both used the resulting data to guide their decisions. The process also provides the foundation for community led action, funded through a village bank account. The case study emphasises two key factors for success: gaining high level  support, and using the right facilitators.

If powerful data can be created from local people’s perceptions for an entire country, then surely we can use the same approach to assess NGOs’ programmes.

Listening to Those Who Matter Most, F. Twersky, P. Buchanan and V. Threllfal

listening_to_beneficiaries-640x776The summary of this new paper inthe influential Stanford Social Innovation Review reads:

The views and experiences of the people who benefit from social programs are often overlooked and underappreciated, even though they are an invaluable source of insight into a program’s effectiveness.”

The paper is part statement of belief and part rallying call – which, as Keystone put it, is pretty much the state of the art of feedback at the moment.

The authors lay out the general case for listening better to beneficiaries. They list promising global beneficiary feedback initiatives, such as: CDA’s Listening Project, GlobalGiving’s Storytelling Project and Keystone Accountability.

They describe two initiatives in the US in detail, to help schools listen better to their students and hospitals listen better to their patients. Both are powerful examples of how feedback helps organisations provide better services, supported by independent evaluations.

Where do we go from here?

The book and the paper both close with discussions of why listening to beneficiaries is difficult.

The paper identifies three challenges: it’s expensive and time-consuming; it’s difficult to get (candid) responses; and it makes us (managers) uncomfortable.

The book goes further. In an Afterword, Robert Chambers makes an impassioned case that these techniques have been around for decades. But they have not been widely adopted. He identifies three blockages: they are not seen as ‘rigorous’; risk-aversion and interia within organisations; and limited skills and examples of good practice.

These insights start to chart the difficult waters we have to navigate to implement feedback systems at scale. The problems are both practical and political – particularly when feedback systems bring up conflicts between different groups of beneficiaries, managers and donors.

Chambers calls for an organisation dedicated to participatory statistics and for personal commitments from all involved in development to use them. As ever, it is an inspiring vision.

At Plan, we’re working to deepen this analysis of existing experience, in order to identify practical management implications. Then we aim to pilot approaches with the potential to help busy managers run feedback systems simply and effectively. (Or at least, more simply and more effectively.)

I hope it’ll be a useful contribution to the wider debate. We’re committed to learning from others working on the same issues. Please get in touch!

One Response

  1. Empirical evidence that local “voice” is more than just a “nice to have” part of development. This working paper from Uganda and iiG shows that when PARENTS decided what would be given in school evaluations, the beneficiary crowd (e.g. parents) was able to score schools on the things that mattered to outcomes (better student performance). The parallel arm in this RCT was education “experts” deciding how schools should be rated, and moving these ratings did not improve student performance.

    Therefore beneficiary-led evaluations are a much better approach to identifying what the parts of a problem should be addressed first with the limited resources available.

    And I built a tool that can be used for this:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: