“You can’t measure what we do”
I can’t remember how many times people have said this to me when I’ve been talking about measuring social impact. It’s a common misconception that measuring your impact will be reduced to something similar to counting a box of widgets.
What I think you can achieve is the production of a great narrative about what you do and what changes you make in people’s lives. You get to tell your story and the stories of the people you support and you measure the things that are most important to you and to them.
Getting started
You do need to start with some numbers – so people understand the extent of the changes you make. Telling one amazing story is not the whole picture, so it’s important to record how many people you have helped, how many times they have attended and how long they attended, and so on. But you can also capture things people say about your services and what difference they have made to people….
“I could never have come here on the bus when I first started.”
“I used to spend all day in the house on my own and wouldn’t speak to anyone. I have made new friends here and it’s good to have a reason to go out.”
“I feel more confident and volunteering had made me feel valued.”
How many times do people say these things to you? Do you keep a record of these comments, either in formal feedback or informal chats? If you start to collect this sort of information, regularly and systematically, you have started to measure your impact on the people you support.
You need to show what changes you have made – things that are directly attributable to what you did, and how much people value them. This doesn’t have to mean placing a monetary value on the impact, but the aim should be to demonstrate to people outside your organisation, as well as internally, what difference you make.
What methods to use
There are many tools and resources out there to help you measure your impact, some are now pretty well established, valued by the organisations that have used them and recognised by funders and commissioners. Increasingly there are IT programs that ‘interpret’ your data and spit out a report that tells you how you are doing. This might look like a great option but there can be problems, as with some uses of AI – if you don’t understand how the outcomes and impact are calculated, how can you justify them to others?
Generally speaking, people tell me they want something that is easy to use, won’t take much time and is free – and is also robust and able to reflect the complex experiences people have. This is impossible. You have to balance the choice between tools that work for your organisation, that are manageable within your resources (including time) and how complex you want the analysis to be. Lots of organisations want to design their own tools (or pay for someone to develop a bespoke tool), but in my experience this is unnecessary. It is better to customise tried and tested models than to start from scratch.
The things you really should be doing are:
- Start with a benchmark – how are people when they first arrive, or how was that bit of land before you worked on it? Collect interviews, photos or whatever type of evidence seems useful to your project
- Ideally, measurement should be on-going, throughout the life of the project, showing changes that are happening over time. If a participant suddenly leaves, you still have a record of some changes having taken place
- Keep systematic and consistent records – a simple spreadsheet is fine but always bear in mind how you will retrieve the information later. If you have lots of open questions and answers, someone is going to have to read them all individually – OK if you have 10 people but not so good if you have 100.
- Collect information in a way that isn’t intrusive – think about ways to have informal conversations with people, rather than sitting there with a clipboard and a list of questions. Are there more active ways to get answers, putting counters in a jar, or stickers on a chart?
- Tell the story – collect good case studies of the typical user experience, not the one amazing story, but the changes you make every day for lots of people
And remember, this is just the first part of the process. Once you’ve collected this information (data) you need to analyse it and work out what it is telling you and then put it in a format so you can tell others.
In the next of these blogs, I’ll be looking at the theory of change (much simpler than it sounds) and some of the tools and approaches that you could consider using and giving you my ten top tips for measuring impact.
Nicky Stevenson is a Senior Lecturer at Anglia Ruskin University. She is also a SEEE Board Member.