Blog: When co-producing Evaluation, what really matters? - Co-production Network for Wales
Mike Corcoran is a long-term associate of the Co-production Network for Wales, supporting their work in evaluation and leading the development of the Measuring What Matters tool. Here he explores the importance of evaluating co-production.
As co-producers, evaluation is critical to much of what we do. Without it, we can't fully understand the changes and effects our work is having, or the ways in which those changes and effects are brought about. We can't make a compelling case to our funders to support our endeavours, or to our policy makers to change the way our public services are designed and delivered. And without it, we can't learn, grow, and support more people, better, more of the time.
In spite of this importance, we also know that many of us struggle when it comes to evaluation. It can be something of an afterthought, sitting awkwardly and clumsily at the end of a carefully curated project. Those tasked with carrying out evaluations are often inexperienced and overstretched, and all too often an evaluation strategy consists of no more than asking 'what did we do last time?'
In many ways, when undertaking evaluation, we should think of ourselves as experimenters: finding appropriate ways to systematically investigate the variables which interest us, and the way in which they impact upon the data points from which our analysis will be derived. But unlike most experimenters, as co-producers, things are a little different for us. Our data points are people! And the variables which interest us are the relationships, systems and services which affect those people's lives, their wellbeing and their happiness! To forget that, even for one moment, is something we all do at our peril.
Co-productive evaluation is evaluation where power and responsibility are shared with all stakeholders. Working in this way means we cannot forget what, and who, we are dealing with. It also means embracing the unknown, adding warm fuzzy stories to cold hard stats, and accepting that things will get messy. Of course, not every person will want to co-produce every part of your evaluation, and nor should they – in fact, giving people the power to 'opt out' is itself the co-productive thing to do. But in plenty of places, in our experience, those people you work with will jump at the chance to evaluate with you. In doing so, your evaluation will make a seismic shift from a tangential exercise at the periphery of your work, to a fundamental part of it, empowering those you work with, building your relationship with them, their trust and efficacy, and activity and consciously impacting upon the results you are evaluating!
An experimental purist may squirm at the thought of an experiment that actively interferes with its own results, but as we have established, we are not normal experimenters. And 'interfering with our results' does not mean conceding an iota of their reliability, quality or validity. A wealth of emergent methods, for example Most Significant Change, which we use in our own evaluations, simultaneously permit for the co-productive gathering of rich, qualitative and messy stories, and the organising of the data they produce in ways which stand up to scrutiny.
Born out of a desire to make decisions about evaluation easier, this year we created our own thinking tool, Measuring What Matters, which aims to help you decide when you should be co-producing your evaluation, how you should be doing it, and connecting you with reliable and respected published guidance to support you throughout the whole process.
It starts with a simple question, 'what matters most to the people your activity supports?' If you're to remember one thing about co-productive evaluation, it should be that.
The Co-production Network for Wales is an inclusive, member-led network of doers and thinkers who inspire and empower one another. Find out more here.
