Scottish Co-production Network Learning Event: Evaluating Co-Production 

ESS_logo_medium_resolution.JPGThursday 12th April
Glasgow Woman’s Library, Bridgeton 

 Hugh Mackay is an Australian social scientist and academic who warned against the ‘cult of perfection’ by writing that

Nothing is perfect. Life is messy. Relationships are complex. Outcomes are uncertain. People are irrational.

In this learning event, held in partnership with Evaluation Support Scotland, we embraced the messiness of life, albeit within our responsibilities to funders, to our organisations and most importantly to the community at large, while delivering work and activities that create change.

‘Evaluate to improve, not prove co-production’

DalR6uAWAAAEhVt.jpgThe event came about following a thought provoking blog written by Steven Marwick, Director of Evaluation Support Scotland.  

Steven gave an excellent contribution to the event on how we measure the magic of working with communities without crushing the creativity.  Steven argued that we do not need to prove that a co-productive approach is a good thing, the evidence is already there. Rather we should evaluate the coproduction process, the outcomes arising during that process and those outcomes that are a result of the work as a whole.

The purpose of evaluating co-production is threefold, to improve our practice, to make sure we are working as well as we can, to understand more about what co-production is and is not, and, to celebrate what we achieve.

It’s important to celebrate achievements because we can become stuck in life’s messiness and go through difficult times – celebrating as well as learning is important to our well-being and motivation.

Steven spoke about the importance of language, with ‘evaluation’ being a barrier word to some – something that resonates with many of us promoting ‘co-production’. Terms such as reflection, review and debrief were offered.

Steven called for a collective and collaborative approach to evaluating co-production that is led by people with lived experience, or recipients of services rather than ‘professional experts, concluding that we should involving people in evaluation efforts that are creative and focus more on stories than numbers.  

You can read Steven’s presentation here and we commend the Evaluation Support Scotland guide Evidencing Genuine Co-production in the Third Sector

We commend Dawn Smiths’ (@dsmith_edi) reflections on Steven’s input and on the event as a whole, which you can read here.

Sharing our stories

DalkZrAX4AAikgh.jpgWhen registering to attend we asked people to consider how they are evaluating their own co-production processes and share what’s worked and where are the challenges? We were delighted that the following people agreed to share their experiences with us all. 

Lewis Hou spoke from Fun Palaces Science Ceilidh – an educational project bringing people together through science and music, breaking down science topics into their core steps which are represented by ceilidh dance.

For Lewis, evaluation is important because professional reflection helps us do our job better. When working with young people, Lewis evaluates little and often by, for example, setting aside ten minutes at the end of a two hour activity. Using the term, ‘debrief’, participants are asked to sit and think quietly for a moment before discussing what worked and what didn’t.

In the longer term, Lewis recommends the use of really well designed anonymous questionnaires that capture empowerment by asking what are you most proud of?  Did you feel your voice was heard?  Lewis suggested using Cava or other packages for design.  Follow Lewis on @fiddlebrain.

---------------------

Dall5siXkAAj2r-.jpgLiz Ellis, University of the Highlands and Island’s Centre for Health and Science shared her experiences of co-producing research with people with learning disabilities, exploring collaborative decision-making within an inclusive research project.

Liz talked through some of the challenges of co-production, around how we define co-productive processes and the impact and power of commissioners of work. A familiar challenge of naming co-production arose, with Liz telling us that some of the most beautifully true examples of co-production which she has experienced wasn’t recognised as co-production by those involved.  Conversely, others claiming to co-produce simply are not.

Liz agreed that co-producing is often messy but that we should revel in that messiness, advising us not to be put off by how daunting co-productive approaches can seem, “aim for it and try to hit it”.

You can read some of Liz’s research here and follow at @LizEllisPhD

--------------------

DalgFOmXcAAQHIF.jpgLouise Christie spoke of her experiences supporting Scottish Recovery Network’s work.  She told us that in their Making Recovery Real initiative brave decisions were taken to ‘figure it out’ as they go, working in different communities who decided to find their own routes with no fixed end points or predetermined outcomes.

What united the work was the centrality of lived experience in all that they did and a commitment that how they work matters alongside what is achieved.

The Scottish Recovery Network evaluation is variously ‘reflecting’, ‘celebrating’ and ‘planning for the future’, in order to learn about the difference made by working co-productively, to recognise and celebrate the change they achieve and to make changes for the future.  However described, their evaluation approach is appreciative and strengths based and the intention is to further empower all who take part.

Louise commend a Making Recovery Real film which you can see here.

------------

DalncPZW0AElkl-.jpgNoreen Blanluet joined us from our sister network, Co-production Network for Wales who are working to achieve the collective goal of: ‘Social justice, and the permanent shift of power from state to citizens. Nothing less’.

Noreen shared their efforts to evaluate the growth of co-productive practice in Wales by developing and using a Co-production Audit and the Most Significant Change approach, reflecting on the strengths and limitations of both approaches. Noreen argued that there is a gap for customised, co-produced tools where each individual or project decides what matters to measure. 

Calling on this gap to be considered at the beginning of a piece of work, Noreen recommended working collaboratively to uncover valid, creative and credible evidence.  

Table discussions

The overall view that emerged from table discussions was a resounding yes to the question, can we evaluate co-productively. 

  • That we should consider “What do we want to know and why?”
  • That we must be flexible “don’t be too decided on things”, “Ask people how they want to be involved and what is important to them”
  • That “Evaluation is a process and not an event. Needs to be embedded from the start and co-produced”
  • And that there is no one way to co-produce, it’s not a case of What Works but rather What Worked in a particular place and time.  This means that we should talk more about co-productive approaches. 

shaking_hands_crop.jpg

We went on to consider - What needs to change? What tools or support is needed? How can we create those?

The following emerged:

  • We should evaluate to understand more about ‘success’ – and the factors that enable a co-productive process to happen
  • Perfection is the enemy of good enough, we need to be more honest and appreciate that we learn about co-production by experiencing it – it’s not about training.
  • Varied and creative valuation tools work, as does powerful questions such as ‘appreciative enquiry’ or ‘what was the most significant change’ or ‘was your voice valued?’ – these unlock rich data
  • As with all facets of co-pro, evaluation should take place in an open, safe space where all voices and ‘truths’ are equal and recorded

Our evaluation

Steven facilitated our own evaluation offering us a choice of methods, including an ‘emotional touch point’ as a ‘spark’.  For each method Steven invited a moment’s quiet refection before recording ‘something that you would take away with you’.

These included:

  • Lived experience should always be at the centre
  • There is great value in getting interested people together to discuss working like this – nattering on can be productive work
  • There is no right way to ‘do’ co-production – also want to say that I feel energised and excited by being amongst people talking co-pro!
  • Greater recognition of the messiness/openness/indefinability of what co-pro is! This is where its strength lies, but also the challenges. Taking away lots of food for thought. Practical idea for evaluating in a co-productive and creative way.
  • Be brave and take risks with evaluation and co-pro
  • 3 key reflections: Stop putting myself under pressure regarding the time it takes to work in co-pro; Use of reflection at the end of events; evaluation is about celebrations along the way – value unexpected outcomes
  • I’m taking away re-assurance. I feel more confident about the work I do.

A great thanks to everyone to took part in the day. To learn more about this event please contact david.reilly@scdc.org.uk.