Principles of Pragmatic Evaluation event

How can we meet the call for more appropriate and robust evaluation? How do we generate more rigorous and relevant evidence?

These were some of the questions explored by more than 30 people with an interest in evaluation, including commissioners, researchers and the public, at the Principles of Pragmatic Evaluation workshop on Wednesday 16 December 2015 organised by the West of England Evaluation Strategy Group.

The interactive workshop explored the principles of pragmatic evaluation, and was led by Dr Peter Brindle, Leader for Commissioning Evidence Informed Care at the West of England Academic Health Science Network (AHSN) and Evaluation Lead at the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West). It was jointly organised by CLAHRC West, the West of England AHSN, Avon Primary Care Research Collaborative (APCRC) and Bristol Health Partners.

Before the event, attendees were provided with:

Workshop

On the day, delegate packs included the following additional documents:

Dr Peter Brindle set the scene of the workshop as a chance to learn, listen and discuss the opportunities and challenges for evaluation. He was clear that this was the beginning of a collaborative effort to create a shared common vision, strategy and, especially, an action plan for evaluation in the West of England with the potential for collaboration with similar initiatives across the country. He went on to provide his own reflections on why evaluation is important, the barriers to evaluation and some of solutions going forward, explaining that we need to:

“Choose the right tool for the job whether it is quality improvement, evaluation or research.”

The key note speaker for the day was Professor Jo Rycroft-Malone, Director of the NIHR Health Services and Delivery Research (HS&DR) Programme. Jo described some of the challenges, opportunities and implications of pragmatic evaluation for the programme. These included the implication for research methods, the need to agree what is ‘good enough’ evaluation, how to bridge the gap between those knowing and those doing, and what the rules of engagement are.

Download the collated slides for the event (PDF)

Lunch was followed by round table discussions on the following topics:

How do we best involve patients, carers and the public in evaluation?
How do we ensure that evaluation is of value to health and social care?
How do we build a culture of evaluation?
How do we create an ethical approach to evaluation?
How do we make the trade-off between rigor and relevance?

These table sessions inspired a broad range of discussions and reflections as well as a chance to share good practice and innovative ideas and solutions. These sessions were fed back to the room for further discussion. Themes emerging included the need for:

genuine and early co-production in evaluation
good early evaluation planning with all the right stakeholders and information
development of capacity, skills and understanding of evaluation
both rigorous and relevant evaluations

There was also interest in the researcher-in-residence model being driven by Martin Marshall at University College London, Peter Brindle through his work at the Avon Primary Care Research Collaborative and Lesley Wye at the University of Bristol. This was acknowledged as an excellent way of bridging the ‘know-do’ gap and supporting putting evidence into practice.

Outputs

The final step involved participants making a commitment to action as a result of the workshop discussions, whether something they personally intended to do or that they would take back to their organisation. A large number of actions were pledged and these will be pulled together and followed up in the coming weeks. Examples include:

I am going to “lobby for equal status of evaluation and research”

I am going to “ensure evaluation is embedded in transformation at the outset”

I am going to develop a “key guidance sheet”

Throughout the event there was a positive and energetic atmosphere in the room, and post event feedback found that the event met attendees’ expectations fully (89%) or at least partially (11%), and supported networking opportunities fully (61%) or at least partially (39%). Comments included:

“An excellent event, really well put together”

“I think it has laid some great foundations”

Dr Peter Brindle said:

“I feel that the event has helped to create an energetic and supportive community of action in the West of England, to overcome the barriers to integrating evaluation into service change and innovation.”

The organisers would like to say a big thank you to all those who participated for their excellent contributions and commitment to actions to help drive this agenda forward – Thank you!

For more information or to get involved contact Jo Bangoura, Evaluation and Commissioning Lead on j.bangoura@nhs.net.