Feedback, monitoring and learning are central to the practice of quality improvement. Evaluation helps us understand whether our project has been a success. An evaluation has to be specifically designed to address the questions being asked and the nature of the intervention being evaluated.

Besides demonstrating that an intervention has been a success – or, equally importantly, that an intervention did not achieve what was planned – many other things can be learned. Asking the question “Does it work?” can lead to more probing questions, such as if it is working now, will it continue to work in future? What made it work? and so on.

By addressing these wider questions, an evaluation can help make informed decisions about whether an intervention has made beneficial changes in the most effective way. Evaluation captures insights that might otherwise be lost over time and generates new knowledge, so others can benefit from lessons learned. Communicated in the right way, this can help steer the development of new policies and new ways of working.

However, evaluation that is done inadequately, or not done at all, can render an intervention at best a wasted effort. At worst, evaluation can lack credibility, especially if there is a bias towards emphasising success and ignoring failure, which can undermine efforts to improve services.

Types of evaluations

There are two key basic distinctions in the evaluation process:

  • Summative evaluations examine the effects or outcomes and try to determine the overall impact of the intervention as well as relative costs. A summative evaluation can be seen as a ‘summing up’ of the overall effect of the intervention.
  • Formative evaluations are designed to help form or shape an intervention. They are used as the project develops and enable feedback to be gathered by looking at the delivery of the project, understanding of organisational context, staffing procedures and other inputs.

In reality, any evaluation is likely to have both summative and formative elements, to address whether something works and understand why it produces specific results for future iterations

These three short videos offer a useful summary of the benefits of planning and running an evaluation for a Quality Improvement (QI) project with tips on useful tools and approaches.

What is evaluation and why is evaluation important for quality improvement?

What should I consider when planning an evaluation?

Evaluation tools and approaches

Resources

The Centre for Effective Services in partnership with the HSE (Ireland) National Quality Improvement Team have an excellent guide to planning an evaluation, this includes templates, planning tools and examples of how to conduct self-evaluation in healthcare and other settings. The workbook allows you to fill in your specific aims for the evaluation, shows you how to develop a Theory of Change, and provides you with an evaluation project plan timetable. Learn more and download the toolkit.

If you are commissioning an evaluation from an evaluation partner, Better Evaluation have a guide to managing evaluations that you may find useful.