Researchers

Ray Pawson, University of Leeds
Research fellowship from 1 March 2003 - 31
November 2003

Annette Boaz, ESRC Centre for Evidence-based Policy and Practice, Queen Mary University of London

Background to the research question

Systematic reviews of available evidence have become the instrument of choice in evidence based policy. However, methods of research synthesis remain in their infancy. Two limitations are of particular concern:

• the restricted set of evidence addressed
• the restricted set of policy questions addressed

Aims and Objectives

• To demonstrate that the methods underpinning theory-driven evaluation (theories-of-change and realist evaluation) can be modified as a strategy for the systematic review of existing evidence;
• To investigate ways of involving the commissioners and users of systematic reviews more actively in the review process in order to improve subsequent usage of the end product.

Methodological aspects

Systematic reviews are an ideal test bed for some of the most contentious issues in social research and the project aims to make a contribution here:

• does social science cumulate
• how can we assess the quality of evidence
• to what extent can policy be evidence-based
• what are the limits of user-involvement in research

Research Design

The methodological claims of the project will be tested out by producing two demonstration reviews on 'mentoring' programmes. Mentoring initiatives feature in current interventions in education, criminal justice, worklessness, etc.

1. Review one will focus on trying to discern what it is about mentoring that works for whom and in what circumstances and what respects.
2. Review two will focus on the implementation, management, refinement of programmes.

Several reviews of the efficacy of such programmes exist. The purpose of replication is to discover whether methodological innovation can bring added value to the existing work.

Outputs

In addition to all the conventional channels, output to users will be a feature of this work. We have the involvement of a group of 'critical friends' drawn from the National Mentoring Network and several government departments.