Improving Access to Robust Evidence for HIA: Final Report to the Department Of HealthImproving access to robust evidence for HIA
Policy Research Project ref no. 030 0072
Final Report to the Department of HealthFinal report to the Department of Health of research funded by the DH Policy Research Programme. This project was (a) to develop a Guide to reviewing evidence for use in health impact assessment and (b) to use the guide to appraise existing reviews and mane those of sufficient quality more widely available. Executive Summary Introduction Health Impact Assessment (HIA) assesses the positive and negative effects of a project, programme or policy on health, and on health inequalities through the distribution of those effects. Many tools, guidelines and frameworks exist to assist in the conduct of HIA. This project does not set out to replicate any of this work, and is not a guide to conducting HIA. HIA has been described as “the use of the best available evidence to assess the likely effect of a specific policy in a specific situation”. However, the current HIA methodology has been criticised for lack of rigour in collecting and analysing evidence. Three types of knowledge are combined in HIA: that provided by stakeholders, local data, and evidence from previous studies. This project considers the last of these. A number of difficulties are acknowledged in reviewing evidence for health impact assessment (HIA). These include: * the focus on complex interventions or policy proposals and their diverse effects on determinants of health; * the consequent diversity of the questions to be addressed; * the diversity of the evidence in terms of relevant disciplines, study designs, quality criteria and sources of information because of the wide range of interventions/approaches that may contribute to improving health, ie the need to search, obtain, and appraise a broad literature; * the need for, but paucity of, evidence on the reversibility of adverse factors damaging to health (most evidence being of associations between factors and adverse effects, not studies of reversing these); * the broad range of stakeholders involved; * the need to seek evidence about potential impacts on inequalities as well as on overall effects; * the need to apply health impact assessment within the realities of policymaking, planning and decision-making processes, which can often mean short timescales and limited resources generally available; * the pragmatic need to inform decision-makers regardless of the quality of the evidence. These factors have implications for the commissioning and conducting of reviews. There is a need to set standards and provide guidance to assure a quality product. Although there are many sets of guidelines and frameworks for conducting HIA, none address the issue of reviewing published evidence for use in HIA. Nor does existing guidance on conducting systematic reviews address these particular circumstances of HIA that cause difficulties with reviewing evidence, particularly the lack of time that would be required for conducting a systematic review. The London Health Observatory, in association with London University academics involved in HIA and the former Health Development Agency, therefore undertook a Department of Health funded project within the Department of Health’s Policy Research Programme to improve access to robust evidence for HIA. Aims and Objectives The key aims of the project were: 1. to improve the evidence base for HIA by: a. collating quality criteria for different types of evidence, b. developing guidelines for robust reviews of evidence, and c. piloting and evaluating the guidelines; and 2. to improve access to robust evidence for HIA by peer-review of existing reviews of evidence already undertaken and publishing quality-assured evidence reviews. The guidelines were intended to be authoritative and of practical help in commissioning, conducting or assessing reviews of published evidence for HIA. They would be developed for both brief and more comprehensive reviews of evidence in HIA. Methods A small steering group was set up to manage the project. An Advisory Group comprising HIA academics, HIA practitioners, and experts in producing systematic reviews, was formed to help develop the underlying principles; comment on draft guidelines and advise the steering group. A set of principles for reviewing evidence for HIA and for development of the guidelines were agreed. Qualitative research examined the format and presentation of the guidelines. An extensive background document drew on published literature to initiate the development of the guidelines. Iterations of subsequent revised drafts were discussed by email by the steering and advisory groups and by other HIA practitioners and academics and by discussion at workshops at conferences. The steering group amended the guidelines in response to each round of consultation. Version 3.2 incorporated both the comments on content in response to circulation of earlier drafts and also the results of the qualitative research regarding format and presentation. Version 4 was piloted and version 5 peer-reviewed by HIA practitioners and academics who had not been involved with the guidelines’ development. The final version was applied to a selection of existing reviews of evidence which had been developed for use in HIA. The objectives were to increase access to existing reviews considered of sufficient quality to be useful in other HIAs and to assess whether the guidelines would assist in appraising whether such evidence reviews would be of wider use in other HIAs. These appraisals were conducted by volunteers in the public health field not involved with the project. Results In response to qualitative research, the terminology used was changed from ‘rapid review’ to ‘brief review’; those containing more of the elements required for a systematic review were termed ‘more comprehensive’. The ‘guidelines’ were renamed Guide for reviewing published evidence for use in HIA. The ‘final’ version (6.2, Appendix H) is a 12 page, two-colour, A4 document. After two pages of introduction to the document, it uses three columns to state the minimum elements required in a ‘brief’ review; additional elements to make the review more comprehensive; and gives tips and links to useful resources. The target audience for the Guide is those commissioning, conducting or assessing reviews of published evidence for use in HIA. Whereas most such documents deal either with qualitative or quantitative types of evidence, we have developed a Guide that combines the two in a single document. We also included references to a range of other existing sources and created some additional documents - including a glossary, a review of approaches to assessing causality, and a document on quality criteria for appraising published studies - as an introduction to some of the concepts and tools required in appraising published literature. Use of the Guide requires an understanding of critical appraisal. During the project, it became obvious that, rather than additional quality criteria (most types of work are covered by existing guidance), HIA practitioners need help to find quality criteria relevant to the type of study they are evaluating and advice on how to interpret what were sometimes rather contradictory advice from sources dealing with different types of evidence. We therefore tried to distil some simplified advice and to provide a comprehensive list of gateways to alternative sets of criteria with some advice about the circumstances under which each is likely to be useful. Five existing reviews used in HIA were identified and appraised using version 6.1 of the Guide. Although each review had some flaws, four of them were considered of sufficient quality to be made more widely available through the LHO website. The final section of the report is a detailed discussion of the research project. The report concludes by outlining potential future research building on this project.
| URL/File | | | Format | Adobe PDF (.pdf)
| | Home Location | LHO | | National or Regional | National |
start of decorative frame for |  | start of content for
| Publication date | 11 Sep 2006 | | Creator(s) | Dr J Mindell | | Contributor(s) | SE Curtis, AL Boaz, L Taylor, M Joffe, J Biddulph |
end of content for |
end of decorative frame for
start of decorative frame for | | start of content for end of content for |
end of decorative frame for
start of decorative frame for | | start of content for end of content for |
end of decorative frame for
start of decorative frame for | | start of content for end of content for |
end of decorative frame for
start of decorative frame for Coverage |  | start of content for Coverage
| Start date | 13 Dec 2007 | | End date | 13 Dec 2007 | | Geographical area | International
|
end of content for Coverage |
end of decorative frame for Coverage
end of decorative frame for Type and Language
|