With the continual increase in the volume of medical literature being produced health care providers are finding it increasingly difficult to keep up to date with the latest evidence. For example a search for “ACE inhibitors” in PubMed produced 1588 possible articles in 2009 alone. Traditionally, people referred to review articles in the hope that these will provide the latest evidence and hence reduce the need for them to perform exhaustive personal appraisal of the available literature. However, review articles are susceptible to bias as they (potentially) convey only the authors’ views on the topic; furthermore, unless authors perform a comprehensive literature review, the article may not include contemporaneous, practice changing papers. Systemic reviews aim to reduce bias in review articles by providing readers with the best available information on the topic and, with meta-analysis, try to combine data from several studies to produce a single result. In this chapter we will describe how a systematic review is produced and how to critically appraise a systematic review. The systematic approach is also useful when trying to answer day-to-day clinical questions in your own clinical practice. Detailed guidance on producing a systematic review can be found at the Cochrane Collaboration’s Website where their handbook can be downloaded. A checklist to ensure all important information is reported in a systematic review was produced by the QUOROM collaboration and provides a useful tool for guiding the appraisal of systematic reviews; a modified version is presented in Table 72-1.

What is the clinical question that needs to be answered? A careful articulation of the question is critical, as it provides the scope of the review. Thus, a systematic review on insulin treatment for type 2 diabetes will be a far greater undertaking than a systematic review of the best injection sites for insulin. When reading a systematic review, one must always ascertain that the reviewers are answering the question originally asked.

The completeness of the search strategy will determine the completeness of review, although the more exhaustive the search the greater the effort will be of producing the systematic review. Usually the search for studies takes place in several areas:

• Electronic databases.There are a wide range of searchable electronic databases, often specific to a certain area of health care (ie, CINAHL for nursing and allied health care studies) but for medicine the common databases are Medline/PubMed, Embase, and the Cochrane Library. These databases display studies that meet the provided search strategy. Usually the search strategy uses Boolean language (AND, NOT, OR). It must be decided what databases are to be searched (this is often specific to the topic), the search strategy (the more general the search the more complete it will be, but the longer it will take as it will produce more results that need careful review) and the number of people who will perform the search (one person may make mistakes; therefore, it is better if the search is duplicated and the success of duplicate searching measured statistically, using correlation statistics). The search will produce a large quantity of results, many of which will not be relevant. For efficiency, many of these early results can be excluded on the basis of their title and abstract. However, more detailed review of individual papers is required for those papers passing the initial screen.

It is best to establish, a priori, criteria for accepting papers. These criteria should be explicit and the most rigorous reviews record the specific reasons for including or excluding all papers identified in the literature search. Specific recording for each paper not only reduces the risk of bias, it also allows rapid reassessment should the rationale for exclusion of one or more papers be called into question.

When appraising a review you need to determine how complete the database search is. Are all relevant databases searched? Might the condition have more than one name or spelling (hemolytic anemia in the US/Canada and haemolytic anaemia in the UK)? Could the search strategy have excluded relevant papers by being too specific (eg, when determining the effect of beta-blockers on hypertension a search strategy might be “hypertension AND beta-blockers,” but this may miss a paper which primarily looked at survival but reported hypertension in the text).

Consultation with a skilled librarian will almost always improve the quality and yield of a review. It should be strongly considered in all cases, particularly if the investigator is less experienced in systematic reviews.

• Conference abstracts. Searching conference abstracts may provide studies that have yet to be published in full. Hence it is useful to search the last few years for relevant conferences. Studies that demonstrate inconclusive results for an intervention are less likely to be published, but are still valuable evidence, and searching conference abstracts may find these. Care has to be taken when applying the results of conference abstracts to a systematic review, as the abstracts will not have not undergone the rigorous peer review of journal articles.
  
• Handsearching. As a final check, the reference lists of included studies should be checked for papers not found by other means. One can also search journals in which papers on the subject of the review are likely to be published.
  
• Contacting researchers. Writing to researchers active in the area may provide results of studies yet to be presented or published; however, care has to be taken with this information, as with conference abstracts, it has not undergone a rigorous peer review process. Furthermore, most investigators will be hesitant to provide unpublished information as its inclusion in a systematic review may hamper subsequent publication. Perhaps the greatest utility of inquiring with researchers is gaining knowledge of papers that are about to be published’delaying the review article will allow inclusion of these articles and thus make the review more timely.