The Quality of Evidence



Introduction





In the past, physicians passively applied their knowledge of pathophysiology and pharmacology to treat their patients. While a keen understanding of human physiology (and disease processes) is crucial, several groundbreaking epidemiologists believed it was not enough for the care of patients. Especially in this modern era of explosive growth in technology and new drugs on the market, there is a surplus of information available to health care professionals.






In 1981, a group led by Dr. David Sackett introduced the concept of critical appraisal. Critical appraisal was a term that implied an ability to systematically scrutinize medical literature and apply the findings to patients. However, it was not until 1991 when Dr. Gordon Guyatt published an article in ACP Journal Club where he coined the now ubiquitous term evidence-based medicine. Sackett defines it as the “integration of the best research evidence with clinical expertise and patient values.” Therefore, the practice of evidence-based medicine (EBM) does not blindly appraise the medical literature; nor does it absolve physicians from their duties to apply common sense and work closely with their patients to determine the best course of care. In fact, to practice EBM, physicians must adhere to two underlying principles:








  1. “Best evidence” is determined using a rigorous process of data extraction and interpretation that weights some forms of evidence over others.



  2. Evidence must be interpreted in the setting of the individual patient and his or her characteristics.







| Print

Practice Point




To practice evidence-based medicine, physicians must adhere to two underlying principles:




  1. “Best evidence” is determined using a rigorous process of data extraction and interpretation that weights some forms of evidence over others.



  2. Evidence must be interpreted in the setting of the individual patient and his or her characteristics.







Evidence-based medicine is vital to providing the best patient care. The focus of this chapter will be on evidence-based interpretation of the medical literature; however, other tenets of EBM should not be overlooked.






Basic Concepts





Assessing the quality of evidence and applying evidence-based principals requires familiarity with a number of “buzzwords” and basic concepts. This knowledge provides the foundation for a better understanding of the fundamentals of assessing the quality of medical literature.






Cointerventions



Cointerventions are treatments or interventions that may be differentially applied across experimental and/or control groups that may have an effect on the target outcome, and hence lead to biased results. For example, a study is designed to determine the impact of a novel chemotherapeutic agent to palliate patients with end-stage myeloma. This double-blind, randomized controlled trial (RCT) shows that patients receiving the experimental agent have less bony pain. However, after study completion and careful review, investigators discovered that the chemotherapeutic agent caused an intractable cough that could only be treated with a narcotic-containing syrup. Investigators are now unable to determine whether the reduction in bony pain was secondary to the experimental agent or to the narcotic.






Confounders



Confounders are variables or characteristics of the enrolled patient population (that may be differentially distributed between the experimental and control group) that have an influence on the target outcome. For instance, a large RCT designed to test a new immunomodulatory therapy for patients with antiphospholipid syndrome allocates, by chance, more patients with systemic lupus erythematosus (SLE) to the placebo group as compared with the treatment group. If the results demonstrate the treatment leads to a reduction in the target outcome investigators would be unsure whether this observation was due to the therapy or because patients in the treatment group were less likely to have SLE (as the presence of the disease may have made the placebo group, on average, more likely to suffer study-relevant outcomes).






Bias



Bias is a systematic error, which leads to a distortion of the results; as a result, bias and not treatment may be responsible for the observed treatment effect. Bias may occur at different points in a study and is often difficult to measure. There are many types of bias. To enumerate them all is beyond the scope of this chapter. However, a working knowledge of a few key types of bias is useful for the practicing clinician.





  1. Channeling bias: Channeling bias occurs most frequently in observational studies. This bias occurs when patients with selected baseline or time-dependent characteristics are preferentially allocated to a therapy; if this occurs the differences in baseline characteristics (and not the treatment) explain the observed difference in outcome. An example would be a new test is made available at the same time as a new treatment; when compared with historical outcomes the patients getting the new drug do better compared with historical controls. However, one cannot conclude that the new intervention is “better”—rather, the improved outcome could be due to either the drug, or to better classification of patients leading to fewer patients who do not have the disease (and who therefore cannot respond to the treatment) being allocated to the treatment in more modern studies.



  2. Detection bias: A detection bias is a bias caused by differing abilities to detect a disease or outcome. For example, rates of cardiovascular disease may appear to fall over time. Although this may be due to actual changes in disease prevalence, it may also be due to better diagnostic tests that more accurately assign patients to have, or not have, cardiovascular disease. As the number of false positive tests falls the prevalence of disease will fall; the fall is a result of better detection, not better treatments. A detection bias can lead to a channeling bias.



  3. Publication bias: There are a variety of publication biases. The most prevalent is a propensity for negative studies in general, and small negative studies in particular, to not be published. During literature review (either systematically or nonsystematically), the small missing studies may lead to a misperception that an intervention works. However, and in fact, the intervention does not work—the observed effect is due to failure to include the results of small negative studies.




A list of several biases found in and/or mitigated by randomized controlled trials can be found in Table 70-1.




Table 70-1 Biases that May Be Found and/or Mitigated by Well-Done Randomized Controlled Trials 






How Do We Minimize These Effects?



Designing, analyzing, and reporting clinical trials can be challenging due to the aforementioned issues such as cointerventions, confounders, and biases. How can we best mitigate these issues? There are a number of measures that investigators may take including: (a) randomization, (b) blinding, (c) concealed allocation, (d) uniform follow-up, (e) accurate accounting of cointerventions, and (f) full disclosure of the fate of all patients.






The Evidence-Based Medicine Process





In order to properly identify, evaluate, and apply medical literature, it is important to proceed systematically. The evidence-based medicine process includes: (1) formulating a focused clinical question, (2) finding the highest level of evidence, (3) critically appraise the evidence, and (4) apply the evidence to your patient. For the remainder of the chapter, we take you through these four critical steps to making evidence-based decisions for patient care.




Jun 13, 2016 | Posted by in CRITICAL CARE | Comments Off on The Quality of Evidence

Full access? Get Clinical Tree

Get Clinical Tree app for offline access